Key point detection method, equipment and device

Abstract

The invention relates to the technical field of ultrasonic artificial intelligence, in particular to a key point detection method, equipment and device, and aims to solve the problem of low universality of ultrasonic equipment in the prior art. The method comprises the following steps: in response to a detection operation for a target ultrasonic image triggered by a user, performing feature extraction on the obtained target ultrasonic image to obtain a feature vector of the target ultrasonic image; determining a target object in the target ultrasonic image according to the feature vectors of the plurality of set objects and the feature vector of the target ultrasonic image; determining a target key point detection model corresponding to a target object in the key point detection models corresponding to the plurality of set objects; and based on the target key point detection model, performing key point detection on the target ultrasonic image to obtain a detection result. According to the method, the ultrasonic equipment has a plurality of object detection algorithms, namely a plurality of different detection models, so that key point detection can be performed on ultrasonic images of various objects, and the universality of the ultrasonic equipment is improved.

Description

Technical Field

[0001] The present application relates to the field of ultrasonic artificial intelligence technology, and in particular to a key point detection method, equipment and device. Background Art

[0002] Ultrasonic imaging is widely used in the analysis of various diseases. Ultrasonic imaging can be performed on various biological tissues in order to analyze the characteristics of the biological tissues and detect the status of the biological tissues.

[0003] In the prior art, developers train a set of ultrasonic image automatic measurement algorithms preset in ultrasonic equipment by collecting data from limited medical institutions, and then put the trained ultrasonic equipment into use in the departments adapted to each hospital. For example, if an ultrasonic device is used to detect ultrasonic images of key points, then a set of key point ultrasonic image detection algorithms will be preset in the device. The developer will train the key point ultrasonic image detection algorithm through the collected key point images, and then put the trained device into the departments of key point detection in each hospital for use.

[0004] On the one hand, the R&D cost and cycle of the automatic ultrasound image measurement algorithm in each ultrasound device increase, and the dependence on cooperative medical resources is high. On the other hand, there is only one set of preset and maturely trained automatic ultrasound image measurement algorithms in the ultrasound device, which cannot guarantee that all departments can use the ultrasound device to perform ultrasound image detection. It has a certain degree of specificity, which makes the ultrasound device less versatile. Summary of the invention

[0005] The purpose of this application is to provide a key point detection method, equipment and device to solve the problem of low versatility of ultrasonic equipment in the prior art.

[0006] In a first aspect, the present application provides a key point detection method, which is applied to an ultrasonic device, comprising:

[0007] Acquiring a target ultrasound image;

[0008] In response to a detection operation on the target ultrasound image triggered by a user, extracting features of the target ultrasound image to obtain a feature vector of the target ultrasound image;

[0009] Determine the target object in the target ultrasound image according to the feature vectors of the multiple set objects and the feature vector of the target ultrasound image; wherein the feature vectors of the multiple set objects are obtained by clustering analysis based on the feature vectors of at least one set object included in the first ultrasound image sample set; the first ultrasound image sample set includes multiple ultrasound images; the feature vector of the target object is the feature vector with the highest similarity to the feature vector of the target ultrasound image among the feature vectors of the multiple set objects;

[0010] Determine a target key point detection model corresponding to the target object among the key point detection models corresponding to the multiple set objects respectively; wherein a first key point detection model corresponding to a first set object is obtained by training a second ultrasound image sample set corresponding to the first set object, and the first set object is any set object among the multiple set objects; and the second ultrasound image sample set includes multiple ultrasound images;

[0011] Based on the target key point detection model, key point detection is performed on the target ultrasound image to obtain a detection result.

[0012] In a possible implementation manner, the feature vectors of the multiple setting objects are obtained in the following manner:

[0013] Acquire the first ultrasound image sample set;

[0014] Performing feature extraction on each ultrasonic image included in the first ultrasonic image sample set to obtain a feature vector corresponding to each ultrasonic image included in the first ultrasonic image sample set;

[0015] Based on the similarity between the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set, cluster analysis is performed on the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set to obtain the feature vectors of the plurality of set objects.

[0016] In a possible implementation manner, the similarity between the feature vector of each set object and the feature vector of the target ultrasound image is: the cosine similarity between the feature vector of each set object and the feature vector of the target ultrasound image.

[0017] In a possible implementation manner, the similarity between the feature vector of each set object and the feature vector of the target ultrasound image is: an inner product value of the feature vector of each set object and the feature vector of the target ultrasound image.

[0018] In a possible implementation manner, the first key point detection model corresponding to the first setting object is trained according to the following method:

[0019] Acquire a second ultrasound image sample set corresponding to the first setting object; the second ultrasound image sample set includes a plurality of ultrasound images and key point annotation data corresponding to each ultrasound image; the plurality of ultrasound images in the second ultrasound image sample set all include the first setting object;

[0020] For any one ultrasound image, based on the initial first key point detection model, perform key point detection on the ultrasound image to obtain key point detection data; determine the loss value based on the key point detection data and the key point annotation data corresponding to the ultrasound image;

[0021] Based on the loss value corresponding to each ultrasonic image included in the second ultrasonic image sample set, the parameter information of the initial first key point detection model is adjusted to obtain a trained first key point detection model.

[0022] In a possible implementation, the method further includes:

[0023] Acquire a key point annotation result of the target ultrasound image by the user;

[0024] When it is determined that the detection result and the annotation result are different, adding the target ultrasound image to the second ultrasound image sample set corresponding to the target object to obtain a third ultrasound image sample set corresponding to the target object;

[0025] Based on the third ultrasound image sample set corresponding to the target object, a target key point detection model corresponding to the target object is trained.

[0026] In a possible implementation, the method further includes:

[0027] After performing key point detection on the target ultrasound image based on the target key point detection model to obtain a detection result, in response to a training operation of the target key point detection model corresponding to the target object triggered by the user, the target ultrasound image is added to a second ultrasound image sample set corresponding to the target object to obtain a fourth ultrasound image sample set corresponding to the target object;

[0028] Based on the fourth ultrasound image sample set corresponding to the target object, a target key point detection model corresponding to the target object is trained.

[0029] In a possible implementation, the method further includes:

[0030] Real-time detection of the current resource usage rate of the ultrasound device;

[0031] When the current resource usage rate of the ultrasound device is lower than the preset usage rate, the first key point detection model corresponding to the first setting object is trained based on the second ultrasound image sample set corresponding to the first setting object.

[0032] In a second aspect, an embodiment of the present application provides an ultrasound device, the device comprising an image acquisition unit, a processor, and a display screen;

[0033] The image acquisition unit is used to acquire a target ultrasound image;

[0034] The processor is configured to extract features of the target ultrasonic image in response to a detection operation triggered by a user for the target ultrasonic image, and obtain a feature vector of the target ultrasonic image; determine a target object in the target ultrasonic image according to feature vectors of a plurality of set objects and the feature vector of the target ultrasonic image; wherein the feature vectors of the plurality of set objects are obtained by clustering analysis based on feature vectors of at least one set object included in a first ultrasonic image sample set; the first ultrasonic image sample set includes a plurality of ultrasonic images; the feature vector of the target object is a feature vector with the highest similarity to the feature vector of the target ultrasonic image among the feature vectors of the plurality of set objects; determine a target key point detection model corresponding to the target object in the key point detection models respectively corresponding to the plurality of set objects; wherein the first key point detection model corresponding to the first set object is obtained by training a second ultrasonic image sample set corresponding to the first set object, and the first set object is any one of the plurality of set objects; the second ultrasonic image sample set includes a plurality of ultrasonic images; based on the target key point detection model, perform key point detection on the target ultrasonic image to obtain a detection result;

[0035] The display screen is used to display the detection result in a display interface.

[0036] In a third aspect, an embodiment of the present application provides a key point detection device, the device comprising:

[0037] An acquisition module, used for acquiring a target ultrasound image;

[0038] a feature vector determination module, configured to extract features of the target ultrasound image in response to a detection operation triggered by a user and to obtain a feature vector of the target ultrasound image;

[0039] an object determination module, configured to determine a target object in the target ultrasound image according to feature vectors of a plurality of set objects and a feature vector of the target ultrasound image; wherein the feature vectors of the plurality of set objects are obtained by clustering analysis based on feature vectors of at least one set object included in a first ultrasound image sample set; the first ultrasound image sample set includes a plurality of ultrasound images; and the feature vector of the target object is a feature vector having the highest similarity with a feature vector of the target ultrasound image among the feature vectors of the plurality of set objects;

[0040] A model determination module is used to determine a target key point detection model corresponding to the target object among the key point detection models corresponding to the multiple set objects respectively; wherein the first key point detection model corresponding to the first set object is obtained by training a second ultrasound image sample set corresponding to the first set object, the first set object is any set object among the multiple set objects; the second ultrasound image sample set includes multiple ultrasound images;

[0041] The detection module is used to perform key point detection on the target ultrasound image based on the target key point detection model to obtain a detection result.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by an electronic device, the electronic device can perform a key point detection method as described in any one of the first aspects above.

[0043] In a fifth aspect, the present application provides a computer program product, comprising a computer program: when the computer program is executed by a processor, the key point detection method as described in any one of the above-mentioned first aspects is implemented.

[0044] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0045] In the embodiment of the present application, in response to the detection operation for the target ultrasonic image triggered by the user, the acquired target ultrasonic image is feature extracted to obtain the feature vector of the target ultrasonic image; the target object in the target ultrasonic image is determined according to the feature vectors of multiple set objects and the feature vector of the target ultrasonic image; the target key point detection model corresponding to the target object in the key point detection models corresponding to the multiple set objects is determined; based on the target key point detection model, the key point detection is performed on the target ultrasonic image to obtain the detection result. Thus, the target key point detection model corresponding to the target ultrasonic image in the multiple key point detection models stored in the device is determined according to the feature vector of the target ultrasonic image, and then the target key point detection model is used to detect the target ultrasonic image. In this process, the ultrasonic device has detection algorithms for multiple objects, that is, multiple different detection models, so that key point detection can be performed on ultrasonic images of various objects, which improves the versatility of the ultrasonic device and enhances the user experience.

[0046] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 This is a structural block diagram of an ultrasonic device according to an embodiment of the present application;

[0049] Figure 2 This is a schematic diagram of the principle of realizing ultrasonic imaging by an ultrasonic device according to an embodiment of the present application;

[0050] Figure 3 This is an application scenario diagram of a key point detection method according to an embodiment of the present application;

[0051] Figure 4 A schematic diagram of a key point detection method provided in an embodiment of the present application;

[0052] Figure 5 A structural diagram of the training process of a key point detection method provided in an embodiment of the present application;

[0053] Figure 6A schematic flow chart of a method for determining feature vectors of multiple setting objects provided in an embodiment of the present application;

[0054] Figure 7 A schematic diagram of the structure of an identification module provided in an embodiment of the present application;

[0055] Figure 8 A flowchart of a method for training a first key point detection model corresponding to a first setting object provided in an embodiment of the present application;

[0056] Fig. 9 A structural diagram of an acquisition module provided in an embodiment of the present application;

[0057] Fig.10 A schematic diagram of a model provided for an embodiment of the present application;

[0058] Fig.11 A schematic diagram of a multi-task model provided in an embodiment of the present application;

[0059] Fig.12 A flowchart of a method for determining key point detection data provided in an embodiment of the present application;

[0060] Fig.13 A structural diagram of a generation module provided in an embodiment of the present application;

[0061] Fig.14 A structural diagram of a determination module provided in an embodiment of the present application;

[0062] Fig.15 A flowchart of a key point detection model training process provided in an embodiment of the present application;

[0063] Fig.16 A training structure diagram of a key point detection model provided in an embodiment of the present application;

[0064] Fig.17 A flowchart of a key point detection model training process provided in an embodiment of the present application;

[0065] Fig.18 A flowchart of a key point detection model training process provided in an embodiment of the present application;

[0066] Fig.19 A schematic diagram of the structure of an ultrasonic device provided in an embodiment of the present application;

[0067] Fig. 20 A schematic diagram of the structure of a key point detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Among them, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0069] Furthermore, in the description of the embodiments of the present application, unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0070] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more.

[0071] Ultrasonic imaging is widely used in the analysis of various diseases. Ultrasonic imaging can be performed on various biological tissues in order to analyze the characteristics of the biological tissues and detect the status of the biological tissues.

[0072] In the prior art, developers train a set of ultrasonic image automatic measurement algorithms preset in ultrasonic equipment by collecting data from limited medical institutions, and then put the trained ultrasonic equipment into use in the departments adapted to each hospital. For example, if an ultrasonic device is used to detect ultrasonic images of key points, then a set of key point ultrasonic image detection algorithms will be preset in the device. The developer will train the key point ultrasonic image detection algorithm through the collected key point images, and then put the trained device into the departments of key point detection in each hospital for use.

[0073] On the one hand, the R&D cost and cycle of the automatic ultrasound image measurement algorithm in each ultrasound device increase, and the dependence on cooperative medical resources is high. On the other hand, there is only one set of preset and maturely trained automatic ultrasound image measurement algorithms in the ultrasound device, which cannot guarantee that all departments can use the ultrasound device to perform ultrasound image detection. It has a certain degree of specificity, which makes the ultrasound device less versatile.

[0074] In view of this, the present application provides a key point detection method, equipment and device to solve the problem of low versatility of ultrasonic equipment in the prior art.

[0075] The inventive concept of the present application can be summarized as follows: in the embodiment of the present application, in response to the detection operation for the target ultrasonic image triggered by the user, the acquired target ultrasonic image is feature extracted to obtain the feature vector of the target ultrasonic image; the target object in the target ultrasonic image is determined according to the feature vectors of multiple set objects and the feature vector of the target ultrasonic image; the target key point detection model corresponding to the target object in the key point detection models corresponding to the multiple set objects is determined; based on the target key point detection model, the target ultrasonic image is subjected to key point detection to obtain the detection result. Thus, the target key point detection model corresponding to the target ultrasonic image in the multiple key point detection models stored in the device is determined according to the feature vector of the target ultrasonic image, and then the target key point detection model is used to detect the target ultrasonic image. In this process, the ultrasonic device has detection algorithms for multiple objects, that is, multiple different detection models, so that key point detection can be performed on ultrasonic images of various objects, thereby improving the versatility of the ultrasonic device and the user experience.

[0076] After introducing the main inventive concept of the embodiment of the present application, the ultrasonic device used in the key point detection method provided in the embodiment of the present application is introduced in conjunction with the accompanying drawings. Figure 1 As shown, it is a structural block diagram of an ultrasonic device provided in an embodiment of the present application.

[0077] It should be understood that Figure 1 The ultrasound device shown is only an example and the ultrasound device may have more Figure 1 The more or less components shown in the figure can be combined with two or more components, or can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0078] Figure 1 Schematically shows a hardware configuration block diagram of an ultrasound device according to an exemplary embodiment.

[0079] like Figure 1 As shown, the ultrasound device may include, for example: a processor 110, a memory 120, a display unit 130 and a probe 140; wherein:

[0080] A probe 140, used for transmitting an ultrasonic signal;

[0081] A display unit 130, configured to display an ultrasound image;

[0082] The memory 120 is configured to store data required for ultrasound imaging, which may include software programs, application interface data, etc.;

[0083] The processor 110 is connected to the probe 140, the display unit 130 and the memory 120 respectively, and is configured to execute: acquiring a target ultrasound image; in response to a detection operation for the target ultrasound image triggered by a user, extracting features of the target ultrasound image to obtain a feature vector of the target ultrasound image; determining a target object in the target ultrasound image according to feature vectors of multiple set objects and feature vectors of the target ultrasound image; wherein the feature vectors of the multiple set objects are obtained by clustering analysis based on feature vectors of at least one set object included in a first ultrasound image sample set; the first ultrasound image sample set includes multiple ultrasound images; the feature vector of the target object is a feature vector with the highest similarity to the feature vector of the target ultrasound image among the feature vectors of the multiple set objects; determining a target key point detection model corresponding to the target object in key point detection models corresponding to the multiple set objects respectively; wherein the first key point detection model corresponding to the first set object is obtained by training a second ultrasound image sample set corresponding to the first set object, and the first set object is any set object among the multiple set objects; the second ultrasound image sample set includes multiple ultrasound images; based on the target key point detection model, performing key point detection on the target ultrasound image to obtain a detection result.

[0084] The display unit 130 displays the detection result obtained after performing key point detection on the target ultrasound image.

[0085] Figure 2 Schematic diagram of the application principle according to an embodiment of the present application. Figure 1 The partial modules or functional components of the ultrasonic device shown are implemented, and only the main components will be described below, while other components, such as memory, controller, control circuit, etc., will not be described here in detail.

[0086] like Figure 2 As shown, the application environment may include a user interface 210 , a display unit 220 for displaying the user interface, and a processor 230 .

[0087] The display unit 220 may include a display panel 221 and a backlight assembly 222. The display panel 221 is configured to display an ultrasound image, the backlight assembly 222 is located behind the display panel 221, and the backlight assembly 222 may include a plurality of backlight partitions (not shown in the figure), each of which may emit light to illuminate the display panel 221.

[0088] The processor 230 may be configured to control the brightness of the backlight source of each backlight partition in the backlight assembly 222 , and to control the probe to transmit an ultrasonic signal, and to receive and analyze an ultrasonic echo signal to obtain an ultrasonic image.

[0089] Among them, the processor 230 can perform feature extraction on the target ultrasound image to obtain a feature vector of the target ultrasound image; then determine the target object in the target ultrasound image based on the feature vectors of multiple set objects and the feature vector of the target ultrasound image, thereby determining a target key point detection model corresponding to the target object in the key point detection models corresponding to the multiple set objects respectively; then, based on the target key point detection model, perform key point detection on the target ultrasound image to obtain a detection result.

[0090] like Figure 3 FIG. 1 is an application scenario diagram of a key point detection method provided by an embodiment of the present application. The diagram includes: a user 10, an ultrasonic device 20, and a memory 30;

[0091] The user 10 uses the probe of the ultrasound device 20 to collect a target ultrasound image corresponding to the biological tissue, and stores the collected target ultrasound image in the memory 30 .

[0092] When the ultrasound device 20 processes the target ultrasound image, it first obtains the target ultrasound image from the memory 30 through the network, and then, in response to the detection operation for the target ultrasound image triggered by the user 10 in the display interface of the ultrasound device 20, performs feature extraction on the target ultrasound image to obtain a feature vector of the target ultrasound image; then, it obtains feature vectors of multiple set objects from the memory 30 through the network, and determines the target object in the target ultrasound image according to the feature vectors of the multiple set objects and the feature vector of the target ultrasound image; and obtains key point detection models corresponding to the multiple set objects from the memory 30 through the network, and determines the target key point detection model corresponding to the target object in the key point detection models corresponding to the multiple set objects; based on the target key point detection model, performs key point detection on the target ultrasound image to obtain a detection result;

[0093] Finally, the detection result is displayed in the display interface of the ultrasound device 20 and presented to the user 10, so that the user 10 can analyze the biological tissue according to the detection result to detect the state of the biological tissue.

[0094] In the description of the present application, only a single ultrasound device and a memory are described in detail, but it should be understood by those skilled in the art that the ultrasound device 20 and the memory 30 shown are intended to indicate that the technical solution of the present application involves the operation of the ultrasound device and the memory. The description of a single ultrasound device and a memory is at least for the convenience of explanation, and does not imply a limitation on the number, type, or location of the ultrasound device and the memory. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed from it, the underlying concept of the exemplary embodiment of the present application will not be changed.

[0095] It should be noted that the memory 30 in the embodiment of the present application may be, for example, a cache system, a hard disk storage, a memory storage, etc. In addition, the key point detection method proposed in the present application is not only applicable to Figure 3 The application scenario shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto.

[0096] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0097] In order to improve the versatility of ultrasonic equipment, the present application embodiment provides a key point detection method. Figure 4 FIG. 1 is a flow chart of a key point detection method, which includes the following steps:

[0098] In step 401, a target ultrasound image is acquired.

[0099] In step 402, in response to a detection operation on a target ultrasound image triggered by a user, feature extraction is performed on the target ultrasound image to obtain a feature vector of the target ultrasound image.

[0100] In step 403, a target object in the target ultrasound image is determined according to the feature vectors of the plurality of set objects and the feature vector of the target ultrasound image.

[0101] The feature vectors of the multiple set objects in the embodiment of the present application are obtained by clustering analysis based on the feature vectors of at least one set object included in the first ultrasound image sample set. The first ultrasound image sample set includes multiple ultrasound images; the feature vector of the target object is the feature vector with the highest similarity to the feature vector of the target ultrasound image among the feature vectors of the multiple set objects.

[0102] In step 404, a target key point detection model corresponding to the target object among the key point detection models corresponding to the plurality of set objects is determined.

[0103] Among them, the first key point detection model corresponding to the first setting object in the embodiment of the present application is obtained by training the second ultrasound image sample set corresponding to the first setting object; wherein the first setting object is any setting object among multiple setting objects; wherein the second ultrasound image sample set includes multiple ultrasound images.

[0104] In step 405, based on the target key point detection model, key point detection is performed on the target ultrasound image to obtain a detection result.

[0105] For ease of understanding, the following is a detailed description of the training process of the key point detection model and the process of performing key point detection on the target ultrasound image based on the key point detection model:

[0106] 1. Training process of key point detection model

[0107] The training process of the key point detection method provided in the embodiment of the present application includes the following steps: Figure 5 The two parts shown are as follows: the dotted box is the first part: determining the feature vectors of multiple set objects and multiple set objects based on the multiple ultrasound images included in the first ultrasound image sample set. The part outside the dotted box is the second part: training the corresponding key point detection model based on the second ultrasound image sample set corresponding to each set object.

[0108] The second ultrasound image sample set is a subset of the first ultrasound image sample set.

[0109] In a possible implementation, in the embodiment of the present application, determining feature vectors of multiple set objects based on multiple ultrasound images included in the first ultrasound image sample set, and multiple set objects can be specifically implemented as follows: Figure 6 Steps shown:

[0110] In step 601, a first ultrasound image sample set is acquired.

[0111] In step 602, feature extraction is performed on each ultrasonic image included in the first ultrasonic image sample set to obtain a feature vector corresponding to each ultrasonic image included in the first ultrasonic image sample set.

[0112] In step 603, based on the similarity between the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set, cluster analysis is performed on the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set to obtain feature vectors of a plurality of set objects.

[0113] Among them, the similarity between the feature vectors corresponding to each ultrasonic image included in the first ultrasonic image sample set in the embodiment of the present application is: the cosine similarity between the feature vectors corresponding to each ultrasonic image, or the inner product value between the feature vectors corresponding to each ultrasonic image.

[0114] When implementing it, Figure 7 As shown, it is a structural diagram of the feature vectors of multiple set objects and the recognition module of multiple set objects in this application. The core of the recognition module is a deep convolutional neural network encoder that pre-trains a large-scale ultrasound data set, that is, a deep convolutional neural network encoding process is performed on multiple ultrasound images included in the first ultrasound image sample set. Its function is to convert the input ultrasound image into a feature vector of the embedding space through the image encoder. After a period of use, each ultrasound device will accumulate a large number of ultrasound images. After encoding these ultrasound images, the feature vectors of their respective embedding spaces are extracted to construct a vector gallery. Then, according to the cosine similarity or inner product value between the feature vectors of each ultrasound image, the similarity between the feature vectors of each ultrasound image is obtained; and according to the similarity between the feature vectors of each ultrasound image, these feature vectors are clustered to obtain the feature vectors of multiple set objects included in these ultrasound images, and multiple set objects, that is, the common standard sections obtained by the ultrasound device after a period of use.

[0115] For example, the same type of ultrasound data will naturally have different vector libraries depending on the hospital / department where it is located. For example, if device A is in an obstetric ultrasound room, it will accumulate 10,000 fetal ultrasound images, and after clustering, it will obtain the "feature vectors" corresponding to the set objects (standard sections) such as fetal head circumference, abdominal circumference, and femur. If device B is in a physical examination department, it will accumulate 10,000 superficial ultrasound images, and after clustering, it will obtain the "feature vectors" corresponding to the set objects (standard sections) such as thyroid and breast.

[0116] Therefore, according to Figure 6 The steps shown can determine the feature vectors of multiple set objects based on multiple ultrasound images included in the first ultrasound image sample set.

[0117] In a possible implementation, the number of key point detection models that need to be trained in the embodiment of the present application is the same as the number of feature vectors of multiple set objects determined based on the multiple ultrasound images included in the first ultrasound image sample set. Figure 6 The feature vectors of multiple set objects obtained in the above method are used to determine the number of key point detection models that need to be trained.

[0118] For example, assuming that the first ultrasound image sample set includes 100 ultrasound images, according to Figure 6The steps shown determine that the 100 ultrasound images include feature vectors of the three set objects, and thus it can be determined that three key point detection models need to be trained next, that is, the three set objects correspond to one key point detection model respectively.

[0119] In a possible implementation manner, the first setting object in the embodiment of the present application is any setting object among multiple setting objects, and the training of the first key point detection model corresponding to the first setting object can be specifically performed as follows: Figure 8 Steps shown:

[0120] In step 801, a second ultrasonic image sample set corresponding to a first set object is obtained; the second ultrasonic image sample set includes a plurality of ultrasonic images and key point annotation data corresponding to each ultrasonic image; and the plurality of ultrasonic images in the second ultrasonic image sample set all include the first set object.

[0121] Among them, Fig. 9 As shown in the figure, it is a structural diagram of the module for acquiring key point annotation data corresponding to the ultrasound image in this application. Fig. 9 After clicking the Freeze button in the display interface of the ultrasound device on the left, you will get the following Fig. 9 The target ultrasound image is shown on the right. At the same time, the user will mark multiple key points in the display interface of the ultrasound device. In response to the user's marking operation, this module will obtain the key point marking data corresponding to the target ultrasound image.

[0122] In step 802, for any ultrasound image, key point detection is performed on the ultrasound image based on the initial first key point detection model to obtain key point detection data; based on the key point detection data and the key point annotation data corresponding to the ultrasound image, the loss value is determined.

[0123] In step 803, based on the loss value corresponding to each ultrasonic image included in the second ultrasonic image sample set, the parameter information of the initial first key point detection model is adjusted to obtain a trained first key point detection model.

[0124] In specific implementation, a multi-task deep learning model structure that can be used for semantic segmentation / key point detection, such as UNet, PSPNet, etc., will be preset in the ultrasound device, which will be deployed in the ultrasound device after pre-training on a large-scale ultrasound dataset. Based on the recognition results of the feature vectors of multiple set objects, models with the same structure and different parameters are trained for multiple set objects.

[0125] like Fig.10As shown in the figure, an initial model is preset in a certain ultrasound device. After one month of use and recognition of feature vectors of multiple set objects, feature vectors of two set objects are obtained: thyroid and bladder. The system then generates two key point detection models with the same UNet structure, namely thyroid nodule model and bladder model. Then, the system uses their respective training data (the second ultrasound image sample set) to learn the segmentation and detection of thyroid nodules and bladder respectively, forming different models. When the system obtains the target ultrasound image, it determines the corresponding detection model according to the recognition module to perform key point detection.

[0126] Among them, the initial model preset in the embodiment of the present application can be set according to the empirical value, for example, it can be a deep learning neural network, a coding model, or other models that can detect key points. The present application does not impose any restrictions on this.

[0127] In the implementation method, in order to further improve the intelligence of the system, the recognition module can be combined with the semantic segmentation / key point detection model to obtain the following Fig.11 The end-to-end multi-task model shown in FIG. In this model, the recognition of feature vectors of multiple set objects, the determination of the number of key point detection models, and the training process of the key point detection model for each set object are combined, but Fig.10 The functions of the models are the same and will not be described here.

[0128] In a possible implementation, the key point annotation data corresponding to each ultrasound image in the second ultrasound image sample set in the embodiment of the present application is a plurality of reference key points corresponding to each ultrasound image annotated by the user. Therefore, based on the initial first key point detection model, key point detection is performed on the ultrasound image. When the key point detection data is obtained, the following can also be performed for each ultrasound image: Fig.12 Steps shown:

[0129] In step 1201, edge detection is performed on the ultrasonic image to obtain first edge feature data of the detection object in the ultrasonic image; the first edge feature data of the detection object is filled to obtain second edge feature data of the detection object in the ultrasonic image; the second edge feature data includes a plurality of reference key points;

[0130] In step 1202, a plurality of reference key points of the ultrasound image are normally distributed within a preset range to obtain key point feature data of the ultrasound image;

[0131] In step 1203, the second edge feature data of the detection object in the ultrasonic image and the key point feature data of the ultrasonic image are used as feature data of the ultrasonic image;

[0132] In step 1204, the feature data of the ultrasound image is input into an initial first key point detection model, and key point detection is performed on the ultrasound image to obtain key point detection data.

[0133] Among them, the preset range can be set according to empirical values, and this application does not impose any restrictions on this.

[0134] When implementing it, Fig.13 As shown, it is a structural diagram of the generation module of the characteristic data of the ultrasonic image in this application. The ultrasonic image and multiple reference key points corresponding to the ultrasonic image annotated by the user are input into the generation module, the edge detection of the ultrasonic image is performed, and the closed edges passing through the vicinity of multiple reference key points are retained, and the regional semantic segmentation annotation of the ultrasonic image is formed after filling, that is, the second edge characteristic data of the detection object in the ultrasonic image; then, with each reference key point as the center, a normal distribution of a preset range is generated to obtain the thermal map annotation of the key point, that is, the key point characteristic data of the ultrasonic image. In this way, the characteristic data of the ultrasonic image input into the initial first key point detection model is obtained.

[0135] In a possible implementation, in order to determine the timing of training in the embodiment of the present application, the following can be performed: real-time detection of the current resource utilization rate of the ultrasound device; when the current resource utilization rate of the ultrasound device is lower than the preset utilization rate, training the first key point detection model corresponding to the first set object based on the second ultrasound image sample set corresponding to the first set object.

[0136] In a possible implementation, in order to determine the timing of training in the embodiment of the present application, it can also be performed as follows: real-time detection of the current time point; if the current time point is within a preset time period, training the first key point detection model corresponding to the first set object based on the second ultrasound image sample set corresponding to the first set object.

[0137] Among them, in the embodiment of the present application, the preset usage rate and the preset time period can be set according to experience values, and the present application does not impose any restrictions on this.

[0138] When implementing it, Fig.14 , which is a structural diagram of a module for determining training opportunities in an embodiment of the present application. Fig.14 In the process, the determination module performs resource monitoring tasks and time monitoring tasks. If it is detected that the current resource utilization rate of the ultrasound device is lower than the preset utilization rate and / or the current time point is within the preset time period, a control signal is sent to the training module so that the training module trains the first key point detection model corresponding to the first set object based on the second ultrasound image sample set corresponding to the first set object.

[0139] For ease of understanding, the following Fig.15The steps shown in the figure illustrate the complete process of training the key point detection model:

[0140] In step 1501, a first ultrasound image sample set is acquired.

[0141] In step 1502, feature extraction is performed on each ultrasonic image included in the first ultrasonic image sample set to obtain a feature vector corresponding to each ultrasonic image included in the first ultrasonic image sample set.

[0142] In step 1503, based on the similarity between the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set, cluster analysis is performed on the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set to obtain feature vectors of a plurality of set objects.

[0143] In step 1504 , the number of key point detection models is determined based on the feature vectors of the plurality of set objects.

[0144] In step 1505, for each set object's key point detection model: obtain a plurality of ultrasound images included in a second ultrasound image sample set corresponding to the set object and key point annotation data corresponding to each ultrasound image.

[0145] In step 1506, each ultrasound image in the second ultrasound image sample set is detected to obtain feature data of the ultrasound image.

[0146] In step 1507, if it is detected that the current resource usage rate of the ultrasound device is lower than the preset usage rate and\or the current time point is within the preset time period, the key point detection model corresponding to the set object is trained based on the second ultrasound image sample set corresponding to the set object.

[0147] When implementing it, Fig.16 As shown in FIG. 1 , it is a training structure diagram of the key point detection model in the embodiment of the present application. Fig.16In the method, first, the user clicks the OK (Freeze) button in the display interface of the ultrasound device, and marks multiple key points for each ultrasound image in the display interface of the ultrasound device, and then the acquisition module responds to the user's annotation operation to obtain multiple ultrasound images and key point annotation data corresponding to each ultrasound image. After that, the recognition module converts these ultrasound images into feature vectors of the embedding space by the image encoder, performs cluster analysis, and obtains feature vectors of multiple set objects; the training module determines the number of key point detection models based on the feature vectors of multiple set objects. At the same time, the generation module detects each ultrasound image to obtain feature data of the ultrasound image; when the determination module detects that the current resource utilization rate of the ultrasound device is lower than the preset utilization rate and\or the current time point is within the preset time period, a control signal is sent to the training module; after the training module receives the control signal, the key point detection model of each set object obtains the feature data of multiple ultrasound images included in the second ultrasound image sample set corresponding to the set object output by the generation module, and trains the key point detection model corresponding to the set object according to the feature data of multiple ultrasound images included in the second ultrasound image sample set.

[0148] 2. The process of performing key point detection on the target ultrasound image based on the key point detection model

[0149] After obtaining the feature vectors of multiple set objects and the key point detection model corresponding to each set object according to the training process of the key point detection model, a target ultrasound image is obtained in response to the user's input operation on the display interface of the ultrasound device; then, in response to the detection operation for the target ultrasound image triggered by the user, feature extraction is performed on the target ultrasound image to obtain the feature vector of the target ultrasound image. According to the similarity between the feature vector of each set object obtained through training and the feature vector of the target ultrasound image, a cluster analysis is performed to determine the target object in the target ultrasound image. Then, according to the key point detection models corresponding to the multiple set objects obtained through training, the target key point detection model corresponding to the target object is determined. Finally, based on the target key point detection model, key point detection is performed on the target ultrasound image to obtain the detection result, i.e., the position information of each key point in the target ultrasound image.

[0150] Among them, for each set object: the similarity between the feature vector of the set object and the feature vector of the target ultrasound image is: the cosine similarity between the feature vector of the set object and the feature vector of the target ultrasound image, or the inner product value of the feature vector of the set object and the feature vector of the target ultrasound image.

[0151] The higher the cosine similarity, the higher the similarity between the feature vector of the set object and the feature vector of the target ultrasound image, and the higher the possibility that the target object of the target ultrasound image is the set object.

[0152] Similarly, the higher the inner product value is, the higher the similarity between the feature vector of the set object and the feature vector of the target ultrasound image is, and the higher the possibility that the target object of the target ultrasound image is the set object is.

[0153] In a possible implementation, in order to improve the stability of the algorithm in the ultrasound device and the accuracy of key point detection, the method in the embodiment of the present application can also be performed as follows: Fig.17 Steps shown:

[0154] In step 1701, the key point annotation result of the target ultrasound image is obtained by the user;

[0155] In step 1702, when it is determined that the detection result and the annotation result are different, the target ultrasound image is added to the second ultrasound image sample set corresponding to the target object to obtain a third ultrasound image sample set corresponding to the target object;

[0156] In step 1703, a target key point detection model corresponding to the target object is trained based on the third ultrasound image sample set corresponding to the target object.

[0157] In a possible implementation, in order to improve the stability of the algorithm in the ultrasound device and the accuracy of key point detection, the method in the embodiment of the present application can also be performed as follows: Fig.18 Steps shown:

[0158] In step 1801, after key point detection is performed on the target ultrasound image based on the target key point detection model and the detection result is obtained, in response to a training operation of the target key point detection model corresponding to the target object triggered by the user, the target ultrasound image is added to the second ultrasound image sample set corresponding to the target object, so as to obtain a fourth ultrasound image sample set corresponding to the target object;

[0159] In step 1802, a target key point detection model corresponding to the target object is trained based on a fourth ultrasound image sample set corresponding to the target object.

[0160] Therefore, according to Fig.17 or Fig.18 The method shown can continuously increase the training samples of the key point detection model during the application process, and train the key point detection model based on the new training samples during the application process. It can continue to learn during use, ensuring the stability of the algorithm in the ultrasonic equipment and the accuracy of the key point detection.

[0161] Based on the foregoing description, in an embodiment of the present application, in response to a detection operation on a target ultrasonic image triggered by a user, feature extraction is performed on the acquired target ultrasonic image to obtain a feature vector of the target ultrasonic image; based on the feature vectors of multiple set objects and the feature vector of the target ultrasonic image, a target object in the target ultrasonic image is determined; a target key point detection model corresponding to the target object in the key point detection models corresponding to the multiple set objects is determined; based on the target key point detection model, key point detection is performed on the target ultrasonic image to obtain a detection result.

[0162] Therefore, the target key point detection model corresponding to the target ultrasonic image in the multiple key point detection models stored in the device is determined according to the feature vector of the target ultrasonic image, and then the target key point detection model is used to detect the target ultrasonic image. In this process, the ultrasonic device has detection algorithms for multiple objects, that is, multiple different detection models, so that key point detection can be performed on ultrasonic images of various objects. Moreover, the method can continue to learn during use and can adapt to the operator habits of each device, thereby improving the versatility of the ultrasonic equipment, reducing R&D costs, shortening the R&D cycle, and improving user experience.

[0163] Refer to the following Fig.19 1900 according to this embodiment of the present application will be described. Fig.19 The ultrasound device 1900 is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0164] like Fig.19 As shown, the components of the ultrasound device 1900 may include, but are not limited to: an image acquisition unit 1901, at least one processor 1902, a display screen 1903, at least one memory 1904, and a bus 1905 connecting different system components.

[0165] Bus 1905 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus structures. Memory 1904 may include readable media in the form of volatile memory, such as random access memory (RAM) or cache memory, and may further include read-only memory (ROM).

[0166] Memory 1904 may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0167] The ultrasound device 1900 may also communicate with one or more external devices (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the ultrasound device 1900, or may communicate with any device that enables the ultrasound device 1900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface. Furthermore, the ultrasound device 1900 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), or a public network, such as the Internet) via a network adapter.

[0168] An image acquisition unit 1901 is used to acquire a target ultrasound image;

[0169] Processor 1902 is used to extract features of the target ultrasonic image in response to a detection operation triggered by a user for the target ultrasonic image, and obtain a feature vector of the target ultrasonic image; determine a target object in the target ultrasonic image according to feature vectors of multiple set objects and the feature vector of the target ultrasonic image; wherein the feature vectors of the multiple set objects are obtained by clustering analysis based on feature vectors of at least one set object included in a first ultrasonic image sample set; the first ultrasonic image sample set includes multiple ultrasonic images; the feature vector of the target object is a feature vector with the highest similarity to the feature vector of the target ultrasonic image among the feature vectors of the multiple set objects; determine a target key point detection model corresponding to the target object in the key point detection models respectively corresponding to the multiple set objects; wherein the first key point detection model corresponding to the first set object is obtained by training a second ultrasonic image sample set corresponding to the first set object, and the first set object is any one of the multiple set objects; the second ultrasonic image sample set includes multiple ultrasonic images; based on the target key point detection model, perform key point detection on the target ultrasonic image to obtain a detection result;

[0170] Display screen 1903 is used to display the detection results in a display interface.

[0171] In a possible implementation, the processor 1902 is further configured to:

[0172] Acquire the first ultrasound image sample set;

[0173] Performing feature extraction on each ultrasonic image included in the first ultrasonic image sample set to obtain a feature vector corresponding to each ultrasonic image included in the first ultrasonic image sample set;

[0174] Based on the similarity between the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set, cluster analysis is performed on the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set to obtain the feature vectors of the plurality of set objects.

[0175] In a possible implementation manner, the processor 1902 is further configured to train a first key point detection model corresponding to the first setting object according to the following method:

[0176] Acquire a second ultrasound image sample set corresponding to the first setting object; the second ultrasound image sample set includes a plurality of ultrasound images and key point annotation data corresponding to each ultrasound image; the plurality of ultrasound images in the second ultrasound image sample set all include the first setting object;

[0177] For any one ultrasound image, based on the initial first key point detection model, perform key point detection on the ultrasound image to obtain key point detection data; determine the loss value based on the key point detection data and the key point annotation data corresponding to the ultrasound image;

[0178] Based on the loss value corresponding to each ultrasonic image included in the second ultrasonic image sample set, the parameter information of the initial first key point detection model is adjusted to obtain a trained first key point detection model.

[0179] In a possible implementation, the processor 1902 is further configured to:

[0180] Acquire a key point annotation result of the target ultrasound image by the user;

[0181] When it is determined that the detection result and the annotation result are different, adding the target ultrasound image to the second ultrasound image sample set corresponding to the target object to obtain a third ultrasound image sample set corresponding to the target object;

[0182] Based on the third ultrasound image sample set corresponding to the target object, a target key point detection model corresponding to the target object is trained.

[0183] In a possible implementation, the processor 1902 is further configured to:

[0184] After performing key point detection on the target ultrasound image based on the target key point detection model to obtain a detection result, in response to a training operation of the target key point detection model corresponding to the target object triggered by the user, the target ultrasound image is added to a second ultrasound image sample set corresponding to the target object to obtain a fourth ultrasound image sample set corresponding to the target object;

[0185] Based on the fourth ultrasound image sample set corresponding to the target object, a target key point detection model corresponding to the target object is trained.

[0186] In a possible implementation, the processor 1902 is further configured to:

[0187] Real-time detection of the current resource usage rate of the ultrasound device;

[0188] When the current resource usage rate of the ultrasound device is lower than the preset usage rate, the first key point detection model corresponding to the first setting object is trained based on the second ultrasound image sample set corresponding to the first setting object.

[0189] like Fig. 20 As shown, the embodiment of the present application provides a key point detection device 2000, including:

[0190] Acquisition module 2001, used to acquire target ultrasound images;

[0191] A feature vector determination module 2002 is used to extract features of the target ultrasound image in response to a detection operation on the target ultrasound image triggered by a user, so as to obtain a feature vector of the target ultrasound image;

[0192] The object determination module 2003 is used to determine the target object in the target ultrasound image according to the feature vectors of multiple set objects and the feature vector of the target ultrasound image; wherein the feature vectors of the multiple set objects are obtained by clustering analysis based on the feature vectors of at least one set object included in the first ultrasound image sample set; the first ultrasound image sample set includes multiple ultrasound images; the feature vector of the target object is the feature vector with the highest similarity to the feature vector of the target ultrasound image among the feature vectors of the multiple set objects;

[0193] The model determination module 2004 is used to determine a target key point detection model corresponding to the target object among the key point detection models corresponding to the multiple set objects respectively; wherein the first key point detection model corresponding to the first set object is obtained by training a second ultrasound image sample set corresponding to the first set object, the first set object is any set object among the multiple set objects; the second ultrasound image sample set includes multiple ultrasound images;

[0194] The detection module 2005 is used to perform key point detection on the target ultrasound image based on the target key point detection model to obtain a detection result.

[0195] In a possible implementation manner, the key point detection device 2000 further includes a feature vector training module, which is used to:

[0196] Acquire the first ultrasound image sample set;

[0197] Performing feature extraction on each ultrasonic image included in the first ultrasonic image sample set to obtain a feature vector corresponding to each ultrasonic image included in the first ultrasonic image sample set;

[0198] Based on the similarity between the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set, cluster analysis is performed on the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set to obtain the feature vectors of the plurality of set objects.

[0199] In a possible implementation manner, the key point detection device 2000 further includes a model training module, which is used to train the first key point detection model corresponding to the first setting object according to the following method:

[0200] Acquire a second ultrasound image sample set corresponding to the first setting object; the second ultrasound image sample set includes a plurality of ultrasound images and key point annotation data corresponding to each ultrasound image; the plurality of ultrasound images in the second ultrasound image sample set all include the first setting object;

[0201] For any one ultrasound image, based on the initial first key point detection model, perform key point detection on the ultrasound image to obtain key point detection data; determine the loss value based on the key point detection data and the key point annotation data corresponding to the ultrasound image;

[0202] Based on the loss value corresponding to each ultrasonic image included in the second ultrasonic image sample set, the parameter information of the initial first key point detection model is adjusted to obtain a trained first key point detection model.

[0203] In a possible implementation, the model training module is further used to:

[0204] Acquire a key point annotation result of the target ultrasound image by the user;

[0205] When it is determined that the detection result and the annotation result are different, adding the target ultrasound image to the second ultrasound image sample set corresponding to the target object to obtain a third ultrasound image sample set corresponding to the target object;

[0206] Based on the third ultrasound image sample set corresponding to the target object, a target key point detection model corresponding to the target object is trained.

[0207] In a possible implementation, the model training module is further used to:

[0208] After performing key point detection on the target ultrasound image based on the target key point detection model to obtain a detection result, in response to a training operation of the target key point detection model corresponding to the target object triggered by the user, the target ultrasound image is added to a second ultrasound image sample set corresponding to the target object to obtain a fourth ultrasound image sample set corresponding to the target object;

[0209] Based on the fourth ultrasound image sample set corresponding to the target object, a target key point detection model corresponding to the target object is trained.

[0210] In a possible implementation, the model training module is further used to:

[0211] Real-time detection of the current resource usage rate of the ultrasound device;

[0212] When the current resource usage rate of the ultrasound device is lower than the preset usage rate, the first key point detection model corresponding to the first setting object is trained based on the second ultrasound image sample set corresponding to the first setting object.

[0213] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor to complete the above key point detection method. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0214] In an exemplary embodiment, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, any key point detection method provided in the present application is implemented.

[0215] In an exemplary embodiment, various aspects of a key point detection method provided by the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the key point detection method according to various exemplary embodiments of the present application described above in this specification.

[0216] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0217] The program product for the key point detection method of the embodiment of the present application can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on an electronic device. However, the program product of the present application is not limited to this. In this document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0218] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0219] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0220] The program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "like" language or similar programming languages. The program code may be executed entirely on the user electronic device, partially on the user device, as an independent software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or a server. In cases involving remote electronic devices, the remote electronic device may be connected to the user electronic device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0221] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0222] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

Claims

1. A key point detection method, characterized in that: The method is applied to an ultrasonic device, comprising: Acquiring a target ultrasound image; In response to a detection operation on the target ultrasound image triggered by a user, extracting features of the target ultrasound image to obtain a feature vector of the target ultrasound image; Determine the target object in the target ultrasound image according to the feature vectors of the multiple set objects and the feature vector of the target ultrasound image; wherein the feature vectors of the multiple set objects are obtained by clustering analysis based on the feature vectors of at least one set object included in the first ultrasound image sample set; the first ultrasound image sample set includes multiple ultrasound images; the feature vector of the target object is the feature vector with the highest similarity to the feature vector of the target ultrasound image among the feature vectors of the multiple set objects; Determine a target key point detection model corresponding to the target object among the key point detection models corresponding to the multiple set objects respectively; wherein a first key point detection model corresponding to a first set object is obtained by training a second ultrasound image sample set corresponding to the first set object, and the first set object is any set object among the multiple set objects; and the second ultrasound image sample set includes multiple ultrasound images; Based on the target key point detection model, key point detection is performed on the target ultrasound image to obtain a detection result.

2. The method according to claim 1, characterized in that: The feature vectors of the multiple setting objects are obtained in the following manner: Acquire the first ultrasound image sample set; Performing feature extraction on each ultrasonic image included in the first ultrasonic image sample set to obtain a feature vector corresponding to each ultrasonic image included in the first ultrasonic image sample set; Based on the similarity between the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set, cluster analysis is performed on the feature vectors corresponding to the respective ultrasound images included in the first ultrasound image sample set to obtain the feature vectors of the plurality of set objects.

3. The method according to claim 1, characterized in that The similarity between the feature vector of each set object and the feature vector of the target ultrasound image is: the cosine similarity between the feature vector of each set object and the feature vector of the target ultrasound image.

4. The method according to claim 1, characterized in that: The similarity between the feature vector of each set object and the feature vector of the target ultrasound image is: the inner product value of the feature vector of each set object and the feature vector of the target ultrasound image.

5. The method according to claim 1, characterized in that The first key point detection model corresponding to the first setting object is trained in the following manner: Acquire a second ultrasound image sample set corresponding to the first setting object; the second ultrasound image sample set includes a plurality of ultrasound images and key point annotation data corresponding to each ultrasound image; the plurality of ultrasound images in the second ultrasound image sample set all include the first setting object; For any one ultrasound image, based on the initial first key point detection model, perform key point detection on the ultrasound image to obtain key point detection data; determine the loss value based on the key point detection data and the key point annotation data corresponding to the ultrasound image; Based on the loss value corresponding to each ultrasonic image included in the second ultrasonic image sample set, the parameter information of the initial first key point detection model is adjusted to obtain a trained first key point detection model.

6. The method according to claim 1, characterized in that The method further comprises: Acquire a key point annotation result of the target ultrasound image by the user; When it is determined that the detection result and the annotation result are different, adding the target ultrasound image to the second ultrasound image sample set corresponding to the target object to obtain a third ultrasound image sample set corresponding to the target object; Based on the third ultrasound image sample set corresponding to the target object, a target key point detection model corresponding to the target object is trained.

7. The method according to claim 1, characterized in that The method further comprises: After performing key point detection on the target ultrasound image based on the target key point detection model to obtain a detection result, in response to a training operation of the target key point detection model corresponding to the target object triggered by the user, the target ultrasound image is added to a second ultrasound image sample set corresponding to the target object to obtain a fourth ultrasound image sample set corresponding to the target object; Based on the fourth ultrasound image sample set corresponding to the target object, a target key point detection model corresponding to the target object is trained.

8. The method according to claim 5, characterized in that The method further comprises: Real-time detection of the current resource usage rate of the ultrasound device; When the current resource usage rate of the ultrasound device is lower than the preset usage rate, the first key point detection model corresponding to the first setting object is trained based on the second ultrasound image sample set corresponding to the first setting object.

9. An ultrasonic device, characterized in that: The device includes an image acquisition unit, a processor and a display screen; The image acquisition unit is used to acquire a target ultrasound image; The processor is configured to extract features of the target ultrasonic image in response to a detection operation triggered by a user for the target ultrasonic image, and obtain a feature vector of the target ultrasonic image; determine a target object in the target ultrasonic image according to feature vectors of a plurality of set objects and the feature vector of the target ultrasonic image; wherein the feature vectors of the plurality of set objects are obtained by clustering analysis based on feature vectors of at least one set object included in a first ultrasonic image sample set; the first ultrasonic image sample set includes a plurality of ultrasonic images; the feature vector of the target object is a feature vector with the highest similarity to the feature vector of the target ultrasonic image among the feature vectors of the plurality of set objects; determine a target key point detection model corresponding to the target object in the key point detection models respectively corresponding to the plurality of set objects; wherein the first key point detection model corresponding to the first set object is obtained by training a second ultrasonic image sample set corresponding to the first set object, and the first set object is any one of the plurality of set objects; the second ultrasonic image sample set includes a plurality of ultrasonic images; based on the target key point detection model, perform key point detection on the target ultrasonic image to obtain a detection result; The display screen is used to display the detection result in a display interface.

10. A key point detection device, characterized in that: The device comprises: An acquisition module, used for acquiring a target ultrasound image; a feature vector determination module, configured to extract features of the target ultrasound image in response to a detection operation triggered by a user and to obtain a feature vector of the target ultrasound image; an object determination module, configured to determine a target object in the target ultrasound image according to feature vectors of a plurality of set objects and a feature vector of the target ultrasound image; wherein the feature vectors of the plurality of set objects are obtained by clustering analysis based on feature vectors of at least one set object included in a first ultrasound image sample set; the first ultrasound image sample set includes a plurality of ultrasound images; and the feature vector of the target object is a feature vector having the highest similarity with a feature vector of the target ultrasound image among the feature vectors of the plurality of set objects; A model determination module is used to determine a target key point detection model corresponding to the target object among the key point detection models corresponding to the multiple set objects respectively; wherein the first key point detection model corresponding to the first set object is obtained by training a second ultrasound image sample set corresponding to the first set object, the first set object is any set object among the multiple set objects; the second ultrasound image sample set includes multiple ultrasound images; The detection module is used to perform key point detection on the target ultrasound image based on the target key point detection model to obtain a detection result.

Images