Fetal ultrasound image growth parameter measurement method, apparatus, device, and storage medium

Abstract

The application relates to a fetal ultrasound image growth parameter measurement method, device, equipment and storage medium. The method comprises the following steps: performing feature extraction of different scales on an ultrasound image containing a target structure to obtain a plurality of initial features; performing feature fusion on the initial features by a pixel aggregation feature pyramid network to obtain a plurality of fusion features with different scales; for the fusion features of each scale, performing point multiplication processing on the fusion features by using a convolution module perceiving the target structure and the processing result of the fusion features; based on the result of the point multiplication processing and a detection head module, obtaining a predicted measurement point of the target structure under each scale; performing weighted average operation on the predicted measurement points of the target structure under multiple scales to obtain positioning information of a measurement point of the target structure in the ultrasound image, and determining a growth parameter value of a fetus in the ultrasound image based on the positioning information. The method can improve the accuracy of the predicted measurement point.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of measurement point prediction, and in particular to a fetal ultrasound image growth parameter measurement method, device, equipment and storage medium. BACKGROUND

[0002] With the development of Internet technology, the requirement for parameter measurement of each structure in an ultrasound image is also higher and higher. The common automatic method at present is to segment the structure region to be measured by using a segmentation method, then to obtain a measurement line segment by a fitting method, and then to determine whether an abnormality is generated according to a measurement value. In addition, there is also a method of performing convolution processing on an ultrasound image, and detecting each structure in the ultrasound image by using the result of the convolution processing.

[0003] However, the existing method has the following two problems. One is that the data volume is limited, because the training data of the segmentation network needs to be manually labeled by a doctor, so it is difficult to construct a certain scale of training data; the other is that there are two parts of errors in the measurement value, that is, segmentation measurement object and fitting measurement point from the segmented structure, and segmentation incompleteness and fitting inaccuracy will seriously affect the measurement accuracy. This will cause the problem of low prediction accuracy of the measurement point. SUMMARY

[0004] Therefore, it is necessary to provide a fetal ultrasound image growth parameter measurement method, device, equipment and storage medium capable of improving the prediction accuracy of the measurement point in view of the above technical problems.

[0005] In a first aspect, the present application provides a fetal ultrasound image growth parameter measurement method, which comprises:

[0006] performing feature extraction of different scales on an ultrasound image containing a target structure to obtain a plurality of initial features;

[0007] performing feature fusion on each of the initial features by a pixel aggregation feature pyramid network to obtain a plurality of fusion features with different scales;

[0008] for each scale of the fusion features, performing point multiplication processing on the fusion features by using a convolution module perceiving the target structure and a processing result of the fusion features, and obtaining a predicted measurement point of the target structure at each scale based on a result of the point multiplication processing and a detection head module; the predicted measurement point is a single-stage predicted measurement point without segmentation and fitting;

[0009] performing weighted average operation on the predicted measurement points of the target structure at multiple scales to obtain positioning information of the measurement point of the target structure in the ultrasound image, and determining a growth parameter value of a fetus in the ultrasound image based on the positioning information.

[0010] In one of the embodiments, the method is implemented by a detection model; a training process of the detection model comprises:

[0011] performing location prediction on the training ultrasound images containing the biological structure based on the initial detection model to determine a plurality of prediction boxes of the biological structure;

[0012] determining a model loss of the initial detection model based on each of the prediction boxes of the biological structure and a ground truth box of the biological structure;

[0013] updating parameters in the initial detection model according to the model loss to obtain a trained detection model.

[0014] In one of the embodiments, before the performing location prediction on the training ultrasound images containing the biological structure based on the initial detection model to determine a plurality of prediction boxes of the biological structure, the method comprises:

[0015] obtaining a scanning video stream of the biological structure, each frame image in the scanning video stream corresponding to a different section of the biological structure;

[0016] obtaining training ultrasound images satisfying image selection conditions from the frames of ultrasound images in the scanning video stream;

[0017] determining annotation information of the training ultrasound images; the annotation information comprises coordinates of standard measurement points of the biological structure in the training ultrasound images;

[0018] determining measurement point distances between the standard measurement points of the biological structure based on the coordinates of the standard measurement points of the biological structure;

[0019] determining a ground truth box of the biological structure based on the measurement point distances and the standard measurement points.

[0020] In one of the embodiments, the obtaining training ultrasound images satisfying image selection conditions from the frames of ultrasound images in the scanning video stream comprises:

[0021] detecting the frames of ultrasound images in the scanning video stream in parallel to screen a plurality of candidate ultrasound images satisfying a spatiotemporal dependency relationship from the frames of ultrasound images;

[0022] determining training ultrasound images used as training samples of the initial detection model from the candidate ultrasound images based on a positional relationship of the biological structure in each of the candidate ultrasound images and a classification score of a prediction box.

[0023] In one of the embodiments, the model loss is a combined loss; the method further comprises:

[0024] Obtain the classification loss, localization loss, thermal loss, and geometric loss;

[0025] The weighted sum of the classification loss, the localization loss, the thermal loss, and the geometric loss is determined as the model loss of the initial detection model.

[0026] In one embodiment, the process of determining the localization loss includes:

[0027] Positive samples and their predicted measurement points are determined from multiple prediction frames of the biological structure based on a positive sample allocation strategy.

[0028] The restricted intersection ratio is determined based on the positive samples and the ground truth bounding boxes;

[0029] The center distance is determined based on the distance between the predicted measurement points of the positive samples and the standard measurement points of the ground truth bounding boxes;

[0030] The ratio result is obtained based on the ratio between the center distance and the diagonal length of the circumscribed rectangle; the circumscribed rectangle is the box determined by the ground truth box and the positive sample.

[0031] Obtain the relative positions of the predicted measurement points of the positive samples and the standard measurement points of the ground truth boxes with respect to the top-left corner of the grid where each element of the feature layer in the fused feature is located;

[0032] The positioning loss is determined based on the restricted intersection ratio, the center distance, the ratio result, and the relative position.

[0033] In one embodiment, the process of determining the geometric loss includes:

[0034] Positive samples and their predicted measurement points are determined from multiple prediction frames of the biological structure based on a positive sample allocation strategy.

[0035] Obtain the first distance between the predicted measurement points of the positive sample and the second distance between the standard measurement points;

[0036] Based on the first distance and the second distance, determine the length error between the positive sample and the standard measurement point;

[0037] The first distance and the second distance are processed using a norm method to obtain the included angle between the measured line segment of the positive sample and the actual measured line segment; the measured line segment is formed by connecting the measurement points;

[0038] The geometric loss is determined based on the length error and the included angle.

[0039] In one of the embodiments, the detection head module based on the result of the point multiplication processing obtains the predicted measurement point of the target structure at each scale, including:

[0040] A plurality of point multiplication results are obtained by performing point multiplication processing between each of the fusion features and the processing result of the fusion feature by the convolution module;

[0041] A plurality of groups of measurement point prediction results of the target structure at each scale are obtained by using the detection head module to make a prediction based on each of the point multiplication results;

[0042] The value of the classification score of each measurement point is obtained based on the plurality of groups of measurement point prediction results at each scale;

[0043] The value of the classification score is taken as the evaluation value of the target measurement point, and the result with the maximum value is determined as the predicted measurement point of the target structure at each scale.

[0044] In a second aspect, the present application provides a fetal ultrasound image growth parameter measurement device, the device comprising:

[0045] A feature extraction module is configured to extract features of different scales from an ultrasound image containing a target structure to obtain a plurality of initial features;

[0046] A feature fusion module is configured to fuse the initial features by a pixel aggregation feature pyramid network to obtain a plurality of fusion features with different scales;

[0047] A structure perception module is configured to perform point multiplication processing between the processing result of the fusion feature by a convolution module perceiving the target structure and the fusion feature for each scale of the fusion feature, and obtain the predicted measurement point of the target structure at different scales based on the result of the point multiplication processing and a detection head module;

[0048] A measurement point positioning module is configured to perform weighted average operation on the predicted measurement points of the target structure at a plurality of scales to obtain the positioning information of the measurement point of the target structure in the ultrasound image, and determine the growth parameter value of the fetus in the ultrasound image based on the positioning information.

[0049] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0050] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the above method.

[0051] In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method described above.

[0052] The fetal ultrasound image growth parameter measurement method, device, equipment and storage medium described above can obtain a plurality of fusion features of different scales by performing feature extraction of different scales on the ultrasound image containing the target structure and performing feature fusion on the obtained initial features. The point multiplication processing is performed on each fusion feature and the processing result of the convolution module on the fusion feature through structure perception, so that the detection module can predict the measurement points of the target structure according to the plurality of fusion features of different scales, and the plurality of groups of predicted measurement points of the target structure are the predicted measurement points determined by the fusion features of different scales. The weighted average processing is performed on the group of predicted measurement points with the highest classification score in each scale, the effects of the measurement point prediction by the plurality of fusion features of different scales are comprehensively considered, the accuracy of the positioning of the predicted measurement points of the target structure is improved, and the target structure is accurately evaluated. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 An application environment diagram of the fetal ultrasound image growth parameter measurement method in an embodiment;

[0054] Figure 2 A flowchart of the fetal ultrasound image growth parameter measurement method in an embodiment;

[0055] Figure 3 An ultrasound image diagram for training in an embodiment;

[0056] Figure 4 Another ultrasound image diagram for training in an embodiment;

[0057] Figure 5 A flowchart of the fetal ultrasound image growth parameter measurement method in another embodiment;

[0058] Figure 6 A network structure diagram of the detection model in an embodiment;

[0059] Figure 7 A structural block diagram of the fetal ultrasound image growth parameter measurement device in an embodiment;

[0060] Figure 8 An internal structure diagram of the computer device in an embodiment. DETAILED DESCRIPTION

[0061] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0062] The fetal ultrasound image growth parameter measurement method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 The terminal 102 communicates with the server 104 through a network. The data storage system can store data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on a cloud or other network server. The server 104 performs feature extraction of different scales on the ultrasound image containing the target structure to obtain a plurality of initial features. The server 104 performs fusion processing on each initial feature by using a pixel aggregation feature pyramid network, thereby obtaining a plurality of fusion features of different scales. The server 104 performs point multiplication processing on each fusion feature and the processing result of the convolution module on the fusion feature. The server 104 uses a detection head module to perform prediction processing on the measurement point of the target structure at each scale according to the result of the point multiplication processing, thereby obtaining the predicted measurement point of the target structure. The server 104 performs weighted average operation on the predicted measurement points of the target structure at a plurality of scales to obtain the positioning information of the measurement point of the target structure in the ultrasound image, and determines the growth parameter value of the fetus in the ultrasound image based on the positioning information. The terminal 102 can be, but is not limited to, various personal computers, notebook computers, tablet computers and the like. The server 104 can be implemented by an independent server or a server cluster composed of a plurality of servers.

[0063] In one embodiment, as shown in Figure 2 , a fetal ultrasound image growth parameter measurement method is provided. Taking the server in Figure 1 as an example, the method includes the following steps:

[0064] Step 202, performing feature extraction of different scales on the ultrasound image containing the target structure to obtain a plurality of initial features.

[0065] The target structure and the biological structure both refer to each structure in the image. For example, the posterior horn of the lateral ventricle, the cerebellum, the septum pellucidum cavity, the choroid plexus, the brain midline and the like in the ultrasound image. For another example, the target structure can be determined according to whether the length is abnormal.

[0066] An ultrasound image is an image obtained after scanning a three-dimensional target structure, and there can be multiple target structures in an ultrasound image or only one target structure. For example, only the cerebellum structure of the fetal brain needs to be measured at present, although the ultrasound image of the fetal brain also has structures such as the cavum septi pellucidi, the thalamus, and the brain midline. However, these structures do not need to be measured, so only the cerebellum is the target structure.

[0067] Different scale feature extraction is achieved by using a convolution module. For example, the size of an ultrasound image input into the detection model is 640*640, and after the convolution module is used, an initial feature with a size of 320*320 is obtained. Then, the initial feature with a size of 320*320 is continuously subjected to feature extraction by using the convolution module, and an initial feature with a size of 40*40 is obtained.

[0068] Optionally, the server first uses the convolution module to extract features from the ultrasound image containing the target structure to obtain a first initial feature, and then uses the convolution module to extract features from the first initial feature and repeats the process until multiple initial features are obtained.

[0069] Optionally, the server uses the convolution module to extract features of different sizes from the ultrasound image to obtain multiple initial features.

[0070] In step 204, the pixel aggregation feature pyramid network is used to fuse the features of the initial features to obtain multiple fused features with different scales.

[0071] The pixel aggregation feature pyramid network is based on the up-sampling fused feature pyramid of the feature pyramid network, and adds a bottom-up down-sampling fusion process to enhance the learning of semantic features and positioning features of different scales.

[0072] Feature fusion is to fuse initial features with the same scale. For example, the scales of initial feature A and initial feature B are different, and the scales of A and B need to be adjusted to the same scale. The adjustment methods include up-sampling processing and down-sampling processing. The scale of A can be adjusted to the same scale as B, or the scale of B can be adjusted to the same scale as A, or the scales of A and B can be adjusted to another scale. After the initial features are adjusted to the same scale, feature splicing is performed, and the convolution module is used to process the spliced result to complete the feature fusion operation and obtain multiple fused features.

[0073] The fused features are features with strong semantic representation and strong positioning ability, which can improve the accuracy of the measurement point prediction.

[0074] Optionally, the server samples the initial features of different scales through a pixel aggregation feature pyramid network, so that the initial features are of the same scale when the feature fusion is performed, adjusts the initial features to the same scale, performs feature stitching, and processes the stitching result using a convolution module, thereby completing the feature fusion operation and obtaining a plurality of fused features.

[0075] In step 206, for each scale of the fused feature, the convolution module that perceives the target structure is used to multiply the processing result of the fused feature by the fused feature, and based on the multiplication result and the detection head module, the predicted measurement point of the target structure at each scale is obtained.

[0076] The processing of the convolution module on the fused feature includes convolution, pooling, etc. Before the multiplication, the size of the result processed by the convolution module does not change, that is, the size of the input and output of the convolution module is the same, and only the input data is processed.

[0077] The multiplication processing is to multiply the elements at the corresponding positions of the fused feature and the processing result of the convolution module on the fused feature. For example, for the fused feature and the processing result of the convolution module on the fused feature, both of which are 20*20 in size, the element value at the first position of the fused feature is multiplied by the element value at the first position of the processing result, to obtain the element value at the first position of the multiplication result.

[0078] The number of fused features is multiple, one fused feature is multi-dimensional, and based on the multiplication result of each fused feature, the predicted measurement point of the target structure can be obtained after the measurement point prediction processing of the detection head module. For example, three fused features are A, B and C, the fused feature A and the processing result of the convolution module on the fused feature A are multiplied, the detection head module performs measurement point prediction processing according to the multiplication result, thereby obtaining the predicted measurement point P1 of the target structure X; the fused feature B and the processing result of the convolution module on the fused feature B are multiplied, the detection head module performs measurement point prediction processing according to the multiplication result, thereby obtaining the predicted measurement point P2 of the target structure X; the fused feature C and the processing result of the convolution module on the fused feature C are multiplied, the detection head module performs measurement point prediction, thereby obtaining the predicted measurement point P3 of the target structure X.

[0079] Two measurement points constitute a group of measurement points of the target structure. When the predicted measurement points of the target structure at one scale are multiple groups, the group with the highest classification score is selected from the multiple groups of predicted measurement points as the final predicted measurement point of the target structure at the scale. The classification score is the sum of the classification scores of the two measurement points in the same group, and the classification scores of the two measurement points are directly obtained from the prediction result. The single-stage predicted measurement point without segmentation and fitting refers to a process of obtaining a predicted measurement point, in which a detection model is used to obtain the predicted measurement point from the entire ultrasound image without segmenting the target structure in the ultrasound image and fitting the segmented target structure. Because the segmentation of the target structure in the ultrasound image is not complete and the fitting is not accurate, the prediction accuracy is affected, that is, the two steps of segmentation and fitting will bring two parts of errors. In the present application, only a single-stage process is performed to obtain the predicted measurement point of the target structure in the ultrasound image, that is, the detection model is used to process the ultrasound image, and the predicted measurement point of the target structure is directly output, which can reduce the measurement error.

[0080] Optionally, the server uses the convolution module to perform convolution, pooling and the like on each fusion feature, and performs dot multiplication processing on the processed result and the fusion feature itself, and uses the detection head module to perform measurement point prediction processing on the dot multiplication processing result, so as to obtain the predicted measurement point of the target structure at each scale.

[0081] In one specific application, the server uses the convolution module to perform convolution, pooling and the like on the fusion feature A, and performs dot multiplication processing on the processed result and the fusion feature A itself, and uses the detection head module to perform measurement point prediction processing on the dot multiplication result, so as to obtain multiple groups of predicted measurement points of the target structure. The server selects a group with the highest classification score from the multiple groups of predicted measurement points as the final predicted measurement point corresponding to the fusion feature A.

[0082] In step 208, the weighted average operation is performed on the predicted measurement points of the target structure at multiple scales to obtain the positioning information of the measurement point of the target structure in the ultrasound image, and the growth parameter value of the fetus in the ultrasound image is determined based on the positioning information.

[0083] The prediction measurement points at multiple scales refer to a group of prediction measurement points with the highest classification score at each scale. The weighted average operation is determined according to the weight of the feature layer where the fusion feature at each scale is located and the prediction measurement points at each scale to determine the positioning information of the measurement point of the target structure. For example, a group of prediction measurement points obtained based on the fusion feature in the feature layer with a scale of A is a, the weight of the feature layer is x, a group of prediction measurement points obtained based on the fusion feature in the feature layer with a scale of B is b, and the weight of the feature layer is y. The positioning information of the measurement point of the target structure in the ultrasound image is ax+by. The growth parameter value refers to the length between the prediction measurement points of the target structure. Based on the growth parameter value, the doctor can analyze whether the target structure is diseased.

[0084] Optionally, the server performs a weighted average operation on the data of the prediction measurement points at each scale according to the prediction measurement points of the target structure at each scale with the highest classification score based on the fusion features at different scales and the weight of the feature layer where the fusion feature at each scale is located, thereby obtaining the positioning information of the measurement point of the target structure in the ultrasound image, that is, the coordinate position of the prediction measurement point. The server calculates the distance between the measurement points of the target structure according to the obtained positioning information of the measurement point of the target structure, and determines the value of the distance as the growth parameter value of the fetus in the ultrasound image to realize the evaluation of the target structure.

[0085] In the above fetal ultrasound image growth parameter measurement method, different scale feature extraction is performed on the ultrasound image containing the target structure, and the obtained initial features are fused to obtain multiple fusion features at different scales. Point multiplication processing is performed on each fusion feature and the processing result of the convolution module on the fusion feature through structure perception, so that the detection head can predict the measurement point of the target structure according to the multiple fusion features at different scales, so that the multiple groups of prediction measurement points of the target structure are the prediction measurement points determined by the fusion features at different scales. Through weighted average processing on the group of prediction measurement points with the highest classification score in each scale, the effects of measuring the measurement points by using multiple fusion features at different scales can be considered comprehensively, so as to improve the accuracy of the positioning of the prediction measurement points of the target structure and realize accurate evaluation of the target structure.

[0086] In one embodiment, the fetal ultrasound image growth parameter measurement method is implemented through a detection model. The training process of the detection model includes:

[0087] Based on the initial detection model, the positioning prediction of the training ultrasound image containing the biological structure is performed to determine multiple prediction boxes of the biological structure.

[0088] Based on each prediction box of the biological structure and the real box of the biological structure, the model loss of the initial detection model is determined.

[0089] updating the parameters in the initial detection model according to the model loss, to obtain a trained detection model.

[0090] The initial detection model is a model that has not been trained, and the detection model is a model that has been trained. The input data of the initial detection model is a training ultrasound image and annotation information of each biological structure in the training ultrasound image. For example, the annotation information is the coordinates of standard measurement points of each biological structure and the length and width of a real box. The input of the detection model is an ultrasound image. The output of the initial detection model and the detection model is the predicted measurement point coordinates of each structure in the image.

[0091] The prediction box is determined based on the coordinates and distance between two measurement points in the same set of measurement points of the biological structure. The measurement point is the center of the prediction box, and the length and width of the prediction box are determined by the distance between the two measurement points. For example, if the distance between the two measurement points is L, then the length and width of the prediction box are set to a random value. The real box is determined according to the standard measurement points of the biological structure. When the length and width of the real box and the prediction box are the same, the loss of the aspect ratio does not need to be considered when calculating the model loss, thereby simplifying the calculation process and improving the efficiency of model training.

[0092] The model loss of the initial detection model is determined by the combined loss of the weighted average of the classification loss, the positioning loss, the heat loss, and the geometric loss. The determination process of the classification loss is as follows: based on the respective predicted classification scores of each prediction box of the biological structure and the positive sample assignment strategy, the positive sample of the biological structure is determined from the multiple prediction boxes of the biological structure, and the quality score of the positive sample is calculated, wherein, in order to align the classification loss and the positioning loss, the quality score of the positive sample is taken as the real classification score of the positive sample; then the loss is obtained based on the quality score of the positive sample and the predicted classification score of the positive sample. The quality score of the positive sample can be understood as the possibility of the positive sample belonging to a certain biological structure, and the predicted classification score of the positive sample can be understood as the possibility of the predicted positive sample belonging to a certain biological structure. For example, for the biological structure x, the quality score of the positive sample A is 0.9, and the predicted classification score is 0.7, then the classification score of the positive sample A belonging to the biological structure x is 0.9, and the score of the positive sample A belonging to the biological structure x after prediction is 0.7.

[0093] The calculation formula of the classification loss is wherein L cls is the classification loss, BCE represents binary cross entropy, p i represents the predicted classification score of the i-th positive sample, t i represents the quality score of the i-th positive sample, and n represents the number of positive samples.

[0094] The heat loss is a loss obtained based on a real heat value of a standard measurement point of the biological structure and a predicted heat value of a predicted measurement point. The heat loss is calculated by the following formula where S x S represents a size of the feature map, represents a heat value of the i-th position of the predicted measurement point, g i represents a real heat value of the i-th position of the standard measurement point generated by the Gaussian distribution. a and β are parameter factors.

[0095] Optionally, the server determines a real box according to the standard measurement point of the biological structure, and then uses the initial detection model to perform measurement point positioning prediction on the training ultrasound image containing the biological structure to obtain a plurality of predicted boxes and predicted measurement points of the biological structure. The server determines a model loss of the initial detection model based on the plurality of predicted boxes and the real box of the biological structure, and according to a weighted sum result between the classification loss, the positioning loss, the heat loss, and the geometric loss. The server feeds back the model loss to the initial detection model, and updates the parameters in the initial detection model, so as to obtain the trained detection model.

[0096] In the embodiment, the model loss of the initial detection model can be calculated according to the predicted box and the real box of the biological structure, and the parameters in the initial detection model are updated through the model loss, so that the initial detection model can complete the model training to obtain the trained detection model.

[0097] In one embodiment, before determining the plurality of predicted boxes of the biological structure based on the initial detection model performing positioning prediction on the training ultrasound image containing the biological structure, the method comprises:

[0098] Obtaining a scanning video stream of the biological structure, each frame of image in the scanning video stream corresponding to a different section of the biological structure.

[0099] From each frame of ultrasound image in the scanning video stream, a training ultrasound image satisfying an image selection condition is obtained.

[0100] Determine the annotation information of the training ultrasound image. The annotation information includes the coordinates of the standard measurement points of the biological structure in the training ultrasound image.

[0101] Determine the measurement point distance between the standard measurement points of the biological structure based on the coordinates of the standard measurement points of the biological structure.

[0102] Determine the real box of the biological structure based on the measurement point distance and the standard measurement points.

[0103] The image selection condition refers to satisfying a spatiotemporal dependency and an evaluation score exceeding a set threshold. The spatiotemporal dependency refers to a relationship between a historical frame before a current frame of the scanned video stream and the current frame, and needs to satisfy a specified number of consecutive frames to be determined as an image containing a biological structure. The evaluation score is a comprehensive score obtained according to a scoring mechanism constructed based on a position relationship of the biological structure and a classification score of the prediction box. The classification score of the prediction box can be understood as a possibility of the prediction box belonging to a certain biological structure. For example, according to the relationship between the historical frame before the current frame of the scanned video stream and the current frame, a lateral ventricle horizontal cross-section or a cerebellum horizontal cross-section satisfying the specified number of consecutive frames is determined, and a comprehensive score is obtained according to a scoring mechanism constructed based on a position relationship of a posterior horn of the lateral ventricle, a septum pellucidum cavity, a choroid plexus, a brain midline and the like and the classification score of the prediction box, and an image with a total score higher than 90 points is selected as a standard ultrasound image, that is, a training ultrasound image of the initial detection model, as shown in FIGS. Figure 3 and Figure 4 .

[0104] The annotation information is a standard measurement point coordinate of the biological structure and a length and width of a real box. Two standard measurement points of the same biological structure are a group of measurement points, that is, the classification of the same group of standard measurement points is the same. The real box of each biological structure is determined with the standard measurement point as the center and the distance between the standard measurement points as the length and width. The real box is a rectangular box with equal length and width, which is determined according to the standard measurement points before the standard measurement points are input into the initial detection model.

[0105] Optionally, the server obtains a scanned video stream containing a biological structure, and then determines an image satisfying a spatiotemporal dependency and an evaluation score exceeding a set threshold from the scanned video stream as a training ultrasound image, and determines a standard measurement point coordinate of each biological structure in the training ultrasound image and a measurement point distance between standard measurement points of each biological structure. The server further determines a respective real box of each biological structure according to the standard measurement point coordinate of each biological structure and the measurement point distance between the standard measurement points of each biological structure.

[0106] In this embodiment, by taking the image satisfying the image selection condition in the scanned video stream as the training ultrasound image, images that are not clear or not standard can be excluded to interfere with model training, so that the initial detection model learns more useful information.

[0107] In one embodiment, the training ultrasound image satisfying the image selection condition is obtained from each frame ultrasound image in the scanned video stream, including:

[0108] The respective frame ultrasound image in the scanned video stream is detected in parallel, and a plurality of candidate ultrasound images satisfying the spatiotemporal dependency are screened from the respective frame ultrasound image.

[0109] Based on the positional relationship of the biological structure in each candidate ultrasound image and the classification score of the prediction box, a training ultrasound image used as a training sample of the initial detection model is determined from each candidate ultrasound image.

[0110] Wherein, the parallel detection refers to detecting adjacent frames of images in the scanning video stream at the same time, which can save the determination time of the training ultrasound image. The space-time dependence relationship refers to the relationship between the historical frames before the current frame and the current frame of the scanning video stream, and the candidate ultrasound image needs to satisfy a specified number of consecutive frames.

[0111] Optionally, the server simultaneously detects adjacent frames of images in the scanning video stream, and then screens out the candidate ultrasound image that satisfies the specified number of consecutive frames according to the relationship between the historical frames before the current frame and the current frame of the scanning video stream. The server determines the training ultrasound image used for training the initial detection model from the multiple candidate ultrasound images according to the positional relationship of each biological structure in the candidate ultrasound image and the classification score of the prediction box.

[0112] In a specific application, the server simultaneously detects adjacent frames of images in the scanning video stream, and then screens out the candidate ultrasound image that satisfies the specified number of consecutive frames according to the relationship between the historical frames before the current frame and the current frame of the scanning video stream. The server screens out the image with a total score higher than 90 points as the training ultrasound image used for training the initial detection model according to the scoring mechanism constructed according to the positional relationship of each biological structure in the candidate ultrasound image and the classification score of the prediction box.

[0113] In the embodiment, by first screening out the candidate ultrasound image that satisfies the space-time dependence relationship from the scanning video stream, and then determining the training ultrasound image used for training the initial detection model from the candidate ultrasound image according to the positional relationship and the classification score of the prediction box, the interference of non-standard images on model training can be reduced, so that the model learns more useful information and improves the effect of model training.

[0114] In an embodiment, the fetal ultrasound image growth parameter measurement method further comprises:

[0115] Obtain the classification loss, the positioning loss, the heat loss, and the geometric loss.

[0116] Determine the weighted sum result of the classification loss, the positioning loss, the heat loss, and the geometric loss as the model loss of the initial detection model.

[0117] Wherein, the determination process of the classification loss is: based on the prediction classification score of each prediction box of the biological structure and the positive sample allocation strategy, the positive sample of the biological structure is determined from the multiple prediction boxes of the biological structure, and the quality score of the positive sample is determined by calculation, and then the classification loss is obtained based on the quality score of the positive sample and the corresponding prediction classification score.

[0118] The heat loss is a loss obtained based on a real heat value of a standard measurement point of the biological structure and a predicted heat value of a predicted measurement point. The predicted heat value is obtained by performing heat prediction of the predicted measurement point through upsampling processing of the fusion feature at each scale into the fusion feature of the same scale, adding elements at the same positions of the fusion features of the same scale, and performing convolution.

[0119] Optionally, the server uses a weighted sum result among the classification loss, the positioning loss, the heat loss, and the geometry loss as a model loss of the initial detection model.

[0120] In the embodiment, the loss of the model is calculated from different angles, the predicted measurement point and the predicted box obtained through the predicted measurement point are comprehensively evaluated, the loss of the model is more accurate, the beneficial effect of the model training is improved, and thus the trained detection model can output accurate predicted measurement points.

[0121] In one embodiment, the determination process of the positioning loss includes:

[0122] The positive sample and the predicted measurement point of the positive sample are determined from the multiple predicted boxes of the biological structure based on a positive sample distribution strategy.

[0123] The limited intersection-over-union value is determined based on the positive sample and the real box.

[0124] The center distance is determined based on the distance between the predicted measurement point of the positive sample and the standard measurement point of the real box.

[0125] The ratio result is obtained based on the ratio between the center distance and the diagonal length of the circumscribed rectangular box. The circumscribed rectangular box is determined by the real box and the positive sample.

[0126] The relative positions of the predicted measurement point of the positive sample and the standard measurement point of the real box relative to the top-left corner of each element of the feature layer in the fusion feature are obtained.

[0127] The positioning loss is determined based on the limited intersection-over-union value, the center distance, the ratio result, and the relative positions.

[0128] The positioning loss is composed of DIoU (Distance-Intersection over Union) and DFL (Distributed Focal Loss) loss. In order not to let the model spend a lot of effort to learn the size of the predicted box, a flexible mechanism is mainly used, that is, as long as the learned predicted box b i In the interval So, the formula of DIoU is modified by replacing IoU with RIR (Restricted Intersection Ratio) in the formula, which can make the predicted box b i as close as possible to the real box and make the center of the predicted box as close as possible to the center of the real box, as shown in equation (1). In equation (2), d i and are the relative positions of the positive sample and the real box to the top-left corner of the grid where each element of the feature layer is located, respectively. In order to align the classification loss and the positioning loss and not to increase the confidence of the additional branch prediction, the quality score t i of the positive sample is used as the weight. In equation (2), c represents the center distance, that is, the distance between the standard measurement point of the real box and the predicted measurement point of the positive sample, r represents the diagonal length of the circumscribed rectangular box, and γ is a proportionality coefficient, for example, γ = 0.25.

[0129]

[0130]

[0131]

[0132] where L reg is the positioning loss, L DIoU is the distance intersection ratio loss, L RIR is the restricted intersection ratio loss, and L DFL is the distribution cross-entropy loss.

[0133] The relative position refers to the relative value with the top-left corner of the grid where each element of the feature layer is located as the center. For example, if the center position is (1, 0), then the position (1, 2) relative to the center position is (0, 2).

[0134] The feature layer is determined according to the source of the positive sample. For example, if the positive sample is obtained based on the feature layer where the fusion feature A is located, then when determining the relative positions of the positive sample and the real box to the top-left corner of the grid where each element of the feature layer is located, the relative positions of the positive sample and the real box to the top-left corner of the grid where each element of the feature layer where the fusion feature A is located are determined.

[0135] Optionally, the server processes the multiple prediction boxes of the biological structure using the positive sample assignment strategy, and determines a positive sample of the biological structure and a predicted measurement point of each positive sample therefrom. The server determines a limited intersection-over-union value based on the positive sample and the real box. The server obtains a center distance according to a distance between the predicted measurement point of the positive sample of the biological structure and the standard measurement point of the real box. The server determines a ratio result according to a ratio between the center distance and a diagonal length of the circumscribed rectangular box. The server further respectively obtains relative positions of the predicted measurement point of the positive sample and the standard measurement point of the real box relative to a top-left corner of a grid where each element of the feature layer is located. The server obtains the positioning loss of the model according to a calculation manner of the formulas (1) and (2) based on the limited intersection-over-union value, the center distance, the quality score of the positive sample, the ratio result, and the relative positions.

[0136] In the embodiment, by adding the limited intersection-over-union loss in the positioning loss, the prediction box can contain the real box as much as possible, and the center point of the prediction box, that is, the predicted measurement point, can be as close as possible to the center point of the real box, that is, the standard measurement point.

[0137] In one embodiment, the determination process of the geometric loss includes:

[0138] Determining a positive sample and a predicted measurement point of the positive sample from the multiple prediction boxes of the biological structure based on the positive sample assignment strategy.

[0139] Obtaining a first distance between the predicted measurement points of the positive sample and a second distance between the standard measurement points.

[0140] Determining a length error between the positive sample and the standard measurement point based on the first distance and the second distance.

[0141] Performing norm processing on the first distance and the second distance using a norm manner to obtain an included angle between a measurement line segment of the positive sample and a real measurement line segment. The measurement line segment is connected by the measurement points.

[0142] Determining the geometric loss based on the length error and the included angle.

[0143] The calculation formula of the length error L p is The calculation formula of the included angle Θ p is l i represents a distance between the predicted measurement points of the positive sample box, represents a distance between the standard measurement points of the real box, ||x||2 is a 2-norm of x, and ε is a minimum value that can prevent a denominator from being 0. The difference between the included angle Θ p and the length error L p is the geometric information G p , and the geometric loss Lgeo = 1 - G p .

[0144] The real measurement line segment is connected by the standard measurement points of the real box.

[0145] Optionally, the server determines the positive sample from the plurality of predicted boxes of the biological structure based on a positive sample distribution strategy, and determines the predicted measurement points of each positive sample. The server obtains the length error between the positive sample and the standard measurement points of the biological structure according to the first distance between the predicted measurement points of the positive sample and the second distance between the standard measurement points of the real box. The server obtains the included angle between the measurement line segment of the positive sample and the real measurement line segment according to the norm processing of the first distance and the second distance. The server determines the geometric information according to the difference between the included angle and the length error, and determines the geometric loss of the model according to the geometric information.

[0146] In this embodiment, by training the model based on the geometric loss, the positions of the same type of predicted measurement points output by the model can be associated and constrained, and the real result is continuously optimized, thereby improving the accuracy of the prediction result.

[0147] In one embodiment, as shown in Figure 5 , based on the result of the dot product processing and the detection head module, the predicted measurement points of the target structure at each scale are obtained, including:

[0148] Step 502, obtaining a plurality of dot product results between each fusion feature and the processing result of the fusion feature by the convolution module.

[0149] Step 504, using the detection head module to perform prediction based on each dot product result to obtain a plurality of groups of measurement point prediction results of the target structure at each scale.

[0150] Step 506, obtaining the value of the classification score of each measurement point based on the plurality of groups of measurement point prediction results at each scale.

[0151] The value of the classification score refers to the result of adding the classification scores of two measurement points in a group of measurement point prediction results.

[0152] Step 508, taking the value of the classification score as the evaluation value of the target measurement point, and selecting the result with the maximum value as the predicted measurement point of the target structure at each scale.

[0153] The target measurement point refers to any one of the plurality of groups of measurement point prediction results.

[0154] Optionally, the server performs dot multiplication on elements at the same position in the fusion features and the results after the convolution module processes the fusion features, obtains a plurality of dot multiplication results after performing dot multiplication on each fusion feature and the corresponding processing result, and predicts measurement points of the target structure using the detection head module according to each dot multiplication result to obtain a plurality of groups of measurement point prediction results of the target structure at each scale. The server further screens a group of measurement points with the highest classification score from the plurality of groups of measurement point prediction results at each scale as the predicted measurement points of the target structure at each scale.

[0155] In this embodiment, the predicted measurement points with the target value of the classification score are taken as the predicted measurement points of the target structure at each scale, which can exclude the interference of the predicted measurement points with a lower classification score, so that the positioning information of the final obtained measurement points is more accurate, thereby facilitating accurate evaluation of the biological structure.

[0156] The application further provides an application scenario of the fetal ultrasound image growth parameter measurement method. Specifically, the fetal ultrasound image growth parameter measurement method is applied in the application scenario as follows: adjacent frames of images in a scanning video stream are detected in parallel, and a plurality of candidate ultrasound images satisfying a space-time dependent relationship are screened out; a training ultrasound image used for training an initial detection model is determined from the plurality of candidate ultrasound images according to a positional relationship of biological structures in each candidate ultrasound image and a classification score of a prediction box. Coordinate information of standard measurement points of the biological structures in the training ultrasound image and real heat values of the standard measurement points are obtained, and real boxes of the biological structures are determined based on the coordinate information. The training ultrasound image, the coordinate information of the biological structures in the training ultrasound image, and the real boxes of the biological structures are input into the initial detection model, and the initial detection model is trained. The initial detection model uses a convolution module to perform feature extraction of the training ultrasound image at different scales to obtain a plurality of features at different scales, and then performs fusion processing on the plurality of features at different scales to obtain a plurality of fused features at different scales. The detection head module is used to perform measurement point prediction processing on a dot product result between each scale of the fused features and a processing result of the convolution module on the fused features, so that prediction measurement points of the biological structures at each scale can be obtained. A prediction box is determined based on a group of prediction measurement points of the biological structures, and a plurality of prediction boxes can be determined based on a plurality of groups of prediction measurement points. A positive sample of the biological structures is determined from the plurality of prediction boxes using a positive sample distribution strategy, and a classification loss of the model is calculated based on a classification score of the positive sample and a classification score of the prediction box. A limited intersection-over-union value is determined based on the positive sample and the real box. A center distance is obtained according to a distance between the prediction measurement points of the positive sample of the biological structures and the standard measurement points of the real box. A ratio result is determined according to a ratio between the center distance and a diagonal length of a circumscribed rectangular box. The relative positions of the prediction measurement points of the positive sample and the standard measurement points of the real box relative to the top left corner of a grid where each element of a feature layer is located are obtained, respectively. A positioning loss of the model can be obtained based on the limited intersection-over-union value, the center distance, a quality score of the positive sample, the ratio result, and the relative positions. A length error between the positive sample and the real box and an included angle between measurement line segments can be obtained based on a first distance between the prediction measurement points of the positive sample and a second distance between the standard measurement points of the real box. A geometric loss of the model can be obtained based on the length error and the included angle. The fused features at each scale are upsampled to become fused features at the same scale, and then elements at the same position of the fused features at the same scale are added, and then heat prediction of the prediction measurement points is performed through convolution to obtain a prediction heat value. A heat loss of the model is obtained based on the prediction heat value and the real heat value.

[0157] The classification loss, the positioning loss, the heat loss and the geometric loss are weighted and averaged to obtain a model loss. The model loss is fed back to the initial detection model to update parameters in the initial detection model, and a trained detection model is obtained. Finally, the ultrasound image is input into the detection model, and the measurement point positioning information of the target structure in the ultrasound image is obtained. According to the distance between the measurement points, the growth parameter value of the fetus in the ultrasound image is determined, so that the abnormal evaluation of the target structure can be realized. The network structure of the detection model is as shown in Figure 6

[0158] It should be understood that, although each step in the flowchart involved in each of the above-described embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each of the above-described embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0159] Based on the same inventive concept, the embodiments of the present application also provide a fetal ultrasound image growth parameter measurement device for implementing the fetal ultrasound image growth parameter measurement method described above. The problem-solving implementation scheme provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more fetal ultrasound image growth parameter measurement device embodiments provided below can refer to the limitations of the fetal ultrasound image growth parameter measurement method described above, which will not be repeated here.

[0160] In one embodiment, as shown in Figure 7 a fetal ultrasound image growth parameter measurement device is provided, comprising:

[0161] The feature extraction module 702 is configured to extract features of different scales from the ultrasound image containing the target structure to obtain a plurality of initial features.

[0162] The feature fusion module 704 is configured to perform feature fusion on each initial feature by a pixel aggregation feature pyramid network to obtain a plurality of fusion features with different scales.

[0163] The structure perception module 706 is configured to, for each scale of the fusion feature, perform point multiplication processing on the fusion feature based on a processing result of the fusion feature by a convolution module perceiving the target structure and a detection head module, to obtain a predicted measurement point of the target structure at each scale.​

[0164] The measurement point positioning module 708 is configured to perform a weighted average operation on the predicted measurement points of the target structure at multiple scales to obtain positioning information of the measurement points of the target structure in the ultrasound image, and determine a growth parameter value of the fetus in the ultrasound image based on the positioning information.

[0165] In one embodiment, the fetal ultrasound image growth parameter measurement device comprises:

[0166] The prediction box determination unit is configured to perform positioning prediction on the training ultrasound images containing the biological structure based on the initial detection model, and determine a plurality of prediction boxes of the biological structure.

[0167] The model loss determination unit is configured to determine a model loss of the initial detection model based on each prediction box of the biological structure and the ground truth box of the biological structure.

[0168] The model training unit is configured to update the parameters in the initial detection model according to the model loss to obtain the trained detection model.

[0169] In one embodiment, the prediction box determination unit comprises:

[0170] The video acquisition subunit is configured to acquire a scanning video stream of the biological structure, and each frame image in the scanning video stream corresponds to a different section of the biological structure.

[0171] The training ultrasound image determination subunit is configured to acquire the training ultrasound images satisfying the image selection condition from the ultrasound images in the scanning video stream.

[0172] The coordinate determination subunit is configured to determine annotation information of the training ultrasound image. The annotation information comprises coordinates of standard measurement points of the biological structure in the training ultrasound image.

[0173] The measurement point distance determination subunit is configured to determine measurement point distances between the standard measurement points of the biological structure based on the coordinates of the standard measurement points of the biological structure.

[0174] The ground truth box determination subunit is configured to determine the ground truth box of the biological structure based on the measurement point distances and the standard measurement points.

[0175] In one embodiment, the training ultrasound image determination subunit comprises:

[0176] The candidate ultrasound image determination subunit is configured to perform parallel detection on the ultrasound images in the scanning video stream, and screen a plurality of candidate ultrasound images satisfying a spatiotemporal dependency relationship from the ultrasound images.

[0177] The image screening subunit is configured to determine, from each candidate ultrasound image, a training ultrasound image used as a training sample of the initial detection model based on a positional relationship of the biological structure in each candidate ultrasound image and a classification score of the bounding box.

[0178] In an embodiment, the fetal ultrasound image growth parameter measurement device further comprises:

[0179] The loss obtaining unit is configured to obtain the classification loss, the positioning loss, the heat loss, and the geometric loss.

[0180] The model loss determination unit is configured to determine a weighted sum result of the classification loss, the positioning loss, the heat loss, and the geometric loss as a model loss of the initial detection model.

[0181] In an embodiment, the loss obtaining unit comprises:

[0182] The first positive sample determination subunit is configured to determine, based on a positive sample assignment strategy, the positive sample and a predicted measurement point of the positive sample from the multiple bounding boxes of the biological structure.

[0183] The limited intersection-over-union value determination subunit is configured to determine a limited intersection-over-union value based on the positive sample and the real bounding box.

[0184] The center distance determination subunit is configured to determine a center distance based on a distance between the predicted measurement point of the positive sample and the standard measurement point of the real bounding box.

[0185] The ratio result determination subunit is configured to obtain a ratio result based on a ratio between the center distance and a diagonal length of a circumscribed rectangular box determined by the real bounding box and the positive sample.

[0186] The relative position determination subunit is configured to obtain relative positions of the predicted measurement point of the positive sample and the standard measurement point of the real bounding box with respect to top-left corners of grids of each element in the feature layer of the fusion feature.

[0187] The positioning loss determination subunit is configured to determine the positioning loss based on the limited intersection-over-union value, the center distance, the ratio result, and the relative positions.

[0188] In an embodiment, the loss obtaining unit comprises:

[0189] The second positive sample determination subunit is configured to determine, based on a positive sample assignment strategy, the positive sample and a predicted measurement point of the positive sample from the multiple bounding boxes of the biological structure.

[0190] The distance determination subunit is configured to obtain a first distance between the predicted measurement points of the positive sample and a second distance between the standard measurement points.

[0191] The length error determination subunit is configured to determine a length error between the positive sample and the standard measurement point based on the first distance and the second distance.

[0192] The included angle determination subunit is configured to perform norm processing on the first distance and the second distance using a norm method to obtain an included angle between a measurement line segment of the positive sample and a real measurement line segment. The measurement line segment is connected by measurement points.

[0193] The geometric loss determination subunit is configured to determine a difference between the length error and the included angle as a geometric loss.

[0194] The above modules in the fetal ultrasound image growth parameter measurement device can be implemented by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to the above modules.

[0195] In one embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in Figure 8 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store ultrasound images, initial features, fused features, results of dot multiplication processing, predicted measurement points of a target structure at each scale, and positioning information data of measurement points of the target structure in the ultrasound images. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with terminals outside through a network connection. The computer program is executed by the processor to implement a fetal ultrasound image growth parameter measurement method.

[0196] Those skilled in the art can understand that Figure 8 The structure shown in the above

[0197] In an embodiment, a computer device is also provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program.

[0198] In an embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the steps in the above method embodiments when executed by a processor.

[0199] In an embodiment, a computer program product is provided, comprising a computer program, and the computer program implementing the steps in the above method embodiments when executed by a processor.

[0200] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0201] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0202] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0203] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method of fetal ultrasound image growth parameter measurement, the method comprising: The method comprises: performing feature extraction of different scales on an ultrasound image containing a target structure to obtain a plurality of initial features; performing feature fusion on each of the initial features through a pixel aggregation feature pyramid network to obtain a plurality of fused features with different scales; for each scale of the fused features, performing point multiplication processing on the fused features by using a convolution module perceiving the target structure and a processing result of the fused features, and obtaining a predicted measurement point of the target structure at each scale based on a result of the point multiplication processing and a detection head module; the predicted measurement point is a single-stage predicted measurement point without segmentation and fitting; performing weighted average operation on the predicted measurement points of the target structure at multiple scales to obtain positioning information of a measurement point of the target structure in the ultrasound image, and determining a growth parameter value of a fetus in the ultrasound image based on the positioning information, wherein the obtaining of the predicted measurement point of the target structure at each scale based on the result of the point multiplication processing and the detection head module comprises: obtaining a plurality of point multiplication results of the point multiplication processing between each of the fused features and a processing result of the fused features by using the convolution module; obtaining a plurality of groups of measurement point prediction results of the target structure at each scale by using the detection head module based on each of the point multiplication results; obtaining a value of a classification score of each measurement point based on the plurality of groups of measurement point prediction results at each scale; taking the value of the classification score as an evaluation value of a target measurement point, and selecting a result with the maximum value as the predicted measurement point of the target structure at each scale.

2. The method of claim 1, wherein, The method is implemented by using a detection model, and a training process of the detection model comprises: performing positioning prediction on a training ultrasound image containing a biological structure based on an initial detection model to determine a plurality of predicted bounding boxes of the biological structure; determining a model loss of the initial detection model based on each of the predicted bounding boxes of the biological structure and a real bounding box of the biological structure; updating parameters in the initial detection model according to the model loss to obtain a trained detection model.

3. The method of claim 2, wherein, Before the performing of the positioning prediction on the training ultrasound image containing the biological structure based on the initial detection model to determine the plurality of predicted bounding boxes of the biological structure, the method comprises: obtaining a scanning video stream of a biological structure, each frame of image in the scanning video stream corresponding to a different section of the biological structure; obtaining a training ultrasound image satisfying an image selection condition from each frame of ultrasound image in the scanning video stream; determining annotation information of the training ultrasound image; the annotation information comprises coordinates of standard measurement points of the biological structure in the training ultrasound image; determining measurement point distances between the standard measurement points of the biological structure based on the coordinates of the standard measurement points of the biological structure; determining a real bounding box of the biological structure based on the measurement point distances and the standard measurement points.

4. The method of claim 3, wherein, The obtaining of the training ultrasound image satisfying the image selection condition from each frame of ultrasound image in the scanning video stream comprises: detecting each frame of ultrasound image in the scanning video stream in parallel, and screening a plurality of candidate ultrasound images satisfying a space-time dependency relationship from each frame of ultrasound image; Based on the positional relationship of the biological structure in each of the candidate ultrasound images and the classification score of the prediction box, a training ultrasound image used as a training sample of the initial detection model is determined from each of the candidate ultrasound images.

5. The method of claim 2, wherein, The model loss is a combined loss; the method further comprises: obtaining a classification loss, a positioning loss, a heat loss, and a geometric loss; determining a weighted sum result between the classification loss, the positioning loss, the heat loss, and the geometric loss as the model loss of the initial detection model.

6. The method of claim 5, wherein, The determination process of the positioning loss comprises: determining a positive sample and a predicted measurement point of the positive sample from the multiple prediction boxes of the biological structure based on a positive sample allocation strategy; determining a limited intersection ratio value based on the positive sample and the real box; determining a center distance based on the distance between the predicted measurement point of the positive sample and the standard measurement point of the real box; obtaining a ratio result based on the ratio between the center distance and the diagonal length of the circumscribed rectangular box; the circumscribed rectangular box is determined by the real box and the box determined by the positive sample; obtaining the relative positions of the predicted measurement point of the positive sample and the standard measurement point of the real box relative to the top left corner of each element in the feature layer of the fusion feature, respectively; determining the positioning loss based on the limited intersection ratio value, the center distance, the ratio result, and the relative positions.

7. The method of claim 5, wherein, The determination process of the geometric loss comprises: determining a positive sample and a predicted measurement point of the positive sample from the multiple prediction boxes of the biological structure based on a positive sample allocation strategy; obtaining a first distance between the predicted measurement points of the positive sample and a second distance between the standard measurement points; determining a length error between the positive sample and the standard measurement point based on the first distance and the second distance; performing norm processing on the first distance and the second distance using a norm method to obtain an included angle between the measurement line segment of the positive sample and the real measurement line segment; the measurement line segment is connected by measurement points; determining the geometric loss based on the length error and the included angle.

8. A fetal ultrasound image growth parameter measurement apparatus, characterized by, The device comprises: a feature extraction module configured to perform feature extraction of different scales on an ultrasound image containing a target structure to obtain multiple initial features; a feature fusion module configured to perform feature fusion on each of the initial features by a pixel aggregation feature pyramid network to obtain multiple fusion features with different scales; a structure perception module configured to, for each scale of the fusion feature, perform point multiplication processing on a processing result of the fusion feature by a convolution module perceiving the target structure and the fusion feature based on a result of the point multiplication processing and a detection head module to obtain predicted measurement points of the target structure at different scales; a measurement point positioning module configured to perform weighted average operation on the predicted measurement points of the target structure at multiple scales to obtain positioning information of a measurement point of the target structure in the ultrasound image, and determine a growth parameter value of a fetus in the ultrasound image based on the positioning information, wherein the structure perception module is further configured to: A plurality of point multiplication results between each of the fusion features and a processing result of the convolution module on the fusion features are obtained; a plurality of sets of measurement point prediction results of the target structure at each scale are obtained by using a detection head module to make a prediction based on each of the point multiplication results; a value of a classification score of each measurement point is obtained based on the plurality of sets of measurement point prediction results at each scale; the value of the classification score is taken as an evaluation value of a target measurement point, and a result with the largest value is determined as a predicted measurement point of the target structure at each scale.

9. The apparatus of claim 8, wherein, The fetal ultrasound image growth parameter measurement device further includes: A prediction box determination unit is configured to determine a plurality of prediction boxes of the biological structure based on the initial detection model to make a positioning prediction on the training ultrasound image containing the biological structure; A model loss determination unit is configured to determine a model loss of the initial detection model based on each prediction box of the biological structure and a real box of the biological structure; A model training unit is configured to update parameters in the initial detection model according to the model loss to obtain a trained detection model.

10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.

Images