A Multi-Angle Fusion Analysis System and Method for Ultrasonic Images Based on Target Tracking

Abstract

A target-tracking-based ultrasound image target multi-angle fusion analysis system and method are disclosed. The system includes: a single-frame image preprocessing module, a target detection and localization module, a target trajectory tracking multi-angle information acquisition module, and a target multi-angle information fusion analysis module. The invention uses a lightweight target recognition network to identify and locate the target of interest in ultrasound detection images, and acquires key frame sequences related to the target of interest from multiple angles to complete the reconstruction of the target's three-dimensional model and the analysis of the target's three-dimensional model morphological features. Based on the three-dimensional morphological analysis, a description of the target's internal properties and external contour is provided.

Description

Technical Field

[0001] This invention relates to a technology in the field of image processing, specifically a target tracking-based ultrasound image target multi-angle fusion analysis system and method that identifies targets of interest in ultrasound images, combines target tracking to extract a series of keyframes from different observation angles of the target region in the ultrasound image, and achieves three-dimensional modeling of the target through keyframe fusion. Background Technology

[0002] Medical ultrasound, as one of the non-invasive imaging modalities for acquiring and detecting lesion features, is an important application area of ​​artificial intelligence technology in medical image analysis. Currently, the main function of computer-aided ultrasound image analysis, ultrasound image target recognition, generally adopts two methods: 1) Manually identifying suspicious lesion areas and then handing the suspicious images over to the computer for further processing; 2) Using a lightweight target recognition network to locate the target in real-time ultrasound images, assisting in manual confirmation. Due to the characteristics of ultrasound images, the extracted keyframes reflect images from a specific observation perspective. Therefore, most current ultrasound image target classification and semantic segmentation methods adopt single-angle or two-dimensional analysis methods, failing to conduct multi-angle observation and comprehensive analysis following the movement of the detection equipment. That is, they can generally only detect the features of target lesions in two-dimensional images, making it difficult to provide three-dimensional spatial information about the target lesions. Summary of the Invention

[0003] To address the shortcomings of the existing technologies, this invention proposes a multi-angle fusion analysis system and method for ultrasound images based on target tracking. The system uses a lightweight target recognition network to identify and locate the target of interest in ultrasound images, and acquires key frame sequences related to the target of interest from multiple angles to complete the reconstruction of the target's three-dimensional model and the analysis of the target's three-dimensional model morphological features. Based on the three-dimensional morphological analysis, the system provides a description of the target's internal properties and external contour.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a three-dimensional modeling image analysis system based on ultrasound images, comprising: a single-frame image preprocessing module, a target detection and localization module, a target trajectory tracking multi-angle information acquisition module, and a target multi-angle information fusion analysis module. Specifically: the single-frame image preprocessing module performs image noise reduction, image contrast enhancement, and image standardization on the original ultrasound images in a single-frame manner to optimize image quality; the target detection and localization module extracts candidate keyframes and outputs target detection and localization boxes by calculating the classification confidence of the located target in each frame of the standardized image sequence; the target trajectory tracking multi-angle information acquisition module records the relative movement trajectory of the same target of interest through target tracking and calculates a multi-angle keyframe sequence; and the target multi-angle information fusion analysis module performs three-dimensional fusion modeling on the multi-angle keyframe sequence of the same target.

[0006] The target detection and localization module employs a binary classification network for target classification, using the class probability values ​​output by the fully connected layers of the convolutional network as the classification confidence level. Specifically, the neural network maps each identified target in the input image to a probability array for its classification into each possible class. The array size is the number of potential classes for that target, and the i-th value within the array represents the confidence level of classifying the target into the i-th class. When any identified target within a frame reaches a certain maximum classification confidence level, that frame is considered a potential keyframe for subsequent steps of 3D reconstruction and fusion analysis of the high-confidence target.

[0007] This invention relates to a target-tracking-based ultrasound image multi-angle fusion analysis method based on the above-mentioned system, comprising:

[0008] Step 1: Perform image noise reduction, image contrast enhancement, and image standardization on the original ultrasound images.

[0009] Step 2: Calculate the target localization and target classification confidence scores for each frame in the standardized image sequence, and extract all candidate keyframes. This includes:

[0010] 2.1) Feature extraction is performed on the preprocessed image using convolutional layers;

[0011] 2.2) Use fully connected layers to classify the target using the feature vectors;

[0012] 2.3) Calculate the confidence score for the benign or malignant classification of the target of interest in each frame. Images with high confidence scores for any classification category are selected as high-quality frames in terms of image quality and viewing angle. The classification results of the target are not intended as any external opinions or suggestions, but are only used to increase the reliability of the classification confidence score by refining the classification categories.

[0013] The target classification step uses a binary classification network, which classifies the target into class 1, class 2 and other classes.

[0014] Step 3: Based on the target detection and localization bounding box obtained in the target detection and localization step, maintain the trajectory linked list of multiple identified targets in real time. By tracking the movement trajectory of the same target under the ultrasonic probe, capture key frames of the target from multiple angles.

[0015] In the aforementioned multi-angle keyframe capture of the target, the target of interest identified in each ultrasound image is defined as follows: There are five target tracking states: target start state, target update state, target lost state, target re-match state, and scan end state.

[0016] Step three specifically includes:

[0017] 3.1) Target Startup: Receive the new target location output by the target detection and localization module, establish a trajectory recording list and a keyframe recording list for it; label the newly located target, the label includes the target number and benign / malignant classification; record the first coordinate of the target as (0, 0), which is used to represent its position on the x / y axis of the image.

[0018] 3.2) Target update: Due to the spatial correlation between adjacent frames of ultrasound images, there is a certain overlap between the target location box in one frame and the previous frame. By calculating whether the intersection-union ratio (IOU) of the two location boxes reaches the threshold, it can be determined that the two target recognition results are the same target.

[0019] The Cross-Union Ratio (CUI) is calculated based on pixel proportion. The proportion threshold is calculated using the default probe's fastest moving speed V (cm / s), the average frame rate F (1 / s) of image acquisition within the frame update interval, the ultrasound image resolution E (pixels / cm), and the corresponding target frame size L*W (area multiplied by the length and width in units of pixels). The formula for calculating the CUI threshold M and the method for judging two targets D... i D i+1 The expression for determining whether they are the same target is: Wherein: S i This is a state variable describing the situation of targets being the same. When the calculated intersection-union ratio (IU) is greater than or equal to M, the two identified targets in the next and next frames (frame i and frame i+1) are considered to be the same target, i.e., S. i =1.

[0020] Once the new target is confirmed to be the same target as the one in the previous frame, the target trajectory list is updated, and the relative pixel distance and displacement vector between the target in the current frame and the target in the first frame are calculated. If the relative pixel distance and displacement vector reach a specified threshold, it is considered to belong to a keyframe at a different angle, and the keyframe is added to the keyframe sequence.

[0021] 3.3) Target Loss Handling: When a target is lost, the existing trajectory table and key image sequence of the lost target will be saved. When the total number of targets identified simultaneously in the current frame is less than the maximum number of targets that can be recorded simultaneously, once a new target is identified, the newly appearing target will be assumed to be the previously discarded target, and the target rematching state of that target will be activated.

[0022] 3.4) Target Rematching: For newly appearing targets, perform correlation filtering on all keyframes in the keyframe list left by the lost targets. When the grayscale energy of the filtered image reaches a specified threshold, the new target is considered to be the same target as a previously lost target; otherwise, it is considered a new target and is allowed to enter the target initial state of the new target.

[0023] The correlation filtering first converts the region of interest to the frequency domain using a two-dimensional Fast Fourier Transform (FFT-2D), then multiplies the two frequency domain graphs to obtain the correlation. The quantized value of the matching degree is obtained by calculating the average intensity of the mid-to-high frequency components in the multiplied graph, as shown below, D. i With D j For the two targets to be matched, P i P is a state variable describing the target matching situation. i A value of 1 indicates that the targets in the two images match.

[0024] 3.5) Scan ends: The track list of the tracked target and the multi-angle keyframe list of the captured target are transmitted to the target multi-angle information fusion and analysis module.

[0025] Step 4: Perform 3D fusion modeling on keyframes of the same target from multiple angles and analyze its stereo characteristics, specifically including:

[0026] 4.1) Obtain the target's 2D contour in each keyframe using an edge segmentation operator. Specifically: First, read all keyframe sequences of all known targets, and then crop the target region images sequentially. Each cropped image contains only the outermost contour of one target. Use an edge detection operator to extract the target edges, resulting in a target segmentation map that only displays the target region. Use Scale Invariant Feature Transform (SIFT) to extract feature points and corresponding feature description vectors containing the mean gray intensity and gray distribution direction vector of each feature point in the target segmentation map sequence. Specifically: Where: G(x,y,σ) represents the Gaussian-scale feature map of the original image I(x,y) at the σ blur ratio; L(x,y) represents the gradient map of the pixel-level feature points corresponding to the original image I(x,y); m(x,y) represents the gradient magnitude of L(x,y) at position (x,y); and θ(x,y) represents the gradient direction of L(x,y) at position (x,y). This yields the position (x,y) and scale (i.e., blur ratio σ, intensity m, and direction θ) of each pixel-level feature point within an image. Using a 4x4 pixel block as a practical feature point for reference and matching, let r... k R represents the feature vector at the k-th pixel in the image. i The eigenvector of any 4x4 feature point in the graph is expressed as: r k =((x,y) k ,σ k ,m k ,θ k ), R i =(r i1 ,r i2 ,…,r i16 ).

[0027] 4.2) Extract fixed feature points of each target 2D contour and match their relative positions in 3D space. Specifically, this involves using a matching method based on the minimum difference distance between feature description vectors to determine the relative positional relationships of feature points in each image. i This indicates that the other image is to be compared with R. i The 4x4 feature points for matching and comparison (corresponding to pixel feature vector t) k ), its expression is t k =((x,y) k ,σ k ,m k ,θ k ), T i =(t i1 ,t i2 ,…,t i16 ),Depend on The matching degree of feature points is calculated based on the shortest distance between vectors. The method for selecting feature points when calculating the matching degree is to traverse all feature points in the entire image.

[0028] 4.3) Map all feature points sequentially onto the 3D model until all feature points have been mapped. Specifically, using Structure of Motion (SFM), the target segmentation images already located in 3D space are sequentially mapped onto a view of the 3D pixel matrix model to be generated from a certain angle, according to their acquisition time sequence. This mapping is repeated for all target segmentation images generated from keyframe sequences until the 3D reconstruction of the target model is complete. Specifically, for the same point s of interest in 3D space, the matching points of that target in all 2D images are obtained. For any two most matching feature vectors, the eigenvector matrix F is obtained, and R can be calculated. i With T i The angle and distance relationships in three-dimensional space are used to sequentially calculate the mapping relationship of all pixels, specifically: Traditional 3D model generation methods can map the position and corresponding grayscale value of each pixel in 3D space.

[0029] Preferably, after the target three-dimensional model is reconstructed, the target multi-angle information fusion analysis module analyzes the internal pixel characteristics and appearance contour characteristics of the target three-dimensional model to obtain the uniformity, average value and regularity of pixel value distribution in the target area of ​​the ultrasound image. Specifically, the module uses the intensity of its mid-to-high frequency components in the three-dimensional frequency domain to determine the smoothness of its contour; and uses its three-dimensional gray-level co-occurrence matrix to calculate the uniformity of the internal gray-level value distribution and the average gray-level value. Technical effect

[0030] This invention combines ultrasound images of a target from multiple angles using target tracking to achieve comprehensive multi-angle analysis, overcoming the inaccuracies inherent in single-image analysis in previous automated ultrasound image analysis software. Because the characteristics of the target region differ from multiple angles, multi-angle comprehensive analysis of ultrasound images yields more accurate image descriptions and conclusions. This invention automatically captures static keyframe images from the ultrasound video stream, eliminating the need for manual cropping of key target images by doctors, as in other existing solutions. Compared to existing technologies, this invention automatically extracts target-related candidate keyframes and reconstructs a three-dimensional model of the target by mapping multiple keyframes from different angles of the target in a time-series order. Furthermore, based on the target's three-dimensional model, it obtains the average value, distribution uniformity, and smoothness of the three-dimensional shape of the internal pixel grayscale values, which cannot be reflected in the two-dimensional image. Attached Figure Description

[0031] Figure 1 This is a flowchart of the present invention;

[0032] Figure 2 This is a schematic diagram of the system in the embodiment;

[0033] Figure 3A state transition diagram for the multi-angle information acquisition module for target trajectory tracking;

[0034] Figure 4 This is a schematic diagram of 3D reconstruction in the example. Detailed Implementation

[0035] This embodiment uses thyroid ultrasound examination as an example, setting thyroid nodules as the target of interest, such as... Figure 2 As shown, this embodiment involves a three-dimensional modeling and image analysis system based on ultrasound images, including: an application layer, a business processing layer, a data layer, a video stream inlet for inputting thyroid ultrasound images, and an output port for outputting the real-time positioning box of the identified target nodules and the reconstructed model and analysis content.

[0036] The video stream entry point reads external thyroid ultrasound images from the OpenCV open-source library and converts them into ultrasound image sequences in real time. Each image is processed sequentially by modules in the business processing layer through service interface calls.

[0037] The output of the model analysis content is achieved by using OpenCV combined with the Matplotlib open-source library to display the three-dimensional pixel matrix in the user window.

[0038] The data layer uses MongoDB to persist detection logs for each identification and analysis. To reduce memory load and ensure program stability, keyframes and target location information generated during the scanning process are periodically written to MongoDB, while older intermediate identification results are discarded in memory and retrieved from the database when needed.

[0039] The business processing layer includes an image preprocessing unit, a target detection and localization unit, a target trajectory tracking unit, and a multi-angle information fusion and analysis unit. The image preprocessing unit is implemented using the Scikit-Image open-source library. It receives an input sequence of ultrasound images (single-channel grayscale images). Each image undergoes bilateral filtering to remove high-frequency noise without interfering with image edge information. A nonlinear contrast enhancement method based on exponential transform is then used to enhance the contrast of the denoised images, making the target's edges and internal features more prominent. The enhanced images are then normalized using Z-Score, mapping the original grayscale values ​​to a skewed distribution with a mean of 0 and a variance of 1, and the preprocessed image sequence is output. The target detection and localization unit uses the YOLOv5 target detection network as the backbone network of the target detection and localization module and receives the preprocessed image sequence. By replacing the RGB three-channel image input head with a grayscale (G) single-channel image input head, and combining this with neural network pruning, the parameter complexity of the original YOLO backbone network is greatly reduced, resulting in a faster-responding and more adaptable binary classification optimization network for ultrasound image recognition, which only determines target location and benignity / malignancy. Considering the semantic binary classification nature of the network, the pruning process involves pruning the semantic feature layer while retaining the underlying spatial feature layer. During network training, image cropping, image flipping, and image stretching are used as data augmentation methods, and salt-and-pepper noise is added to create interference samples for training to improve the network's anti-interference ability. The target trajectory tracking unit uses the NumPy library to quickly calculate the IOU overlap to determine continuous targets, and uses the FFT function in NumPy to implement a fast Fourier transform to match lost targets. The target movement distance is calculated using Euclidean distance, directly obtained from vector distance. The target multi-angle information fusion analysis unit uses the Canny operator to segment the target nodule contour, turning the area outside the target contour in the image black to obtain the target segmentation map. The OpenCV and Math libraries were used to analyze all corner features and feature vectors of the nodule segmentation map. Then, the built-in SFM function in the OpenCV library was used to reconstruct the target 3D model. Finally, a 3D-FFT function implemented in NumPy was used to calculate the frequency domain distribution of the target 3D model and determine whether the grayscale intensity of the mid-to-high frequency components reached the threshold for irregular edges; NumPy matrix operation functions were used to obtain the value of each element in the 3D gray-level co-occurrence matrix, and then it was determined whether the value of a specified element representing uniformity reached the threshold. The reconstructed model and analysis content were saved into a MongoDB database through the data layer interface and returned to the application layer window for display through the service interface.

[0040] The coordinates of the positioning frame and the target classification confidence array are obtained by reading the underlying modules of the YOLO5 library and output to the target trajectory tracking multi-angle information acquisition module.

[0041] This embodiment is written in Python and its effect is that of a local software installed on the user's computer, which receives the image output port of the ultrasonic testing instrument through a hardware interface and synchronizes the instrument screen and system operation and display screen in real time.

[0042] like Figure 1 The diagram shows the analysis method of the system described above in this embodiment, including:

[0043] Step 1: Perform image noise reduction, image contrast enhancement, and image standardization on the raw ultrasound images.

[0044] Step 2: Calculate the target classification confidence of each frame in the standardized image sequence to extract candidate keyframes.

[0045] Step 3: Based on the target detection and positioning box, maintain the trajectory linked list of multiple identified targets in real time. By tracking the movement trajectory of the same target under the ultrasound probe, determine key frames from different multi-angle perspectives from the candidate key frames.

[0046] Step 4: Perform 3D fusion modeling on keyframes of the same target from multiple angles and provide analysis of the internal voxels and external contours of the image stereo information.

[0047] Table 1 Comparison of Technical Features

[0048] In the technical characteristics comparison table, external performance includes: reliability, execution efficiency, and ease of use.

[0049] Compared to existing technologies that utilize single-image analysis of a target, this invention integrates and analyzes multiple images of a single target acquired from different perspectives, referencing more target region information to provide a more comprehensive image analysis result. This invention eliminates the need for manual cropping of the region of interest; instead, it automatically extracts key images to reconstruct the target's 3D model, resulting in higher operational efficiency compared to existing systems that manually crop key images. Furthermore, the lightweight target localization and classification neural network, after network pruning, ensures real-time performance during target localization. This invention further generates a 3D reconstructed model of the target, along with descriptions of the internal pixel distribution and external contours of the 3D image—features that are unavailable in 2D images produced by conventional ultrasonic testing instruments—for user viewing.

[0050] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A target-tracking-based multi-angle fusion analysis method for ultrasound images based on a target fusion analysis system, characterized in that, The system includes: a single-frame image preprocessing module, a target detection and localization module, a target trajectory tracking multi-angle information acquisition module, and a target multi-angle information fusion and analysis module. Specifically: the single-frame image preprocessing module performs image noise reduction, image contrast enhancement, and image standardization on the original ultrasound images in a single-frame manner to optimize image quality; the target detection and localization module extracts candidate keyframes and outputs target detection and localization boxes by calculating the classification confidence of the located target in each frame of the standardized image sequence; the target trajectory tracking multi-angle information acquisition module records the relative movement trajectory of the same target of interest through target tracking and then calculates a multi-angle keyframe sequence; and the target multi-angle information fusion and analysis module performs three-dimensional fusion modeling on the multi-angle keyframe sequence of the same target. The target detection and localization module uses a binary classification network for target classification, and uses the class probability value output by the fully connected layer of the convolutional network as the classification confidence. The aforementioned method for multi-angle fusion analysis of ultrasound images includes: Step 1: Perform image noise reduction, image contrast enhancement, and image standardization on the raw ultrasound images; Step 2: Calculate the target localization and target classification confidence of each frame in the standardized image sequence and extract all candidate keyframes; Step 3: Based on the target detection and localization bounding box obtained in the target detection and localization step, maintain a trajectory chain list of multiple identified targets in real time. By tracking the movement trajectory of the same target under the ultrasonic probe, capture key frames of the target from multiple angles. Step 4: Perform 3D fusion modeling on keyframes of the same target from multiple angles and analyze its stereo characteristics, specifically including: 4.1) Obtain the target's two-dimensional contour in each keyframe using an edge segmentation operator. Specifically: First, read all keyframe sequences of all known targets, and crop the target region images sequentially. Each cropped image contains only the outermost contour of a single target. Use an edge detection operator to extract the target edges, resulting in a target segmentation map that only displays the target region. Use a scale-invariant feature transformation method to extract feature points and corresponding feature description vectors containing the mean gray intensity and gray distribution direction vector of each feature point in the target segmentation map sequence. Specifically: , , ,in: Original image exist Gaussian scale feature map under blurred scale Original image The corresponding pixel-level feature point distribution gradient map, express exist Gradient magnitude at location, express exist The gradient direction at the location is used to obtain the position of each pixel-level feature point within an image. Scale, i.e., fuzzy proportion ,strength and direction ,use If a pixel block is used as a feature point for actual reference and matching, then let This represents the feature vector at the k-th pixel in the image. Represents any in the diagram The eigenvectors of a feature point are expressed as follows: , ; 4.2) Extract fixed feature points of each target 2D contour and match their relative positions in 3D space. That is, combine the matching method of minimum difference distance between feature description vectors to confirm the relative positional relationship of feature points in each image. Specifically: This indicates that another image is to be... Matching comparison Feature points, corresponding pixel feature vectors , ,Depend on The matching degree of feature points is calculated based on the shortest distance between vectors. The method for selecting feature points when calculating the matching degree is to traverse all feature points in the entire image. 4.3) Map all feature points sequentially onto the 3D model until all feature points have been mapped. Specifically, using the structure-of-motion reconstruction method, the target segmentation map that has been located in 3D space is sequentially mapped onto a view of the 3D pixel matrix model to be generated from a certain angle, according to the time sequence of its acquisition. This process is repeated for all target segmentation maps generated from keyframe sequences until the 3D reconstruction of the target model is complete. Specifically, for the same point of interest in the target in 3D space... This will yield the matching points of the target in all 2D images. For any two most matching eigenvectors, the intrinsic matrix will be used to... Seeking and The angle and distance relationships in three-dimensional space are used to sequentially calculate the mapping relationship of all pixels, specifically: Each pixel is mapped using traditional 3D model generation methods. The position and corresponding gray value in three-dimensional space.

2. The method for multi-angle fusion analysis of ultrasound image targets according to claim 1, characterized in that, Step two specifically includes: 2.1) Feature extraction is performed on the preprocessed image using convolutional layers; 2.2) Use fully connected layers to classify the target using the feature vectors; 2.3) Calculate the confidence score of the benign or malignant classification of the target of interest in each frame. The image with a high confidence score for any identification category is selected as a high-quality frame in terms of image quality and observation angle. The classification result of the target is not used as any opinion or suggestion for external output, but only to increase the reliability of the classification confidence score by refining the classification categories. In the aforementioned multi-angle keyframe capture of the target, for each target of interest identified in the ultrasound image, five target tracking states are defined: target start state, target update state, target loss state, target rematch state, and scan end state.

3. The method for multi-angle fusion analysis of ultrasound image targets according to claim 1, characterized in that, Step three specifically includes: 3.1) Target Startup: Receive the new target location output by the target detection and localization module, establish a trajectory recording list and a keyframe recording list for it; label the newly located target, the label containing the target number and benign / malignant classification; record the first coordinate of the target as (0, 0), which is used to represent its position on the x / y axis of the image; 3.2) Target update: Due to the spatial correlation between adjacent frames of ultrasound images, there is a certain overlap between the target location box in one frame and the previous frame. By calculating whether the intersection-union ratio of the two location boxes reaches the threshold, it can be determined that the two target recognition results are the same target. 3.3) Target loss handling: When a target is lost, the existing trajectory table and key image sequence of the lost target will be saved. When the total number of targets that appear simultaneously in the current frame is less than the maximum number of targets that can appear simultaneously, once a new target is identified, the newly appearing target will be assumed to be the previously discarded target, and the target rematching state of the target will be activated. 3.4) Target rematch: For newly appearing targets, perform correlation filtering on all keyframes in the keyframe list left by the lost targets. When the gray value energy of the filtered image reaches a specified threshold, the new target is considered to be the same target as a previously lost target; otherwise, it is considered a new target and is allowed to enter the target initial state of the new target. 3.5) Scan ends: The track list of the tracked target and the multi-angle keyframe list of the captured target are transmitted to the target multi-angle information fusion analysis module.

4. The method for multi-angle fusion analysis of ultrasound image targets according to claim 3, characterized in that, The Cross-Union Ratio (CUI) is calculated based on pixel proportions. This is based on the default probe's fastest moving speed V (cm / s), the average frame rate F (1 / s) of image acquisition within the frame update interval, the ultrasound image resolution E (pixels / cm), and the corresponding target frame size L*W. Specifically, the proportion threshold is calculated by multiplying the length and width in units of pixels. The formula for calculating the CUI threshold M is used to determine the proportion of pixels between two targets. , The expression for determining whether they are the same target is: , ,in: This is a state variable describing the situation of targets being the same. When the calculated intersection-union ratio (IU) is greater than or equal to M, the two identified targets in the next frame (i.e., the i-th frame and the (i+1)-th frame) are considered to be the same target. ; Once the new target is confirmed to be the same target as the target in the previous frame, the target trajectory list is updated, and the relative pixel distance and displacement vector between the target in the current frame and the target in the first frame are calculated. If the relative pixel distance and displacement vector reach the specified threshold, it is considered to belong to a key frame at a different angle, and the key frame is added to the key frame sequence.

5. The method for multi-angle fusion analysis of ultrasound image targets according to claim 3, characterized in that, The correlation filtering method first converts the region of interest into the frequency domain using a two-dimensional fast Fourier transform, then multiplies the two frequency domain images to obtain the correlation. The quantized value of the matching degree is obtained by calculating the average intensity of the mid-to-high frequency components in the multiplied image, as shown below. and For the two targets to be matched, These are state variables that describe the target matching status. A value of 1 indicates that the targets in the two images match. .

6. The method for multi-angle fusion analysis of ultrasound image targets according to claim 1, characterized in that, in After reconstructing the target's three-dimensional model, the target multi-angle information fusion analysis module analyzes the internal pixel characteristics and external contour characteristics of the target's three-dimensional model to obtain the uniformity, average value, and regularity of the pixel value distribution within the target area of ​​the ultrasound image. Specifically, it uses the intensity of its mid-to-high frequency components in the three-dimensional frequency domain to determine the smoothness of its contour; and uses its three-dimensional gray-level co-occurrence matrix to calculate the uniformity of the internal gray-level value distribution and the average gray-level value.

7. A multi-angle fusion analysis system for ultrasound images of a target implementing the method of any one of claims 1-6, characterized in that, include: Application layer, business processing layer, data layer, video stream inlet for inputting thyroid ultrasound images, and output port for outputting real-time localization boxes of identified target nodules, reconstruction models, and analysis content; The business processing layer includes: an image preprocessing unit, a target detection and localization unit, a target trajectory tracking unit, and a multi-angle information fusion and analysis unit. Specifically: the image preprocessing unit performs bilateral filtering on each image in the input ultrasound image sequence to remove high-frequency noise without interfering with image edge information. Then, it uses a nonlinear contrast enhancement method based on exponential transform to enhance the contrast of the denoised image, making the target's edge and internal features more prominent. Finally, it uses Z-score normalization to map the original grayscale values ​​to a skewed distribution with a mean of 0 and a variance of 1, outputting the preprocessed image sequence. The target detection and localization unit changes the RGB three-channel image input to a single grayscale (G) image input and uses neural network pruning to obtain a binary classification optimization network. The target trajectory tracking unit uses the NumPy library... To quickly calculate the IOU (Intersection over Union) to identify continuous targets, the FFT function in NumPy is used to implement a Fast Fourier Transform to match lost targets. Euclidean distance is used to directly calculate the target's movement distance using vector distance. The target multi-angle information fusion analysis unit uses the Canny operator to segment the target nodule contour, turning the area outside the target contour in the image black to obtain the target segmentation map. After obtaining the target segmentation map, the OpenCV and Math libraries are used to analyze all corner features and feature vectors of the nodule segmentation map. Then, the built-in SFM function of the OpenCV library is used to reconstruct the target's 3D model. Finally, the 3D-FFT function implemented in NumPy is used to calculate the frequency domain distribution of the target's 3D model and to determine whether the gray value intensity of the mid-to-high frequency components reaches the threshold of irregular edges. The matrix operation function of NumPy is used to obtain the value of each element in the 3D gray-level co-occurrence matrix, and then it is determined whether the specified element value representing uniformity reaches the threshold.

8. The ultrasound image target multi-angle fusion analysis system according to claim 7, characterized in that, During the pruning process, semantic feature layer pruning is used, while the underlying spatial feature layer is preserved; When training the network, image cropping, image flipping, and image stretching are used as data augmentation methods. Salt-and-pepper noise is then added to form interference samples for training to improve the network's anti-interference ability.

9. The ultrasound image target multi-angle fusion analysis system according to claim 7, characterized in that, The coordinates of the positioning box and the target classification confidence array are obtained by reading the underlying module of the YOLO5 library and output to the target trajectory tracking multi-angle information acquisition module; The reconstruction model and analysis content are saved into the MongoDB database through the data layer interface and returned to the application layer window for display through the service interface.

Images