Right heart acoustic radiography examination process standardization system

Abstract

The invention relates to the technical field of right heart acoustic radiography image analysis, in particular to a right heart acoustic radiography examination process standardization system. The method comprises the following steps: firstly, acquiring a multi-view dynamic ultrasonic image sequence of the right heart through ultrasonic equipment according to industrial standard operation specifications; then preprocessing the image sequence, extracting right heart function related data by adopting an image processing algorithm, and analyzing wall motion and left heart bubbles; acquiring a global anomaly recognition result through an anomaly recognition algorithm, and positioning an anomaly region by means of a YOLO algorithm and generating a local anomaly recognition result; and finally, the identification result and the metadata are transmitted to a display device for medical staff to check and analyze. In addition, encryption storage and access right control also ensure the security of the check data in the storage and transmission process. According to the method, the radiography data analysis speed and accuracy are improved, and then the right heart acoustic radiography image analysis quality and efficiency are improved.

Description

Technical Field

[0001] The invention relates to the technical field of right heart acoustic contrast imaging image analysis, and in particular to a right heart acoustic contrast imaging examination process standardization system. Background Art

[0002] Right heart echocardiography is an examination method that uses ultrasonic technology to image the right atrium and right ventricle. It is mainly used to evaluate the structure and function of the right heart to detect possible lesions or abnormalities, detect pulmonary hypertension by measuring pressure changes in the right ventricle and right atrium, and evaluate the effects of drugs or other treatments on right heart function.

[0003] The main process of right heart acoustic contrast examination includes: in the preparation stage, the patient lies in a supine or left lateral position, adjusts the appropriate probe and frequency and makes preliminary settings; in the imaging stage, images of the right heart structure are obtained from different perspectives, during which the patient is given intravenous contrast agent to enhance image contrast; in the data analysis stage, the size, systolic function, diastolic function of the right ventricle and the pressure estimation of the right atrium are measured and recorded; finally, the medical staff makes the final summary of the examination.

[0004] In the right heart acoustic contrast examination process, the extraction and analysis of contrast data is one of the more important steps. In the prior art, the extraction of right heart function parameters mainly relies on the built-in tools of the ultrasound equipment for measurement, and there are errors caused by objective factors; and the existing image processing methods do not carry out targeted abnormal auxiliary identification and positioning based on the particularity of ultrasound images. Therefore, it is necessary to construct an image data processing and analysis model to help improve the speed and accuracy of right heart acoustic contrast image data processing and analysis, thereby improving the quality and efficiency of information processing in the contrast examination process.

[0005] To this end, the present invention proposes a right heart acoustic contrast examination process standardization system. Summary of the invention

[0006] The purpose of the present invention is to provide a standardization system for the right heart acoustic contrast examination process. First, according to the industry standard operating specifications, a multi-view dynamic ultrasound image sequence of the right heart is collected by an ultrasound device; then the image sequence is preprocessed, and the image processing algorithm is used to extract right heart function-related data, wall motion analysis and left heart bubble analysis; the global abnormality recognition result is obtained through the abnormality recognition algorithm, and the abnormal area is located with the help of the YOLO algorithm and the local abnormality recognition result is generated; finally, the recognition result and metadata are transmitted to a display device for medical personnel to view and analyze.

[0007] To achieve the above-mentioned object, the present invention provides a right heart acoustic contrast examination process standardization system, comprising an imaging acquisition unit, a data analysis unit, a data integration unit and a security storage unit;

[0008] The imaging acquisition unit is used to set ultrasound parameters and use ultrasound equipment to obtain a multi-view dynamic ultrasound image sequence, which is recorded as a first image sequence; the ultrasound image sequence includes right atrium imaging, right ventricle imaging, related valve imaging and atrial septum imaging;

[0009] The data analysis unit is used to analyze the first image sequence, including image preprocessing, extraction of right heart function related data, global abnormality analysis and abnormal area positioning; perform noise reduction, contrast enhancement and geometric correction on the first image sequence to obtain a second image sequence;

[0010] Using an image processing algorithm to analyze the right heart images of a specific period in the second image sequence, extracting right heart function parameters, wall motion analysis results, and left heart bubble analysis results;

[0011] Extracting the global feature vector in the second image sequence through a 3D convolutional neural network, calculating the probability of each abnormal category in the frame image using a fully connected layer, and obtaining a global abnormality recognition result; locating the abnormal area by presetting a real bounding box and generating a local abnormality recognition result;

[0012] The data integration unit is used to merge the first image sequence, the abnormality identification results and the right heart function related data into reference data and transmit them to the display device; the secure storage unit is used to encrypt, store and control access rights of the right heart function related data, abnormality identification results and related metadata.

[0013] Furthermore, the step of acquiring the multi-view dynamic ultrasound image sequence through an ultrasound device includes: setting ultrasound parameters, acquiring the multi-view dynamic ultrasound image sequence, including the right atrium imaging, the right ventricle imaging, the related valve imaging and the atrial septum imaging; the atrial septum imaging includes imaging before and after the Valsalva maneuver; adjusting the angle and position of the ultrasound device, changing different viewpoints to acquire ultrasound images multiple times, which are recorded as the first image sequence; and recording the number and imaging time of the first image sequence.

[0014] Furthermore, the image preprocessing step includes: for the first image sequence V={I1, I2, ..., I T}decomposed into separate frame images I t , where I t is the t-th frame image, T is the total number of frames; for each frame I t Gaussian filtering is used to reduce high-frequency noise, and histogram equalization is used to enhance image contrast and highlight cardiac structures;

[0015] For contrast-enhanced frames Detect key feature points and convert the frame and adjacent frames Perform feature point matching; by calculating the transformation matrix T t , the frame Align to the reference frame; apply the transformation matrix to perform geometric correction on each frame to obtain the second image sequence

[0016] Further, the step of extracting right heart function related data includes right heart function parameter extraction and wall motion analysis;

[0017] A frame of the right atrium at the end of contraction in the second image sequence within a cardiac cycle is selected, and the contour of the right atrium is obtained by edge detection and contour extraction algorithms, and a contour point set of the right atrium is extracted; the maximum pixel distance of the right atrium in the vertical and horizontal directions is calculated, and converted into the actual upper and lower diameters of the right atrium and the long diameter of the right atrium

[0018] Selecting a frame of the right ventricle at the end of diastole in the second image sequence within a cardiac cycle, detecting the straight line edge of the ventricular inner wall by Hough straight line transform, calculating the pixel distance of the right ventricular inner diameter and converting it into the actual distance;

[0019] Capture frames of the valve in the maximum opening state in the second image sequence within a cardiac cycle, obtain an edge map of the valve area, extract a contour point sequence of the valve opening, and calculate the valve opening area using a contour integration method.

[0020] The process of the wall motion analysis includes: calculating the upper and lower diameters and long diameters of the right atrium, the right ventricular internal diameter and the valve opening area of ​​each frame in the second image sequence;

[0021] The pixel motion between adjacent frames of the second image sequence is calculated, and the displacement and velocity are calculated based on the characteristic points on the myocardium. The radial and longitudinal strains and strain rates of the myocardium are calculated and acquired through the displacement and velocity.

[0022] Furthermore, the process of left heart bubble analysis includes: segmenting the atrial septum boundary of the second image sequence using U-Net to locate the boundary of the foramen ovale region; detecting the position of the bubble of the second image sequence by a brightness threshold segmentation method to generate a binary image;

[0023] In the first frame where the bubble is detected, a unique identifier is assigned to each bubble and a bubble trajectory is initialized, the displacement of the bubble between adjacent frames is calculated by an optical flow method, and the bubble trajectory is updated;

[0024] When no displacement change of the bubble is detected in several consecutive frames, the bubble trajectory is terminated; the starting point coordinates and the end point coordinates of all the bubble trajectories are determined; and the bubble whose starting point coordinates are located in the right heart area and the end point coordinates are located in the left heart area is marked as a bubble to be determined;

[0025] Calculating the positional relationship between the coordinates of the bubble to be determined in the corresponding bubble trajectory and the oval foramen area, and judging whether the bubble trajectory passes through the oval foramen area according to the positional relationship;

[0026] The bubbles to be determined that pass through the foramen ovale region on the bubble trajectory are counted to generate the left heart bubble analysis result.

[0027] Furthermore, the step of global anomaly analysis includes: processing the second image sequence into a time frame sequence X∈R T×H×W×C Input into a neural network consisting of three layers of 3D convolution to obtain the global feature vector F∈R d ; Where H, W and C are the image height, image width and number of channels respectively, and d is the feature dimension.

[0028] The 3D convolution kernel size of the three-layer convolution is 3×3×3, and the number is 32, 64, and 128 respectively; the three-layer convolution all adopts maximum pooling and ReLU activation function;

[0029] The global feature vector F∈R d Calculate the score z of each abnormal category through the fully connected layer i , the probability distribution P of each abnormal category is calculated by the softmax function i ;

[0030]

[0031] Among them, w i is the weight vector of the ith abnormal category, e is the base of the natural logarithm, K is the number of abnormal categories, b i is the bias term of the i-th abnormal category; set the threshold, when P i >T, then determine the time frame sequence X∈R T ×H×W×C There is an i-th type of anomaly in .

[0032] Furthermore, for the second image sequence, each frame of image Adjust to a fixed size of 448×448, divide into S×S grid units, predict B bounding boxes for each grid; match the bounding boxes at the same position in adjacent frames;

[0033] The probability that each of the bounding boxes belongs to each anomaly category is calculated by the YOLOv5 algorithm; the confidence score of each of the bounding boxes for a specific anomaly category is calculated;

[0034] Eliminate redundant overlapping bounding boxes by non-maximum suppression, retain the bounding box with the highest confidence and transform the coordinates to frame I t aligned The actual coordinates on the image size are used to draw a rectangle to mark the abnormal area according to the actual coordinates and the size of the frame image, and the abnormal category and the corresponding confidence score are marked near the bounding box.

[0035] Furthermore, the steps of the data integration unit include: merging the first image sequence, the second image sequence and the second image sequence with abnormal area annotations into an image set; organizing the superior and inferior diameters and long diameters of the right atrium, the inner diameter of the right ventricle, the valve opening area, the wall motion analysis results and the left heart bubble analysis results into a structured table; merging the image set and the structured table into the reference data and transmitting them to the display device.

[0036] Furthermore, the steps of encrypted storage and access permission control include:

[0037] The right heart function parameter, the wall motion analysis result, the left heart bubble analysis result, the global abnormality recognition result, the local abnormality recognition result and related metadata are structured and stored in a private cloud database, and backed up in a local database; the related metadata includes the numbers and imaging times of the first image sequence, the second image sequence and the first image sequence;

[0038] Furthermore, the AES-256 symmetric encryption algorithm is used to encrypt and store the subject's metadata and examination results; a secure transmission protocol is used to encrypt communications during data transmission; access rights are defined for roles in the right heart acoustic contrast examination process, and when access to the subject's metadata and examination results is requested, the visitor's role permissions are checked to determine whether access is allowed.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. In the imaging acquisition unit of the present invention, the multi-view dynamic ultrasound image sequence of the right heart is sequentially acquired by ultrasound equipment, including imaging of the right atrium, right ventricle, related valves and atrial septum. Each view acquires an image sequence of at least one cardiac cycle to ensure the data reliability of subsequent analysis; then the acquired ultrasound image is preprocessed, and the image sequence of the same view is filtered, denoised, histogram equalized, enhanced and inter-frame corrected to obtain a second image sequence; the right heart function parameters, wall motion analysis and left heart bubble analysis are extracted from the second image sequence, including the upper and lower diameters and long diameters of the right atrium, the inner diameter of the right ventricle, the valve opening area, strain and strain rate, and the number of bubbles from the right heart through the oval foramen to the left heart. The present invention acquires rich right heart structure information through multi-view image acquisition and multi-dimensional analysis, providing high-quality input for subsequent abnormality recognition analysis.

[0041] 2. According to the second image sequence of multiple perspectives, the present invention obtains the global feature vector through a neural network including three layers of 3D convolution, and then uses the fully connected layer and the softmax function to calculate the probability distribution of each abnormal category in the global image to obtain the global abnormality recognition result; then the YOLO algorithm is introduced to divide each frame of the second image sequence into multiple grid units, each grid unit predicts multiple bounding boxes, and the probability and confidence score of each bounding box belonging to each abnormal category are calculated according to the special global feature vector; finally, the abnormal area on the second image sequence is identified to generate the local abnormality recognition result. The present invention can quickly detect the possible abnormalities in the ultrasound image through global abnormality recognition, thereby improving the detection efficiency; the local abnormality positioning accurately marks the position of the abnormal area. The combination of global and local abnormality detection realizes the comprehensive analysis of the angiography data, improves the accuracy and reliability of abnormality recognition, and provides data and theoretical support for improving the quality and efficiency of right heart acoustic contrast image analysis.

[0042] 3. The present invention provides a standardized system for the right heart acoustic contrast examination process, including an imaging acquisition unit, a data analysis unit, a data integration unit and a secure storage unit. The imaging acquisition unit uses an ultrasound device to obtain a multi-view dynamic ultrasound image sequence of the right heart in accordance with industry standard operating specifications; the data analysis unit extracts right heart function-related data, performs left heart bubble analysis, global abnormality analysis and abnormal area positioning on the dynamic ultrasound image sequence to obtain abnormality identification results; the data integration unit transmits right heart function-related data, left heart bubble analysis results, abnormality identification results and metadata to a display device for medical personnel to view and analyze; the secure storage unit records and stores relevant data and performs encrypted storage and access control on the data. The present invention realizes the automation of image data processing and mining in the examination process, reduces the impact of human differences on the quality of image analysis; through the collaborative work of multiple units, the overall quality and efficiency of right heart acoustic contrast data analysis are improved, and can provide a basis for auxiliary decision-making for medical personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The figure is a flowchart of a standardization system for the right heart echocardiography examination process;

[0044] Figure 2 The present invention is a schematic diagram of the structure of a standardization system for the right heart acoustic contrast examination process;

[0045] Figure 3 This is a structural diagram of the image data processing model. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The present invention provides a right heart acoustic contrast examination process standardization system, the specific system flow chart and structural schematic diagram refer to Figure 1 and Figure 2 .

[0048] Embodiment 1

[0049] As an embodiment of the present invention, refer to Figure 1 S10 in the embodiment is applied to an imaging acquisition unit of a right heart acoustic contrast examination process standardization system. The imaging acquisition unit is used to obtain a multi-view dynamic ultrasound image sequence using ultrasound equipment according to industry standard operating specifications; the ultrasound image sequence includes right atrium imaging, right ventricle imaging and related valve imaging.

[0050] Specific:

[0051] The ultrasound parameters are set according to the industry standard operating specifications, and the multi-view dynamic ultrasound image sequence is acquired using ultrasound equipment, including right atrium imaging, right ventricle imaging, atrial septum imaging and related valve imaging; atrial septum imaging includes complete cycle imaging before and after the Valsalva maneuver.

[0052] The angle and position of the ultrasound device are adjusted, and ultrasound images are acquired multiple times by changing different viewing angles, which are recorded as a first image sequence; and the number and imaging time of the first image sequence are recorded.

[0053] In this embodiment, right heart function parameter extraction, wall motion analysis and left heart bubble analysis are performed on the historical multi-view dynamic ultrasound image sequence in the database of the inspection unit A; the historical multi-view dynamic ultrasound image sequence is collected by a Doppler ultrasound diagnostic instrument, the model of the ultrasound diagnostic instrument is Philips IE33, the frame rate is set to 60, the image resolution is 0.02 cm / pixel, and the image size is 512×512.

[0054] This embodiment uses ultrasound equipment to collect a multi-view dynamic ultrasound image sequence of the right heart, laying a data foundation for subsequent data analysis; reasonable ultrasound parameter settings and systematic data recording can ensure the consistency of image data quality; multiple images are collected from different viewpoints to ensure that images of at least one cardiac cycle are collected, which can improve the reliability of subsequent data analysis.

[0055] Reference Figure 3 , Figure 3 Corresponds to Figure 1 S30 in the above method is applied to the image data analysis processing model proposed by the data analysis unit. For the first image sequence, the image data processing analysis model is used to perform right heart function parameter extraction, wall motion analysis, left heart bubble analysis, global abnormality recognition and local abnormality recognition.

[0056] This embodiment uses the historical multi-view dynamic ultrasound image sequences in the database of the inspection unit A as training data to train the image data processing and analysis model, including the right heart function data model, the left heart bubble analysis model, the global abnormality analysis model and the local abnormality analysis model. The annotations in the training data are completed by those skilled in the art.

[0057] Specifically, refer to Figure 1 S20 in the figure corresponds to the image preprocessing step of the data analysis unit: for the first image sequence V={I1,I2,...,I T}, decompose it into separate frame images I t , where I tis the t-th frame image, T is the total number of frames; for each frame I t Gaussian filtering is used to reduce high-frequency noise, and histogram equalization is used to enhance image contrast and highlight the cardiac structure. The fast robust feature algorithm (ORB) is used to detect key feature points;

[0058]

[0059] Among them, P t is the feature point set of the tth frame, K t The number of feature points detected for the tth frame; is the pixel coordinate of the i-th feature point detected in the t-th frame.

[0060] The frame and adjacent frames Feature point matching is performed by combining cross-validation matching with the RANSAC method; the process of matching feature points of adjacent frames is shown in the following formula;

[0061]

[0062] M={(p i ,q j )|Match A→B (i) = j and Match B→A (j) = i};

[0063] The above formula describes the process of matching feature points between image A and image B. i is the i-th feature point in image A, q j is the jth feature point in image B, d i is the feature point p i The descriptor of j is the feature point q j The descriptor is used to describe the local image information of the feature point. i ,e j ) represents the feature descriptor d i and e j The Euclidean distance between i ,e j ) is smaller, indicating that the feature point p i and q j The more similar; This means that dist(d i ,e j )The smallest j Match A→B (i) represents the i-th feature point p in image A i Matching results in image B.

[0064] By calculating the transformation matrix T t , the frame Align to the reference frame; apply the transformation matrix to perform geometric correction on each frame to obtain the corrected second image sequence

[0065] This embodiment can help improve the accuracy and efficiency of subsequent analysis by preprocessing the dynamic ultrasound image sequence. Gaussian filtering makes the image clearer by reducing noise, and histogram equalization highlights the cardiac structure by enhancing image contrast. It is convenient for the image processing method in the subsequent analysis model to extract key features; through feature point matching and frame alignment, the consistency of images under different viewing angles is ensured, providing high-quality data input for the subsequent analysis model.

[0066] Furthermore, the step of training the right heart function data model to extract right heart function related data includes right heart function parameter extraction and wall motion analysis; the training process of the right heart function data model is:

[0067] Select a frame at the end of right atrial contraction in the second image sequence within a cardiac cycle The edge detection and contour extraction algorithm is used to obtain the contour of the right atrium and extract the contour point set C of the right atrium. RA ={(x i ,y i )}; Calculate the maximum distance of the right atrium in the vertical and horizontal directions and convert it into the actual upper and lower diameters of the right atrium and the long diameter of the right atrium; the upper and lower diameters of the right atrium D SI and right atrium diameter D L The pixel distance calculation formula is:

[0068] D SI =max(y i )-min(y i ),D L =max(x i )-min(x i );

[0069] In this formula, (x i ,y i ) represents the coordinates of the contour points.

[0070] By D SI,actual =D SI ×r,D L,actual =D L ×rD SI and D L The pixel distance is converted into the actual right atrium upper and lower diameter D SI and right atrium diameter D L, where r is the frame image resolution.

[0071] Select a frame at the end of right ventricular diastole in the second image sequence within a cardiac cycle. The edge image E is obtained by edge detection ED ; For edge image E ED Perform Hough straight line transformation to detect the straight line edge of the ventricular inner wall, calculate the pixel distance of the right ventricular inner diameter and convert it into actual distance;

[0072] The calculation process of the pixel distance of the right ventricular diameter is as follows: select two straight lines representing the ventricular inner wall, calculate the intersection point between the two lines on a certain row and a certain column, and obtain the coordinates of the two points (x1, y1) and (x2, y2); calculate the pixel distance D between the two points RV ;

[0073]

[0074] By D RV,actual =D RV ×r converts the pixel distance of the right ventricular internal diameter into the actual distance.

[0075] Capturing a frame in which the valve is in a maximum opening state in the second image sequence within a cardiac cycle Obtain edge image E of valve area by edge detection valve ; Extract the contour point sequence of the valve opening and calculate the valve opening area using the contour integration method.

[0076]

[0077] In the above formula, in this formula, Z is the number of contour points of the valve opening; j ,y j ) represents the coordinates of the contour points.

[0078] By A valve,actual =A valve × 2 The pixel area A RA Converted to actual valve opening area.

[0079] This embodiment first performs edge detection on the contour of the right atrium to obtain clear structural features, thereby calculating the upper and lower diameters and long diameters of the right atrium; then uses Hough linear changes to detect the inner wall of the right ventricle and accurately measures the inner diameter of the right ventricle; finally, the valve opening area is calculated by capturing the image of the valve in the maximum opening state and performing contour integration. This method measures and calculates right heart function parameters through image analysis methods, reduces the errors caused by the use of built-in tools of ultrasound equipment and differences in operators in the existing technology, and provides accurate data support for improving the quality of right heart acoustic contrast examination.

[0080] The process of the wall motion analysis includes: repeating the above method to calculate the upper and lower diameters and long diameters of the right atrium, the inner diameter of the right ventricle and the valve opening area of ​​each frame in the second image sequence.

[0081] Feature points are selected in the myocardial area, and the optical flow algorithm is used to calculate the displacement and velocity of the feature points between adjacent frames. Then the radial and longitudinal strains and strain rates of the myocardium are calculated.

[0082] For feature point m, calculate its distance from the initial frame to the specified reference point For each frame t, calculate the current distance from feature point m to the reference point Calculate radial strain and strain rate based on the current distance and reference point position in each frame;

[0083]

[0084] In the above formula, is the radial strain, is the radial strain rate; Δt is the frame interval, satisfying f is the frame rate. The calculation method of longitudinal strain and strain rate is the same as that of radial strain and strain rate.

[0085] This embodiment uses the above right heart function data model to process and analyze the multi-view dynamic ultrasound image sequences of 126 subjects, and some examples of the acquired right heart function related data are given in Tables 1 and 2;

[0086] Table 1 Some examples of right heart function parameters of the subjects

[0087]

[0088] Table 2 Partial examples of wall motion analysis of subjects

[0089]

[0090] It should be noted that the right heart function related parameters and wall motion analysis data extracted by the method proposed in this embodiment are not all data. In actual use, the data that can be extracted include but are not limited to the data categories listed in this embodiment.

[0091] This embodiment provides more abundant reference data for right heart acoustic contrast examination by analyzing the wall motion of the right heart. By repeatedly calculating the upper and lower diameters and long diameters of the right atrium, the right ventricular internal diameter and the valve opening area, each frame can be Thrs bt Accurately capture changes in cardiac structure; use optical flow algorithm to calculate the displacement and velocity of feature points, and then further calculate the radial and longitudinal strains of the myocardium. This method enriches the quantitative indicators for evaluating cardiac function and provides quantitative data support for improving the quality of right heart acoustic angiography examinations.

[0092] Furthermore, the training process of the left heart bubble analysis model is: using the U-Net segmentation model to segment the atrial septum boundary of the atrial septum image in the second image sequence, and using the atrial septum boundary data to locate the foramen ovale region.

[0093] Set the brightness threshold Thrs bt , the pixels in each frame are binarized according to the brightness. The area with pixel value 1 represents the bubble, and the area with pixel value 0 represents the background;

[0094]

[0095] In this formula, I(x,y) is the brightness value of the image at the coordinate (x,y), I binary (x, y) is the pixel value at the coordinate (x, y) in the binary image. In each frame, the position of each bubble is detected; in the frame where the bubble is detected for the first time, a unique identifier ID is assigned to each bubble and its bubble trajectory is initialized; the initial coordinates (x0, y0) and frame number t0 of each bubble are stored.

[0096] The displacement of the bubble between adjacent frames is calculated using the optical flow method. The step includes: setting (u, v) as the optical flow displacement vector of the bubble, satisfying the optical flow constraint equation I(x+u,y+v,t+1)=I(x,y,t); wherein I(x,y,t) is the pixel value of the coordinate (x,y) in the tth frame, and I(x+u,y+v,t+1) is the pixel value of the coordinate (x,y) in the t+1th frame.

[0097] By solving the optical flow constraint equation, we can get the new position (x t ,y t ); Set the current position of the bubble (x t ,y t ) and frame number tt Update to the bubble trajectory. When the bubble is not detected to have coordinate changes in N consecutive frames, that is, I(x, y, t) of N consecutive frames has not changed, the bubble trajectory is determined to be terminated and saved; extract the starting point coordinates and the end point coordinates of all bubble trajectories, and determine whether the starting point coordinates are located in the right heart area and whether the end point coordinates are located in the left heart area. If so, mark the bubble as a bubble to be determined.

[0098] Define the foramen ovale area as R hole , whose center coordinates are (x hole ,x hole ), with a radius of r hole ; The position of the bubble vb1 to be judged in the t frame is P(t)=(x t ,y t ), determine whether vb1 is located in the foramen ovale area; the judgment condition is:

[0099]

[0100] If it is satisfied, the bubble vb1 is considered to be a bubble from the right heart to the left heart through the foramen ovale, count(vb)+1, count(vb) is the left heart bubble analysis result. This embodiment uses the above left heart bubble analysis model to process and analyze the atrial septum images of 126 subjects, and some examples of the obtained left heart bubble analysis results are given in Table 3;

[0101] Table 3 Some examples of left heart bubble analysis results

[0102]

[0103] This embodiment first performs boundary segmentation on the atrial septum image sequence before and after the Valsalva maneuver and defines the foramen ovale area, and combines the brightness threshold method to perform bubble binarization processing, which can accurately detect and mark the bubble position; then the optical flow method is used to track the bubble trajectory, and combined with the area range of the foramen ovale, it is determined whether the bubble is a bubble that enters the left heart from the right heart through the foramen ovale; finally, the number of such bubbles is used as the left heart bubble analysis result, enriching the data presentation of right heart angiography examination.

[0104] Furthermore, the training process of the global anomaly analysis model is as follows: the second image sequence is processed into a time frame sequence X∈R T×H×W×C Input into a neural network consisting of three layers of 3D convolution to obtain the global feature vector F∈R d ; Where H, W and C are the image height, image width and number of channels respectively, and d is the feature dimension.

[0105] The 3D convolution kernel size of the three-layer convolution is 3×3×3, and the number is 32, 64, and 128 respectively; the three-layer convolution all adopts maximum pooling and ReLU activation function;

[0106] The global feature vector F∈R d Calculate the score z of each abnormal category through the fully connected layer i , the probability distribution P of each abnormal category is calculated by the softmax function i ;

[0107]

[0108] Among them, w i is the weight vector of the ith abnormal category, K is the number of abnormal categories, b i is the bias term of the i-th abnormal category;

[0109] Set the threshold value, when P i >T, then determine the time frame sequence X∈R T×H×W×C There is an i-th type of abnormality in this embodiment. This embodiment uses the above global abnormality analysis model to process and analyze the multi-view dynamic ultrasound image sequences of 126 subjects, and some examples of the obtained global abnormality analysis results are given in Table 4;

[0110] Table 4 Some examples of global anomaly analysis results

[0111]

[0112] According to the global abnormality analysis results in Table 4, compared with normal right heart ultrasound images, the upper and lower diameters and long diameters of the right atrium (RA) of subject P012 were larger, the valve opening area was smaller, and the other parameters were normal; therefore, the global abnormality category of the ultrasound image of subject P012 was A2. However, the upper and lower diameters and long diameters of the right atrium of subject P014 were larger, the inner diameter of the right ventricle (RV) was larger, the valve opening area, radial strain and longitudinal strain were smaller, so the global abnormality category of the ultrasound image of subject P014 was A4.

[0113] This embodiment can provide data reference for medical staff more efficiently by building a global abnormality analysis model to identify the global abnormality category to which the image sequence belongs. The three-layer 3D convolutional neural network can effectively capture the temporal and spatial features in the image sequence, thereby extracting a global feature vector with high-dimensional information; the score of each abnormal category is calculated through the fully connected layer, and the probability distribution is generated using the softmax function, which can provide an efficient and reliable global abnormality analysis tool for right heart acoustic contrast examination.

[0114] Furthermore, the training process of the local abnormality analysis model is: for the historical multi-view dynamic ultrasound image sequence in the inspection unit A database, the abnormal area is located by the YOLO algorithm, and the local abnormality recognition result is generated. Specifically:

[0115] For the second image sequence, each frame of image Resize to a fixed size of 448×448 and normalize the pixel values ​​to the range [0,1]. The corresponding annotations contain the true bounding box and the anomaly category.

[0116] The ground-truth bounding box is represented as (x center ,y center ,w,h,c), where (x center ,y center ) are the coordinates of the center point of the bounding box, w and h are the width and height of the bounding box respectively, and c is the one-hot encoding vector corresponding to the different categories.

[0117] In this embodiment, the input layer is 448×448×3, and the YOLOv5 deep neural network is used as the basic architecture to convert the frame image It is divided into 14×14 grid cells, and each grid predicts 3 bounding boxes.

[0118] The output of YOLOv5 is 14×14×(3×5+K), where K is the number of abnormal categories and 5 means that each predicted bounding box contains 5 parameters (b x ,b y ,b w ,b h ,b C ), b x and b y represents the position offset of the predicted bounding box relative to the grid cell, b w and b h denote the logarithmic scale of the bounding box width and height, respectively, and b C Rate the confidence level.

[0119] Calculate the predicted bounding box B output by YOLOv5 pred and the ground-truth bounding box B true For each true bounding box, select the predicted bounding box with the highest intersection over union ratio and mark its corresponding grid cell as the bounding box responsible for the prediction. The calculation formula for the intersection over union (IoU) is:

[0120]

[0121] After predicting the bounding box of each frame in the second image sequence, the bounding boxes of the same grid position in adjacent frames are matched. Based on the real bounding boxes in the training data, batch iteration and optimization are performed through the loss function in the YOLOv5 algorithm.

[0122] The redundant overlapping bounding boxes are eliminated by non-maximum suppression, the bounding box with the highest confidence is retained and the coordinates are converted to Real-world coordinates on the image size.

[0123] This embodiment uses the above-mentioned local abnormality analysis model to process and analyze the multi-view dynamic ultrasound image sequences of 126 subjects, and some examples of the local abnormality analysis results are given in Table 5;

[0124] Table 5 Local anomaly analysis results

[0125]

[0126] In Table 5, RA_UP and RA_Long represent the upper and lower diameters and long diameters of the right atrium, RV_In represents the right ventricular inner diameter, and Ra-St represents the average radial strain. The local abnormality analysis results in Table 4 can be described as follows: on the right atrium image of subject P022, in the bounding box with the center position (200,220) and the size of 65×35, the local abnormality category is detected as the upper and lower diameter of the right atrium is too large, with a probability of 75% and a confidence level of 95%;

[0127] According to the actual coordinates and size, a rectangle is drawn to mark the abnormal area and the abnormal category and the corresponding confidence score are marked near the bounding box.

[0128] This embodiment introduces the YOLO algorithm to train the historical multi-perspective dynamic ultrasound image sequence to obtain a local anomaly analysis model, which can efficiently locate the abnormal area in the ultrasound image. The frame image is processed into a grid unit form, and each grid unit predicts multiple bounding boxes, so that the model can capture the anomalies in the image more carefully. This refined processing method ensures the accurate identification of local anomalies and improves the accuracy of anomaly detection in ultrasound images, thereby further improving the efficiency and quality of right heart acoustic contrast data analysis.

[0129] Embodiment 2

[0130] This embodiment is used to describe the specific implementation of the data integration unit and the security storage unit of the right heart acoustic angiography examination process standardization system proposed by the present invention for the examination unit A.

[0131] The steps of the data integration unit include: acquiring a first image sequence and a second image sequence from an image storage system, using an automated tool to annotate a local abnormal region in the second image sequence, and generating an image sequence with a bounding box annotation.

[0132] The Pillow library in Python was used to merge the above three types of images into one image set according to the time series, so that the three types of images maintained consistent resolution and format. The pandas library in Python was used to organize the right atrial superior and inferior diameters and long diameters, right ventricular internal diameters, valve opening area, wall motion analysis results, and left heart bubble analysis results obtained by the data analysis unit into a structured table; a unified table template was designed, including the name, value, and corresponding normal reference range of each parameter.

[0133] The standardized data template was designed using the LaTex data integration tool; the image set and the structured table were embedded in the standardized data template through an automated script to generate reference data in PDF format. The PDF reference data was transmitted to a display device through the API interface owned by the inspection unit A for medical staff to view and analyze.

[0134] This embodiment uses automated tools to acquire and annotate image sequences, and uses the Pillow library in Python to merge images, ensuring the consistency of image resolution and format; organizes right heart function-related data and left heart bubble analysis results into structured tables through the pandas library for intuitive viewing and analysis; merges image sets and structured tables into reference data, and transmits the reference data to a display device through an API interface; this method improves the efficiency and reliability of image processing and data integration as a whole, and provides technical support for improving the work efficiency of the right heart acoustic contrast examination process.

[0135] The steps for recording and storing detailed information of anomaly identification results include:

[0136] Select a relational database and design the structure and association of each table; in this embodiment, the patients table is designed to store the basic information and unique identifier of the subjects; the ImageSequence table is designed to store the metadata of the first image sequence and the second image sequence; the AbnormalResults table is designed to store right heart function parameters, wall motion analysis results, and global and local abnormality recognition results.

[0137] The right heart function parameters, the wall motion analysis results, the global abnormality identification results, the local abnormality identification results and related metadata are structured and stored in a private cloud database through a database tool, and backed up in a local database; the related metadata include the numbers and imaging times of the first image sequence, the second image sequence and the first image sequence.

[0138] In this embodiment, a relational database is selected to store and manage metadata No. 2 analysis data. By designing a reasonable table structure and association relationship, different tables are used to manage different data types, ensuring the systematicness and accessibility of the data. This structured storage makes data retrieval and analysis more efficient and improves the flexibility of data management in the right heart acoustic contrast examination process.

[0139] Furthermore, in order to ensure the security of the subject metadata and the examination results, the subject metadata and the examination results are encrypted and stored using the AES-256 symmetric encryption algorithm. During the data transmission process, the communication is encrypted using a secure transmission protocol, which includes but is not limited to TLS and SSL.

[0140] Roles are defined for all users involved in the right heart acoustic contrast examination process and access rights are configured. When access to the metadata and examination results of the examinee is requested, the role rights of the visitor are checked to determine whether access is allowed.

[0141] The roles defined and the access permission control matrix set for the right heart acoustic contrast examination process of the examination unit A in this embodiment are shown in Table 6.

[0142] Table 6 Access Rights Control Matrix

[0143]

[0144] According to the access control permissions in Table 6, after passing identity authentication, medical personnel can access and view the basic information of the examinee according to their role R1, which helps the medical personnel understand the background and medical history of the examinee; medical personnel can also view the first image sequence, the second image sequence, and the reference data consisting of right heart function data and abnormality recognition results, and can understand and analyze the right heart function based on these, and then make clinical judgments.

[0145] Technicians can access image sequence data to mark abnormal areas, but do not have the right to edit the basic information of the examinee; the examinee can only view his or her own reference data and cannot access other sensitive data.

[0146] This embodiment uses a symmetric encryption algorithm to encrypt and store the subject's personal information and examination data, ensuring the security of medical data in a static state; a secure transmission protocol is used for encrypted communication during data transmission, further protecting the confidentiality of the data during transmission. At the same time, clear access control allows for detailed division of data access rights for different roles in the system, improving the security and compliance of data management. This method improves the quality and security of the right heart acoustic contrast examination process through encrypted storage and access control.

[0147] In order to help improve the speed and accuracy of right heart acoustic contrast imaging image data processing and analysis, thereby improving the quality and efficiency of the information processing of the contrast examination process, this embodiment proposes a right heart acoustic contrast imaging process standardization system. First, the multi-perspective dynamic ultrasound image sequence obtained by the imaging acquisition unit can comprehensively and accurately reflect the structure and functional state of the right heart. The data analysis unit uses advanced image processing algorithms to construct an image data processing and analysis model, effectively extracts right heart function-related data, left heart bubble analysis results, and global and local abnormality analysis results, thereby improving the speed and accuracy of image data processing and analysis. The data integration unit integrates the image data and analysis results to form reference data and transmits them to the display device, providing medical staff with reference data for auxiliary decision-making. In addition, the secure storage unit ensures the security of the contrast examination data, and the access permission control enables various types of users to effectively and compliantly access the data at different levels. Through image analysis and security protection, this embodiment comprehensively improves the efficiency and quality of the right heart acoustic contrast imaging process.

[0148] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A right heart acoustic contrast examination process standardization system, characterized in that: include: Imaging acquisition unit: used to set ultrasound parameters and use ultrasound equipment to obtain a multi-view dynamic ultrasound image sequence, which is recorded as the first image sequence; Data analysis unit: used to analyze the first image sequence, including image preprocessing, extraction of right heart function related data, global abnormality analysis and abnormal area positioning; preprocessing the first image sequence to obtain a second image sequence; using an image processing algorithm to analyze the second image sequence, extracting right heart function parameters, wall motion analysis results and left heart bubble analysis results; extracting the global feature vector in the second image sequence through a 3D convolutional neural network, using a fully connected layer to calculate the probability of each abnormal category in the frame image, and obtaining a global abnormality recognition result; locating the abnormal area by presetting a real bounding box and generating a local abnormality recognition result; A data integration unit: used for combining the first image sequence, the abnormality recognition result, the right heart function related data and the left heart bubble analysis result into reference data and transmitting the reference data to a display device; Secure storage unit: used for encrypting and storing the right heart function related data, the left heart bubble analysis results, abnormality identification results and related metadata and controlling access rights.

2. A right heart acoustic contrast examination process standardization system according to claim 1, characterized in that: The step of acquiring the multi-view dynamic ultrasound image sequence by ultrasound equipment comprises: The ultrasonic parameters are set to acquire the multi-view dynamic ultrasonic image sequence, including the right atrium imaging, the right ventricle imaging, the related valve imaging and the atrial septum imaging; the atrial septum imaging includes the imaging before and after the Valsalva maneuver; the angle and position of the ultrasonic device are adjusted, and the ultrasonic images are acquired multiple times by changing different view angles, which are recorded as the first image sequence; and the number and imaging time of the first image sequence are recorded.

3. A right heart acoustic contrast examination process standardization system according to claim 1, characterized in that: The image preprocessing step comprises: For the first image sequence V={I1, I2, ..., I T }decomposed into separate frame images I t , where I t is the t-th frame image, T is the total number of frames; for each frame I t Gaussian filtering is used to reduce high-frequency noise, and histogram equalization is used to enhance image contrast and highlight cardiac structures; For contrast-enhanced frames Detect key feature points and convert the frame and adjacent frames Perform feature point matching; by calculating the transformation matrix T t , the frame Align to the reference frame; apply the transformation matrix to perform geometric correction on each frame to obtain the second image sequence 4. A right heart acoustic contrast examination process standardization system according to claim 1, characterized in that: The steps of extracting right heart function related data include right heart function parameter extraction and wall motion analysis; A frame of the right atrium at the end of contraction in the second image sequence within a cardiac cycle is selected, and the contour of the right atrium is obtained by edge detection and contour extraction algorithms, and a contour point set of the right atrium is extracted; the maximum pixel distance of the right atrium in the vertical and horizontal directions is calculated, and converted into the actual upper and lower diameters of the right atrium and the long diameter of the right atrium Selecting a frame of the right ventricle at the end of diastole in the second image sequence within a cardiac cycle, detecting the straight line edge of the ventricular inner wall by Hough straight line transform, calculating the pixel distance of the right ventricular inner diameter and converting it into the actual distance; Capture frames of the valve in the maximum opening state in the second image sequence within a cardiac cycle, obtain an edge map of the valve area, extract a contour point sequence of the valve opening, and calculate the valve opening area using a contour integration method.

5. A right heart acoustic contrast examination process standardization system according to claim 4, characterized in that: The process of wall motion analysis includes: Calculating the upper and lower diameters and long diameters of the right atrium, the inner diameter of the right ventricle and the valve opening area of ​​each frame in the second image sequence; The pixel motion between adjacent frames of the second image sequence is calculated, and the displacement and velocity are calculated based on the characteristic points on the myocardium. The radial and longitudinal strains and strain rates of the myocardium are calculated and acquired through the displacement and velocity.

6. A right heart acoustic contrast examination process standardization system according to claim 4, characterized in that: The process of left heart bubble analysis includes: U-Net is used to segment the atrial septum boundary of the second image sequence and locate the boundary of the foramen ovale region; the position of the bubble in the second image sequence is detected by a brightness threshold segmentation method to generate a binary image; a unique identifier is assigned to each bubble in the first frame where the bubble is detected and the bubble trajectory is initialized, the displacement of the bubble between adjacent frames is calculated by an optical flow method, and the bubble trajectory is updated; when no displacement change of the bubble is detected in several consecutive frames, the bubble trajectory is terminated; Determine the starting point coordinates and the ending point coordinates of all the bubble trajectories; mark the bubbles whose starting point coordinates are located in the right heart area and whose ending point coordinates are located in the left heart area as bubbles to be determined; calculate the positional relationship between the coordinates of the bubbles to be determined in the corresponding bubble trajectory and the foramen ovale area, and determine whether the bubble trajectory passes through the foramen ovale area based on the positional relationship; count the bubbles to be determined whose bubble trajectories pass through the foramen ovale area, and generate the left heart bubble analysis result.

7. A right heart acoustic contrast examination process standardization system according to claim 1, characterized in that: The steps of global abnormality analysis include: Process the second image sequence into a time frame sequence X∈R T×H×W×C Input into a neural network consisting of three layers of 3D convolution to obtain the global feature vector F∈R d ; Wherein, H, W and C are image height, image width and number of channels respectively, and d is feature dimension; the 3D convolution kernel size of the three-layer convolution is 3×3×3, and the number is 32, 64 and 128 respectively; the three-layer convolution adopts maximum pooling and ReLU activation function; The global feature vector F∈R d Calculate the score z of each abnormal category through the fully connected layer i , the probability distribution P of each abnormal category is calculated by the softmax function i ; Among them, w i is the weight vector of the ith abnormal category, e is the base of the natural logarithm, K is the number of abnormal categories, b i is the bias term of the i-th abnormal category; set the threshold, when P i >T, then determine the time frame sequence X∈R T×H×W×C There is an i-th type of anomaly in .

8. A right heart acoustic contrast examination process standardization system according to claim 1, characterized in that: The abnormal area positioning includes locating the abnormal area through the YOLO algorithm and generating a local abnormality recognition result; For the second image sequence, each frame of image Adjust to a fixed size of 448×448, divide into S×S grid units, predict B bounding boxes for each grid; match the bounding boxes at the same position in adjacent frames; The probability that each of the bounding boxes belongs to each anomaly category is calculated by the YOLOv5 algorithm; the confidence score of each of the bounding boxes for a specific anomaly category is calculated; redundant overlapping bounding boxes are eliminated by non-maximum suppression, the bounding box with the highest confidence is retained, and the coordinates are converted to the frame The actual coordinates on the image size are used to draw a rectangle to mark the abnormal area according to the actual coordinates and the size of the frame image, and the abnormal category and the corresponding confidence score are marked near the bounding box.

9. A right heart acoustic contrast examination process standardization system according to claim 1, characterized in that: The steps of the data integration unit include: The first image sequence, the second image sequence and the second image sequence with abnormal area annotations are merged into an image set; the superior and inferior diameters and long diameters of the right atrium, the inner diameter of the right ventricle, the valve opening area, the wall motion analysis results and the left heart bubble analysis results are organized into a structured table; the image set and the structured table are merged into the reference data and transmitted to a display device.

10. A right heart acoustic contrast examination process standardization system according to claim 1, characterized in that: The steps of encrypted storage and access permission control include: The right heart function parameters, the wall motion analysis results, the left heart bubble analysis results, the global abnormality identification results, the local abnormality identification results and related metadata are structured and stored in a private cloud database, and backed up in a local database; the related metadata includes the numbers and imaging times of the first image sequence, the second image sequence and the first image sequence; the subject's metadata and examination results are encrypted and stored using the AES-256 symmetric encryption algorithm; the communication is encrypted using a secure transmission protocol during data transmission; access rights are defined for roles in the right heart acoustic contrast examination process, and when access to the subject's metadata and examination results is requested, the visitor's role permissions are checked to determine whether access is allowed.

Images