Intelligent early warning method, device, equipment and medium based on cardiovascular images

Abstract

The present invention discloses an intelligent early warning method, device, equipment and medium based on cardiovascular images, which first pre-processes the cardiovascular image set corresponding to the user information to be evaluated, separates the preset target objects and extracts the structural features, obtains the target object separation result set and the structural dimension features; performs optical flow calculation, ROI dynamic tracking and motion decomposition on the target object separation result set to obtain the current motion curve output result; obtains the user's vital sign data uploaded by the smart wearable device; inputs the structural dimension features, the current motion curve output result and the user's vital sign data into the classification model, obtains the classification output result and generates intelligent early warning prompt information. Through the above method, it is possible to combine the structural dimension features extracted from the cardiovascular image, the motion curve output result and the user's vital sign data to form multi-dimensional data, which is then input into the classification model to obtain a more reliable classification output result, and quickly generate intelligent early warning prompt information and send it to the user for viewing in a timely manner.

Description

Technical Field

[0001] The present invention relates to the field of intelligent decision-making technology in artificial intelligence, and in particular to an intelligent early warning method, device, equipment and medium based on cardiovascular images. Background Art

[0002] At present, convolutional neural networks are often used for image analysis of cardiovascular images. For example, a discriminative coronary tracing model can be used, which is composed of a three-dimensional convolutional neural network, which can iteratively search for complete blood vessels from cardiovascular images and correctly distinguish between coronary arteries and veins in cardiovascular images. When the above discriminative coronary tracing model is used, the cardiovascular structure can be accurately identified and segmented in cardiovascular images. However, after accurately identifying and segmenting the cardiovascular structure in cardiovascular images, the recognition results obtained focus on the structural features of the image, and do not simultaneously consider other multi-dimensional features of the collective image or other source user features for comprehensive analysis, resulting in low reliability of the output classification results. Summary of the invention

[0003] The embodiments of the present invention provide an intelligent early warning method, device, equipment and medium based on cardiovascular images, aiming to solve the problem that when the prior art methods use convolutional neural networks to perform image recognition and analysis on cardiovascular images, they focus on the structural features of the images, and do not simultaneously consider other multi-dimensional features of the collective images or user features from other sources for comprehensive analysis, resulting in low reliability of the output classification results.

[0004] In a first aspect, an embodiment of the present invention provides an intelligent early warning method based on cardiovascular images, which is applied to a server and includes:

[0005] In response to a cardiovascular image recognition instruction, obtaining user information to be evaluated and a cardiovascular image set corresponding to the cardiovascular image recognition instruction; wherein the cardiovascular image set includes multiple frames of ultrasound cardiac images;

[0006] performing image preprocessing on each cardiovascular image in the cardiovascular image set based on a preset image preprocessing strategy to obtain a preprocessed cardiovascular image set;

[0007] Based on a preset image segmentation model, the pre-processed cardiovascular image set is subjected to separation of preset target objects and extraction of structural features to obtain a target object separation result set and structural dimension features;

[0008] Based on the preset centroid optical flow calculation model and ROI dynamic tracking model, optical flow calculation and ROI dynamic tracking are performed on the target object separation result set to obtain the current dynamic tracking analysis result;

[0009] By performing motion decomposition on the current dynamic tracking analysis result, a corresponding current motion curve output result is obtained;

[0010] Obtaining user vital sign data uploaded by a smart wearable device that is in communication with a server and corresponds to the user information to be evaluated;

[0011] The structural dimension features, the current motion curve output results and the user's vital signs data are combined into comprehensive input feature data, and are input into a pre-trained classification model to obtain a classification output result, and intelligent early warning prompt information is generated based on the classification output result and sent to the smart wearable device.

[0012] In a second aspect, an embodiment of the present invention provides an intelligent early warning device based on cardiovascular images, which is configured on a server, wherein the device is used to execute the intelligent early warning method based on cardiovascular images as described in the first aspect above, and the device includes:

[0013] A cardiovascular image set acquisition unit, configured to respond to a cardiovascular image recognition instruction and acquire user information to be evaluated and a cardiovascular image set corresponding to the cardiovascular image recognition instruction; wherein the cardiovascular image set includes a plurality of frames of ultrasonic cardiac images;

[0014] An image preprocessing unit, configured to perform image preprocessing on each cardiovascular image in the cardiovascular image set based on a preset image preprocessing strategy to obtain a preprocessed cardiovascular image set;

[0015] A structural feature extraction unit, used for separating preset target objects and extracting structural features from the preprocessed cardiovascular image set based on a preset image segmentation model, to obtain a target object separation result set and structural dimension features;

[0016] A dynamic tracking and analysis unit, used to perform optical flow calculation and ROI dynamic tracking on the target object separation result set based on a preset centroid optical flow calculation model and ROI dynamic tracking model to obtain a current dynamic tracking and analysis result;

[0017] A motion curve output unit, used for obtaining a corresponding current motion curve output result by performing motion decomposition on the current dynamic tracking analysis result;

[0018] A user vital sign data acquisition unit, used to acquire user vital sign data uploaded by a smart wearable device that is in communication with the server and corresponds to the user information to be evaluated;

[0019] An intelligent warning information generation unit is used to combine the structural dimension features, the current motion curve output results and the user's vital signs data into comprehensive input feature data, and input them into a pre-trained classification model to obtain a classification output result, and generate intelligent warning prompt information based on the classification output result and send it to the smart wearable device.

[0020] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer device executes the computer program, the intelligent early warning method based on cardiovascular images as described in the first aspect above is implemented.

[0021] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the intelligent early warning method based on cardiovascular images as described in the first aspect above is implemented.

[0022] The embodiment of the present invention provides an intelligent early warning method, device, equipment and medium based on cardiovascular images. The method includes: in response to a cardiovascular image recognition instruction, obtaining user information to be evaluated and a cardiovascular image set corresponding to the cardiovascular image recognition instruction; wherein the cardiovascular image set includes multiple frames of ultrasound cardiograms; performing image preprocessing on each cardiovascular image in the cardiovascular image set based on a preset image preprocessing strategy to obtain a preprocessed cardiovascular image set; performing separation of preset target objects and structural feature extraction on the preprocessed cardiovascular image set based on a preset image segmentation model to obtain a target object separation result set and a structural dimension feature; performing optical flow calculation and ROI dynamic tracking on the target object separation result set based on a preset centroid optical flow calculation model and ROI dynamic tracking model to obtain a current dynamic tracking analysis result; performing motion decomposition on the current dynamic tracking analysis result to obtain a corresponding current motion curve output result; obtaining user vital sign data uploaded by a smart wearable device connected to a server and corresponding to the user information to be evaluated; forming comprehensive input feature data with structural dimension features, current motion curve output results and user vital sign data, and inputting them into a pre-trained classification model to obtain a classification output result, and generating intelligent early warning prompt information based on the classification output result and sending it to the smart wearable device. Through the above method, the structural dimensional features extracted from cardiovascular images, the motion curve output results corresponding to the dynamic tracking analysis results, and the user vital signs data collected and uploaded by the smart wearable device can be combined to form a multi-dimensional data synthesis and then input into the classification model. In combination with the classification output results with higher reliability, intelligent early warning prompt information can be intelligently and quickly generated and sent to the user for viewing in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0024] Figure 1 A schematic diagram of an application scenario of an intelligent early warning method based on cardiovascular images provided by an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a process flow of an intelligent early warning method based on cardiovascular images provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a sub-process of an intelligent early warning method based on cardiovascular images provided by an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of a sub-process of an intelligent early warning method based on cardiovascular images provided by an embodiment of the present invention;

[0028] Figure 5 A schematic diagram of a sub-process of an intelligent early warning method based on cardiovascular images provided by an embodiment of the present invention;

[0029] Figure 6 A schematic diagram of a sub-process of an intelligent early warning method based on cardiovascular images provided by an embodiment of the present invention;

[0030] Figure 7 A schematic diagram of a sub-process of an intelligent early warning method based on cardiovascular images provided by an embodiment of the present invention;

[0031] Figure 8 A schematic block diagram of an intelligent early warning device based on cardiovascular images provided by an embodiment of the present invention;

[0032] Fig. 9 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0035] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0036] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0037] See also Figure 1 and Figure 2 , Figure 1 The embodiment of the present invention provides Figure 1 A schematic diagram of an application scenario of an intelligent early warning method based on cardiovascular images provided by an embodiment of the present invention; Figure 2 The flowchart of the intelligent early warning method based on cardiovascular images provided by the embodiment of the present invention is as follows; the intelligent early warning method based on cardiovascular images is applied to the server 10, and a network connection is established between the server 10 and the intelligent wearable device 20 and the user terminal 30 to realize the transmission of data information, and a network connection is established between the intelligent wearable device 20 and the user terminal 30 to realize the transmission of data information; the intelligent early warning method based on cardiovascular images is executed by the application software installed on the server 10. Figure 2 As shown, the method includes steps S110 to S170.

[0038] S110 . In response to a cardiovascular image recognition instruction, obtain to-be-evaluated user information and a cardiovascular image set corresponding to the cardiovascular image recognition instruction.

[0039] The cardiovascular image set includes multiple frames of ultrasonic cardiogram images.

[0040] In this embodiment, each frame of ultrasound image in the cardiovascular image set can be acquired by an ultrasound detector, and each frame of ultrasound image has an image acquisition time. When cardiovascular image recognition is required for a user to be evaluated (i.e., the user corresponding to the user information to be evaluated), the user identification unique number corresponding to the user information to be evaluated can be obtained first (such as based on the user unique identity code as the user identification unique number), and then the target time period is generated based on the generation time of the cardiovascular image recognition instruction (such as using the generation time of the vascular image recognition instruction - T 预设周期 is the starting time of the target time period, and the generation time of the vascular image recognition instruction is the end time of the target time period, T 预设周期Generally, the duration can be set to 12 hours, 24 hours, 7 days, etc. and is not limited to the above examples and can be flexibly set according to actual needs), and the user identification unique number and target time period are used as retrieval conditions to obtain the cardiovascular image set corresponding to the user information to be evaluated in the ultrasound cardiac image library of the server.

[0041] S120 , performing image preprocessing on each cardiovascular image in the cardiovascular image set based on a preset image preprocessing strategy to obtain a preprocessed cardiovascular image set.

[0042] In this embodiment, after obtaining the cardiovascular image set corresponding to the user information to be evaluated, since the image quality may be low, the image preprocessing strategy may be used to preprocess each cardiovascular image in the cardiovascular image set to obtain a preprocessed cardiovascular image set. Afterwards, various image processing and analysis may be performed on the obtained preprocessed cardiovascular image set to obtain a more accurate analysis result.

[0043] In one embodiment, if Figure 3 As shown, step S120 includes:

[0044] S121, sorting the cardiovascular images in the cardiovascular image set in ascending order based on image acquisition time to obtain a sorted cardiovascular image set;

[0045] S122, performing image cropping, image noise reduction and normalization processing on each cardiovascular image frame in the sorted cardiovascular image set to obtain a preprocessed cardiovascular image corresponding to each cardiovascular image frame, and forming the preprocessed cardiovascular image set.

[0046] In this embodiment, since the image acquisition time corresponding to each cardiovascular image set in the cardiovascular image set is known, the cardiovascular images in the cardiovascular image set can be sorted in ascending order based on the image acquisition time to obtain a sorted cardiovascular image set. Moreover, the reason why the cardiovascular images are sorted in ascending order of the image acquisition time is also to facilitate the subsequent image optical flow analysis.

[0047] Afterwards, each cardiovascular image in the sorted cardiovascular image set is cropped, that is, each cardiovascular image is cropped to a preset image size based on a preset image size, and each cropped cardiovascular image forms a cropped cardiovascular image set; each cropped cardiovascular image in the cropped cardiovascular image set is subjected to image denoising based on a Gaussian filter function corresponding to an image denoising model (the Gaussian kernel size can be set based on actual needs) to obtain a denoised cardiovascular image set; each denoised cardiovascular image in the denoised cardiovascular image set is subjected to grayscale normalization based on a grayscale normalization strategy (i.e., the grayscale value of each pixel in the denoised cardiovascular image is adjusted to a range of 0 to 1), and a preprocessed cardiovascular image corresponding to each cardiovascular image is obtained, and the preprocessed cardiovascular image set is formed. It can be seen that based on the above-mentioned specific image preprocessing method, the obtained preprocessed cardiovascular image can highlight the region or structure of key objects (such as heart chambers and blood vessels) in the image.

[0048] S130, based on a preset image segmentation model, performing separation of preset target objects and extraction of structural features on the preprocessed cardiovascular image set to obtain a target object separation result set and structural dimension features.

[0049] In this embodiment, after obtaining a preprocessed cardiovascular image set with high image quality, the contour of a preset target can be further determined for each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set based on an image segmentation model to achieve separation of the preset target object from the background image, and then the feature extraction of the structural dimension is performed on the region of the preset target object in the preprocessed cardiovascular image, thereby obtaining features reflecting the preprocessed cardiovascular image in the structural dimension.

[0050] In one embodiment, the image segmentation model is an improved intelligent scissors model, such as Figure 4 As shown, step S130 includes:

[0051] S131, for each frame of the pre-processed cardiovascular image in the pre-processed cardiovascular image set, performing target object separation results of a preset target object on the pre-processed cardiovascular image based on the improved intelligent scissors model corresponding to the image segmentation model; wherein the preset target object is any one of a cardiac chamber and a blood vessel; the target object separation results corresponding to each pre-processed cardiovascular image constitute the target object separation result set;

[0052] S132: if it is determined that the preset target object is a cardiac chamber, a ventricular ejection fraction assessment model is obtained, and a corresponding first-dimensional structural sub-feature is determined based on the ventricular ejection fraction assessment model for the target object separation result of each frame of preprocessed cardiovascular image, and a first-dimensional structural feature is obtained according to each first-dimensional structural sub-feature and a preset first comprehensive assessment strategy, and the first-dimensional structural feature is used as the structural dimensional feature;

[0053] S133. If it is determined that the preset target object is a blood vessel, a blood vessel stenosis rate assessment model is obtained, and the corresponding second-dimensional structural sub-features are determined based on the target object separation result of each frame of preprocessed cardiovascular image by the blood vessel stenosis rate assessment model, and the second-dimensional structural features are obtained according to each second-dimensional structural sub-feature and a preset second comprehensive evaluation strategy, and the second-dimensional structural features are used as the structural dimension features.

[0054] In this embodiment, if the image segmentation model specifically adopts an improved intelligent scissors model, compared with the traditional improved intelligent scissors model, it not only pays attention to the color features and grayscale features of each pixel point in the pre-processed cardiovascular image, but also pays attention to the texture features and gradient features. When the contour of the preset target object is recognized in the pre-processed cardiovascular image based on the improved intelligent scissors model, the contour positioning result obtained is more accurate.

[0055] After determining the target object separation result corresponding to each frame of the preprocessed cardiovascular image set, when determining that the preset target object is a cardiac chamber, the target object separation result corresponding to each frame of the preprocessed cardiovascular image can be combined with the ventricular ejection fraction evaluation model to determine the first dimensional structural sub-feature corresponding to each target object separation result. The following is an example of the process of determining the corresponding first dimensional structural sub-feature for a frame of preprocessed cardiovascular image and the ventricular ejection fraction evaluation model, wherein the calculation formula corresponding to the ventricular ejection fraction evaluation model is as follows (1):

[0056] D L =(D L1 3 -D L2 3 ) / D L1 3 *100% (1)

[0057] Among them, D L represents the first dimension structural sub-feature corresponding to the target object separation result (it can also be understood as the ventricular ejection fraction corresponding to the target object separation result), D L1 Indicates the ventricular end-diastolic diameter corresponding to the target object separation result, D L2Indicates the end-systolic inner diameter corresponding to the target object separation result; because the actual length corresponding to each pixel in each frame of the preprocessed cardiovascular image is known, the inner diameters of each chamber of the heart can be determined based on the target object separation result, thereby determining the end-diastolic inner diameter of the ventricle (such as the end-diastolic inner diameter of the left ventricle) and the end-systolic inner diameter (such as the end-systolic inner diameter of the left ventricle). Referring to the above process, after determining the first dimensional structural sub-features corresponding to the target object separation results of each frame of the preprocessed cardiovascular image, the first dimensional structural sub-features can be averaged based on the averaging operation corresponding to the first comprehensive evaluation strategy, and the first average value result is used as the first dimensional structural feature, and the first dimensional structural feature is used as the structural dimensional feature.

[0058] After determining the target object separation result corresponding to each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set, when determining that the preset target object is a blood vessel, the target object separation result corresponding to each frame of the preprocessed cardiovascular image can be combined with the vascular stenosis rate assessment model to determine the second dimensional structural sub-feature corresponding to each target object separation result. The following is an example of the process of determining the corresponding second dimensional structural sub-feature based on the target object separation result corresponding to a frame of the preprocessed cardiovascular image combined with the vascular stenosis rate assessment model, where the calculation formula corresponding to the vascular stenosis rate assessment model is as follows (2):

[0059] P 狭窄率 =(D 正常管径 -D 狭窄管径 ) / D 正常管径 *100% (2)

[0060] Among them, P 狭窄率 represents the second dimension structural sub-feature corresponding to the target object separation result (it can also be understood as the vascular stenosis rate corresponding to the target object separation result), D 正常管径 represents the normal diameter of the blood vessel corresponding to the target object separation result (generally, the normal diameter of the blood vessel is obtained from the starting point of the blood vessel branch in the blood vessel area corresponding to the target object separation result), D 狭窄管径Indicates the stenotic diameter of the blood vessel corresponding to the target object separation result (generally, the stenotic diameter of the blood vessel is obtained from the position with the smallest diameter in the blood vessel region corresponding to the target object separation result); because the actual length corresponding to each pixel in each frame of the preprocessed cardiovascular image is known, the blood vessel diameters at various locations in the blood vessel region can be determined based on the target object separation result, thereby determining the normal blood vessel diameter and the stenotic diameter of the blood vessel corresponding to the target object separation result. Referring to the above process, after determining the second dimensional structural sub-features corresponding to the target object separation results of each frame of the preprocessed cardiovascular image, the average value of each second dimensional structural sub-feature can be calculated based on the averaging operation corresponding to the second comprehensive evaluation strategy (which is the same as the averaging operation corresponding to the first comprehensive evaluation strategy), and the obtained second average value result is used as the second dimensional structural feature, and the second dimensional structural feature is used as the structural dimension feature. It can be seen that based on the above target object separation method, not only can the accurate outer contour of the target object be quickly obtained, but also accurate structural dimension features can be extracted from the image region in the preprocessed cardiovascular image based on the target object separation result.

[0061] In one embodiment, if Figure 5 As shown, step S131 includes:

[0062] S1311, obtaining color features, grayscale features, texture features, and gradient features of each pixel in the preprocessed cardiovascular image, and forming a pixel comprehensive feature corresponding to each pixel;

[0063] S1312. Extract the preset target object contour based on the improved intelligent scissors model and the comprehensive features of the pixel points corresponding to each pixel point in the preprocessed cardiovascular image to obtain the target object separation result.

[0064] In this embodiment, when extracting the outer contour of a preset target object from a pre-processed cardiovascular image based on the improved intelligent scissors model, the color features, grayscale features, texture features, and gradient features of each pixel in the pre-processed cardiovascular image can be first obtained. The specific process is as follows:

[0065] 1) obtaining a red channel value and a green channel value of each pixel of the preprocessed cardiovascular image in the RGB color space, and taking the red channel value / green channel value of the pixel as a color feature of the pixel;

[0066] 2) obtaining a grayscale mean corresponding to the grayscale value of each pixel in the preprocessed cardiovascular image as a grayscale feature of the pixel;

[0067] 3) Obtain each pixel in the preprocessed cardiovascular image, select a 3*3 pixel area with it as the central pixel, compare the grayscale values ​​of the other 8 pixels in the 3*3 pixel area with the grayscale value of the central pixel in order from left to right and from top to bottom, if the grayscale value of the pixel is greater than the grayscale value of the central pixel, the comparison result is recorded as 1, otherwise it is recorded as 0, and then the 8 results of 0 or 1 are concatenated to form an 8-bit binary sequence, and finally the binary sequence is converted into a decimal value to obtain the local binary value of the central pixel, and used as the texture feature of the central pixel;

[0068] 4) Obtaining the gradient feature of each pixel in the preprocessed cardiovascular image based on the Sobel operator.

[0069] After each pixel in the pre-processed cardiovascular image has acquired the color feature, grayscale feature, texture feature and gradient feature respectively in the above-mentioned manner, the pixel comprehensive feature corresponding to each pixel can be formed. Then, the pixel comprehensive feature corresponding to each pixel is input into the improved intelligent scissors model to extract the outer contour of the preset target object, and the target object separation result is obtained. Among them, when the improved intelligent scissors model is used to extract the outer contour of the preset target object in the pre-processed cardiovascular image, the difference from the traditional intelligent scissors model that only considers the color feature and grayscale feature of the pixel to determine the outer contour of the analysis object is that the texture feature and gradient feature of the pixel are also considered, so that more features are considered when determining the weight value of the connecting edge between two pixels (such as forming a comprehensive feature vector of adjacent pixels based on the color feature, grayscale feature, texture feature and gradient feature of each of the two adjacent pixels, and inputting it into a pre-trained connecting edge weight setting model such as a convolutional neural network to obtain the connecting edge weight value between the two pixels), and the outer contour extraction result obtained is closer to the actual visual boundary of the cardiovascular image.

[0070] S140, performing optical flow calculation and ROI dynamic tracking on the target object separation result set based on a preset centroid optical flow calculation model and ROI dynamic tracking model to obtain a current dynamic tracking analysis result.

[0071] In this embodiment, after obtaining the target object separation results corresponding to each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set and forming a target object separation result set, optical flow calculation and ROI dynamic tracking can be performed in combination with multiple target object separation results in the target object separation result set to obtain the current dynamic tracking analysis result.

[0072] In one embodiment, if Figure 6 As shown, step S140 includes:

[0073] S141, determining a target optical flow field corresponding to a target object separation result ranked first in the target object separation result set based on the centroid optical flow calculation model and the time sequence arrangement order of the image frames corresponding to the target object separation result set;

[0074] S142, performing ROI dynamic tracking on the optical flow of each pixel in the region of interest of the target optical flow field to obtain the current dynamic tracking analysis result corresponding to the region of interest of the target optical flow field.

[0075] In this embodiment, when performing optical flow calculation and ROI dynamic tracking on the target object separation result set based on the centroid optical flow calculation model and the ROI dynamic tracking model (the full name of ROI is Region of Interest, indicating the region of interest), the image frames corresponding to the centroid optical flow calculation model and the target object separation result set are first arranged in a time sequence. This processing is also to obtain a more time-series order. Then, based on the scale-invariant feature transform (SIFT) and other algorithms corresponding to the centroid optical flow calculation model, when one of the pixels in the target object separation result ranked first in the target object separation result set is taken as the centroid pixel for example, the adjacent pixels in the 3*3 or 4*4 pixel area around the centroid pixel can be obtained accordingly, and the corresponding centroid pixel and its adjacent pixels are matched in the target object separation results corresponding to the subsequent multiple frames of pre-processed cardiovascular images, so as to determine the position of the centroid pixel and its adjacent pixels in the target object separation result ranked first in the target object separation result set in the target object separation result corresponding to the subsequent multiple frames of pre-processed cardiovascular images, thereby determining the displacement of each adjacent pixel of the centroid pixel; finally, the target optical flow field of the centroid pixel is determined based on the weighted sum of the displacements of each adjacent pixel of the centroid pixel. Of course, the above process illustrates the processing process of taking one of the pixels in the target object separation result ranked first in the target object separation result set as the centroid pixel as an example. When other pixels in the target object separation result ranked first in other target object separation result sets are used as the centroid pixel to determine the corresponding target optical flow field, the same process is also used. In the above manner, the target optical flow field corresponding to each pixel in the target object separation result ranked first in the target object separation result set can be quickly obtained.

[0076] After obtaining the target optical flow field corresponding to each pixel point in the target object separation result ranked first in the target object separation result set, it is also possible to specify a plurality of pixel points in the target object separation result set to form a region of interest, and perform ROI dynamic tracking on the target optical flow field (i.e., pixel optical flow) corresponding to each pixel point in the region of interest, that is, calculate and obtain the average vector of the target optical flow field corresponding to each pixel point in the region of interest and use it as the current dynamic tracking analysis result. It can be seen that through the above ROI dynamic tracking, the dynamic tracking analysis result corresponding to the region of interest in the target object separation result can be quickly determined.

[0077] S150, performing motion decomposition on the current dynamic tracking analysis result to obtain a corresponding current motion curve output result.

[0078] In this embodiment, after determining the dynamic tracking analysis result corresponding to the region of interest in the target object separation result, which is a dynamic tracking analysis result expressed in vector form, a plane rectangular coordinate system can be first constructed with the pixel point of the center tip of the region of interest as the origin of the plane rectangular coordinate system, and then the current dynamic tracking analysis result is subjected to motion decomposition (such as the partial displacement along the X-axis direction in the plane rectangular coordinate system, such as the partial displacement along the Y-axis direction in the plane rectangular coordinate system, etc.) based on the angle between the average vector corresponding to the current dynamic tracking analysis result and the plane where the plane rectangular coordinate system is located, and the current motion curve output result is generated according to the partial motion parameters obtained by the decomposition (such as the partial displacement along the X-axis direction in the plane rectangular coordinate system, such as the partial displacement along the Y-axis direction in the plane rectangular coordinate system in the above example). It can be seen that by performing motion decomposition on the current dynamic tracking analysis result, the motion of the region of interest in several motion cycles can be quickly obtained.

[0079] In one embodiment, step S150 includes:

[0080] The orthogonal decomposition result of the region of interest in the preset motion direction in the current dynamic tracking analysis result is obtained, and the current motion curve output result corresponding to the region of interest is correspondingly determined.

[0081] In this embodiment, when the current dynamic tracking and analysis result is specifically subjected to motion decomposition, the orthogonal decomposition result of the region of interest in the current dynamic tracking and analysis result in the preset motion direction can be first obtained, wherein a plane rectangular coordinate system is constructed by taking the pixel point of the center tip of the region of interest as the origin of the plane rectangular coordinate system as an example, and the preset motion direction is the positive direction of the X-axis of the plane rectangular coordinate system or the positive direction of the Y-axis of the plane rectangular coordinate system. After obtaining the orthogonal decomposition result of the region of interest in the current dynamic tracking and analysis result in the preset motion direction (still referring to the above example, the partial displacement of the current dynamic tracking and analysis result along the positive direction of the X-axis in the plane rectangular coordinate system, or the partial displacement of the current dynamic tracking and analysis result along the positive direction of the Y-axis in the plane rectangular coordinate system), the current motion curve output result can be generated by combining the time axis as the horizontal axis and the orthogonal decomposition result as the vertical axis. It can be seen that based on the above method, the current motion curve output result corresponding to the region of interest can be quickly generated.

[0082] S160: Obtain user vital sign data uploaded by a smart wearable device that is in communication with a server and corresponds to the user information to be evaluated.

[0083] In this embodiment, in order to obtain more dimensional data of the user to be evaluated, the method of obtaining the cardiovascular image set can also be referred to. The user identification unique number corresponding to the user information to be evaluated and the target time period corresponding to the cardiovascular image recognition instruction are also used as search conditions. The user's vital sign data (such as heart rate, blood oxygen concentration, blood pressure, sleep duration and other parameters) collected by the smart wearable device (such as a smart watch, smart bracelet, etc.) worn by the user to be evaluated are obtained from the user feature database of the server and the user's vital sign data of the user to be evaluated is determined. More specifically, the average heart rate value, average blood oxygen concentration value, average blood pressure value and average sleep duration of the user to be evaluated in the target time period can be obtained, and the user's vital sign data corresponding to the user information to be evaluated can be formed. It can be seen that based on the above method, more dimensional user-related data can also be obtained based on the smart wearable device, so as to output the classification results more intelligently.

[0084] S170, the structural dimension features, the current motion curve output results and the user's vital signs data are combined into comprehensive input feature data, and are input into a pre-trained classification model to obtain a classification output result, and intelligent early warning prompt information is generated based on the classification output result and sent to the smart wearable device.

[0085] In this embodiment, after obtaining the structural dimension features, the current motion curve output results and the user vital sign data obtained by processing the user data of each dimension of the user to be evaluated corresponding to the user information to be evaluated within the target time period, the comprehensive input feature data can be composed and input into the pre-trained classification model to obtain the classification output result. The obtained classification output result may include classification output results of multiple dimensions, and it can also determine the intelligent warning prompt sub-information based on the classification output results of each dimension, and finally form the intelligent warning prompt information (which can be understood as information with text characters as components). When the intelligent warning prompt information is obtained in the server, it can also be sent to the intelligent wearable device and viewed in time based on the display screen of the intelligent wearable device. Of course, the intelligent warning prompt information can also be viewed by a user terminal (such as a smart phone, etc.) that has a user binding relationship with the intelligent wearable device.

[0086] In one embodiment, if Figure 7 As shown, step S170 includes:

[0087] S171, the structural dimension features and the user vital sign data in the comprehensive input feature data are combined into first dimension comprehensive input feature data, and input into the first dimension classification sub-model in the classification model to obtain a first dimension classification output result;

[0088] S172, inputting the current motion curve output result into the second dimension classification sub-model in the classification model, determining the target preset motion curve output result having the greatest similarity with the current motion curve output result from the preset motion curve output result set based on the second dimension classification sub-model, and taking the target dimension classification output result corresponding to the target preset motion curve output result as the second dimension classification output result;

[0089] S173, determining first dimensional risk warning information corresponding to the first dimensional classification output result based on the pre-constructed first knowledge graph, and determining second dimensional risk warning information corresponding to the second dimensional classification output result based on the pre-constructed second knowledge graph;

[0090] S174. Automatically fill the first-dimensional risk warning information and the second-dimensional risk warning information into the corresponding filling area of ​​the warning information prompt template, generate the intelligent warning prompt information and send it to the smart wearable device.

[0091] In this embodiment, when obtaining the first-dimensional comprehensive input feature data, it can be spliced ​​in the order of structural dimension features first and the user vital signs data later to obtain a first-dimensional comprehensive input feature data (which can be regarded as a row vector or a column vector), and then input it into the first-dimensional classification sub-model (such as a convolutional neural network or a deep neural network) in the classification model to obtain the first-dimensional classification output result (for example, a probability value between 0 and 1).

[0092] When the current motion curve output result is input into the second dimension classification sub-model in the classification model for processing, the target preset motion curve output result with the greatest similarity to the current motion curve output result is determined from the preset motion curve output result set based on the second dimension classification sub-model, and the target dimension classification output result corresponding to the target preset motion curve output result is used as the second dimension classification output result. Among them, taking the calculation of the similarity between the current motion curve output result and the current motion curve output result as an example, the Euclidean distance between the above two motion curves is specifically calculated (the sum of the squares of the corresponding ordinate differences of the two motion curves at each horizontal coordinate point is calculated and then the square root is taken), and the smaller the Euclidean distance between the above two motion curves, the greater the similarity.

[0093] Since the first knowledge graph (which includes multiple entities related to the comprehensive input feature data of the first dimension, and each entity can obtain the first dimension risk warning information) and the second knowledge graph (which includes multiple entities related to the comprehensive input feature data of the second dimension, and each entity can obtain the second dimension risk warning information) are also pre-constructed in the server, the first dimension risk warning information corresponding to the first dimension classification output result can be determined based on the first knowledge graph, and the second dimension risk warning information corresponding to the second dimension classification output result can be determined based on the second knowledge graph. Therefore, after obtaining the risk warning information of the above two dimensions, it can be automatically filled in the corresponding filling area of ​​the warning information prompt template, generate the intelligent warning prompt information and send it to the smart wearable device, so that the intelligent warning prompt information of the user to be evaluated within the target time period can be promptly received by the device of the user to be evaluated.

[0094] It can be seen that the embodiment of the method can combine the structural dimensional features extracted from cardiovascular images, the motion curve output results corresponding to the dynamic tracking analysis results, and the user vital sign data collected and uploaded by the smart wearable device to form a multi-dimensional data synthesis, which is then input into the classification model, and combined with the classification output results with higher reliability to intelligently and quickly generate intelligent warning prompt information and send it to the user for viewing in a timely manner.

[0095] The embodiment of the present invention also provides an intelligent early warning device based on cardiovascular images, which can be configured in a server and is used to execute any embodiment of the aforementioned intelligent early warning method based on cardiovascular images. Figure 8 , Figure 8 A schematic block diagram of an intelligent early warning device based on cardiovascular images provided in an embodiment of the present invention.

[0096] like Figure 8 As shown, the intelligent warning device 100 based on cardiovascular images includes a cardiovascular image set acquisition unit 110, an image preprocessing unit 120, a structural feature extraction unit 130, a dynamic tracking and analysis unit 140, a motion curve output unit 150, a user vital sign data acquisition unit 160 and an intelligent warning information generation unit 170.

[0097] The cardiovascular image set acquisition unit 110 is used to respond to a cardiovascular image recognition instruction and acquire the to-be-evaluated user information and the cardiovascular image set corresponding to the cardiovascular image recognition instruction.

[0098] The cardiovascular image set includes multiple frames of ultrasonic cardiogram images.

[0099] In this embodiment, each frame of ultrasound image in the cardiovascular image set can be acquired by an ultrasound detector, and each frame of ultrasound image has an image acquisition time. When cardiovascular image recognition is required for a user to be evaluated (i.e., the user corresponding to the user information to be evaluated), the user identification unique number corresponding to the user information to be evaluated can be obtained first (such as based on the user unique identity code as the user identification unique number), and then the target time period is generated based on the generation time of the cardiovascular image recognition instruction (such as using the generation time of the vascular image recognition instruction - T 预设周期 is the starting time of the target time period, and the generation time of the vascular image recognition instruction is the end time of the target time period, T 预设周期 Generally, the duration can be set to 12 hours, 24 hours, 7 days, etc. and is not limited to the above examples and can be flexibly set according to actual needs), and the user identification unique number and target time period are used as retrieval conditions to obtain the cardiovascular image set corresponding to the user information to be evaluated in the ultrasound cardiac image library of the server.

[0100] The image preprocessing unit 120 is used to perform image preprocessing on each cardiovascular image in the cardiovascular image set based on a preset image preprocessing strategy to obtain a preprocessed cardiovascular image set.

[0101] In this embodiment, after obtaining the cardiovascular image set corresponding to the user information to be evaluated, since the image quality may be low, the image preprocessing strategy may be used to preprocess each cardiovascular image in the cardiovascular image set to obtain a preprocessed cardiovascular image set. Afterwards, various image processing and analysis may be performed on the obtained preprocessed cardiovascular image set to obtain a more accurate analysis result.

[0102] In one embodiment, the image preprocessing unit 120 is specifically used for:

[0103] Sorting the cardiovascular images in the cardiovascular image set in ascending order based on image acquisition time to obtain a sorted cardiovascular image set;

[0104] Each cardiovascular image in the sorted cardiovascular image set is subjected to image cropping, image noise reduction and normalization processing to obtain a preprocessed cardiovascular image corresponding to each cardiovascular image frame, and the preprocessed cardiovascular image set is formed.

[0105] In this embodiment, since the image acquisition time corresponding to each cardiovascular image set in the cardiovascular image set is known, the cardiovascular images in the cardiovascular image set can be sorted in ascending order based on the image acquisition time to obtain a sorted cardiovascular image set. Moreover, the reason why the cardiovascular images are sorted in ascending order of the image acquisition time is also to facilitate the subsequent image optical flow analysis.

[0106] Afterwards, each cardiovascular image in the sorted cardiovascular image set is cropped, that is, each cardiovascular image is cropped to a preset image size based on a preset image size, and each cropped cardiovascular image forms a cropped cardiovascular image set; each cropped cardiovascular image in the cropped cardiovascular image set is subjected to image denoising based on a Gaussian filter function corresponding to an image denoising model (the Gaussian kernel size can be set based on actual needs) to obtain a denoised cardiovascular image set; each denoised cardiovascular image in the denoised cardiovascular image set is subjected to grayscale normalization based on a grayscale normalization strategy (i.e., the grayscale value of each pixel in the denoised cardiovascular image is adjusted to a range of 0 to 1), and a preprocessed cardiovascular image corresponding to each cardiovascular image is obtained, and the preprocessed cardiovascular image set is formed. It can be seen that based on the above-mentioned specific image preprocessing method, the obtained preprocessed cardiovascular image can highlight the region or structure of key objects (such as heart chambers and blood vessels) in the image.

[0107] The structural feature extraction unit 130 is used to separate the preset target objects and extract the structural features of the pre-processed cardiovascular image set based on a preset image segmentation model to obtain a target object separation result set and structural dimension features.

[0108] In this embodiment, after obtaining a preprocessed cardiovascular image set with high image quality, the contour of a preset target can be further determined for each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set based on an image segmentation model to achieve separation of the preset target object from the background image, and then the feature extraction of the structural dimension is performed on the region of the preset target object in the preprocessed cardiovascular image, thereby obtaining features reflecting the preprocessed cardiovascular image in the structural dimension.

[0109] In one embodiment, the image segmentation model is an improved intelligent scissors model, and the structural feature extraction unit 130 is specifically used for:

[0110] For each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set, a target object separation result of a preset target object is performed on the preprocessed cardiovascular image based on the improved intelligent scissors model corresponding to the image segmentation model; wherein the preset target object is any one of a cardiac chamber and a blood vessel; and the target object separation results corresponding to each preprocessed cardiovascular image constitute the target object separation result set;

[0111] If it is determined that the preset target object is a cardiac chamber, a ventricular ejection fraction evaluation model is obtained, and a corresponding first-dimensional structural sub-feature is determined based on the ventricular ejection fraction evaluation model for the target object separation result of each frame of preprocessed cardiovascular image, and a first-dimensional structural feature is obtained according to each first-dimensional structural sub-feature and a preset first comprehensive evaluation strategy, and the first-dimensional structural feature is used as the structural dimensional feature;

[0112] If it is determined that the preset target object is a blood vessel, a blood vessel stenosis rate assessment model is obtained, and the corresponding second dimensional structural sub-features are determined based on the target object separation results of each frame of preprocessed cardiovascular image based on the blood vessel stenosis rate assessment model, and the second dimensional structural features are obtained according to each second dimensional structural sub-feature and a preset second comprehensive evaluation strategy, and the second dimensional structural features are used as the structural dimensional features.

[0113] In this embodiment, if the image segmentation model specifically adopts an improved intelligent scissors model, compared with the traditional improved intelligent scissors model, it not only pays attention to the color features and grayscale features of each pixel point in the pre-processed cardiovascular image, but also pays attention to the texture features and gradient features. When the contour of the preset target object is recognized in the pre-processed cardiovascular image based on the improved intelligent scissors model, the contour positioning result obtained is more accurate.

[0114] After determining the target object separation result corresponding to each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set, when determining that the preset target object is a cardiac chamber, the target object separation result corresponding to each frame of the preprocessed cardiovascular image can be combined with the ventricular ejection fraction evaluation model to determine the first dimensional structural sub-feature corresponding to each target object separation result. The following is an example of the process of determining the corresponding first dimensional structural sub-feature for the target object separation result corresponding to a frame of preprocessed cardiovascular image combined with the ventricular ejection fraction evaluation model, wherein the calculation formula corresponding to the ventricular ejection fraction evaluation model is as shown in the above formula (1). Referring to the above process, after determining the first dimensional structural sub-features corresponding to the target object separation results of each frame of the preprocessed cardiovascular image, the first dimensional structural sub-features can be averaged based on the averaging operation corresponding to the first comprehensive evaluation strategy, and the first average value result obtained is used as the first dimensional structural feature, and the first dimensional structural feature is used as the structural dimensional feature.

[0115] After determining the target object separation result corresponding to each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set, when determining that the preset target object is a blood vessel, the target object separation result corresponding to each frame of the preprocessed cardiovascular image can be combined with the vascular stenosis rate evaluation model to determine the second dimensional structural sub-feature corresponding to each target object separation result. The following is an example of the process of determining the corresponding second dimensional structural sub-feature for the target object separation result corresponding to a frame of the preprocessed cardiovascular image combined with the vascular stenosis rate evaluation model, wherein the calculation formula corresponding to the vascular stenosis rate evaluation model is as shown in the above formula (2). Referring to the above process, after determining the second dimensional structural sub-features corresponding to the target object separation results of each frame of the preprocessed cardiovascular image, the average value of each second dimensional structural sub-feature can be calculated based on the average value operation corresponding to the second comprehensive evaluation strategy (which is the same as the average value operation corresponding to the first comprehensive evaluation strategy), and the obtained second average value result is used as the second dimensional structural feature, and the second dimensional structural feature is used as the structural dimension feature. It can be seen that based on the above target object separation method, not only can the accurate outer contour of the target object be quickly obtained, but also accurate structural dimension features can be extracted from the image area in the preprocessed cardiovascular image based on the target object separation result.

[0116] In one embodiment, the target object separation result of the pre-processed cardiovascular image based on the improved intelligent scissors model corresponding to the image segmentation model, which presets the target object, includes:

[0117] Acquire color features, grayscale features, texture features, and gradient features of each pixel in the preprocessed cardiovascular image, and compose pixel comprehensive features corresponding to each pixel;

[0118] Based on the improved intelligent scissors model and the comprehensive features of the pixel points corresponding to each pixel point in the preprocessed cardiovascular image, the preset target object contour is extracted to obtain the target object separation result.

[0119] In this embodiment, when the outer contour of the preset target object is extracted from the pre-processed cardiovascular image based on the improved intelligent scissors model, the color features, grayscale features, texture features and gradient features of each pixel in the pre-processed cardiovascular image can be first obtained, and the pixel comprehensive features corresponding to each pixel are formed. Then, the pixel comprehensive features corresponding to each pixel are input into the improved intelligent scissors model to extract the outer contour of the preset target object, and the target object separation result is obtained. Among them, when the outer contour of the preset target object is extracted from the pre-processed cardiovascular image using the improved intelligent scissors model, the difference from the traditional intelligent scissors model that only considers the color features and grayscale features of the pixel points to determine the outer contour of the analysis object is that the texture features and gradient features of the pixel points are also considered, so that more features are considered when determining the weight value of the connecting edge between two pixels (such as forming a comprehensive feature vector of adjacent pixels based on the color features, grayscale features, texture features and gradient features of the two adjacent pixels, and inputting it into a pre-trained connecting edge weight setting model such as a convolutional neural network to obtain the connecting edge weight value between the two pixels), and the obtained outer contour extraction result is closer to the actual visual boundary of the cardiovascular image.

[0120] The dynamic tracking analysis unit 140 is used to perform optical flow calculation and ROI dynamic tracking on the target object separation result set based on a preset centroid optical flow calculation model and ROI dynamic tracking model to obtain a current dynamic tracking analysis result.

[0121] In this embodiment, after obtaining the target object separation results corresponding to each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set and forming a target object separation result set, optical flow calculation and ROI dynamic tracking can be performed in combination with multiple target object separation results in the target object separation result set to obtain the current dynamic tracking analysis result.

[0122] In one embodiment, the dynamic tracking and analysis unit 140 is specifically used for:

[0123] Based on the centroid optical flow calculation model and the time sequence arrangement order of the image frames corresponding to the target object separation result set, determining a target optical flow field corresponding to the target object separation result ranked first in the target object separation result set;

[0124] Perform ROI dynamic tracking on the optical flow of each pixel in the region of interest of the target optical flow field to obtain the current dynamic tracking analysis result corresponding to the region of interest of the target optical flow field.

[0125] In this embodiment, when performing optical flow calculation and ROI dynamic tracking on the target object separation result set based on the centroid optical flow calculation model and the ROI dynamic tracking model (the full name of ROI is Region of Interest, indicating the region of interest), the image frames corresponding to the centroid optical flow calculation model and the target object separation result set are first arranged in a time sequence. This processing is also to obtain a more time-series order. Then, based on the scale-invariant feature transform (SIFT) and other algorithms corresponding to the centroid optical flow calculation model, when one of the pixels in the target object separation result ranked first in the target object separation result set is taken as the centroid pixel for example, the adjacent pixels in the 3*3 or 4*4 pixel area around the centroid pixel can be obtained accordingly, and the corresponding centroid pixel and its adjacent pixels are matched in the target object separation results corresponding to the subsequent multiple frames of pre-processed cardiovascular images, so as to determine the position of the centroid pixel and its adjacent pixels in the target object separation result ranked first in the target object separation result set in the target object separation result corresponding to the subsequent multiple frames of pre-processed cardiovascular images, thereby determining the displacement of each adjacent pixel of the centroid pixel; finally, the target optical flow field of the centroid pixel is determined based on the weighted sum of the displacements of each adjacent pixel of the centroid pixel. Of course, the above process illustrates the processing process of taking one of the pixels in the target object separation result ranked first in the target object separation result set as the centroid pixel as an example. When other pixels in the target object separation result ranked first in other target object separation result sets are used as the centroid pixel to determine the corresponding target optical flow field, the same process is also used. In the above manner, the target optical flow field corresponding to each pixel in the target object separation result ranked first in the target object separation result set can be quickly obtained.

[0126] After obtaining the target optical flow field corresponding to each pixel point in the target object separation result ranked first in the target object separation result set, it is also possible to specify a plurality of pixel points in the target object separation result set to form a region of interest, and perform ROI dynamic tracking on the target optical flow field (i.e., pixel optical flow) corresponding to each pixel point in the region of interest, that is, calculate and obtain the average vector of the target optical flow field corresponding to each pixel point in the region of interest and use it as the current dynamic tracking analysis result. It can be seen that through the above ROI dynamic tracking, the dynamic tracking analysis result corresponding to the region of interest in the target object separation result can be quickly determined.

[0127] The motion curve output unit 150 is used to obtain a corresponding current motion curve output result by performing motion decomposition on the current dynamic tracking analysis result.

[0128] In this embodiment, after determining the dynamic tracking analysis result corresponding to the region of interest in the target object separation result, which is a dynamic tracking analysis result expressed in vector form, a plane rectangular coordinate system can be first constructed with the pixel point of the center tip of the region of interest as the origin of the plane rectangular coordinate system, and then the current dynamic tracking analysis result is subjected to motion decomposition (such as the partial displacement along the X-axis direction in the plane rectangular coordinate system, such as the partial displacement along the Y-axis direction in the plane rectangular coordinate system, etc.) based on the angle between the average vector corresponding to the current dynamic tracking analysis result and the plane where the plane rectangular coordinate system is located, and the current motion curve output result is generated according to the partial motion parameters obtained by the decomposition (such as the partial displacement along the X-axis direction in the plane rectangular coordinate system, such as the partial displacement along the Y-axis direction in the plane rectangular coordinate system in the above example). It can be seen that by performing motion decomposition on the current dynamic tracking analysis result, the motion of the region of interest in several motion cycles can be quickly obtained.

[0129] In one embodiment, the motion curve output unit 150 is specifically used for:

[0130] The orthogonal decomposition result of the region of interest in the preset motion direction in the current dynamic tracking analysis result is obtained, and the current motion curve output result corresponding to the region of interest is correspondingly determined.

[0131] In this embodiment, when the current dynamic tracking and analysis result is specifically subjected to motion decomposition, the orthogonal decomposition result of the region of interest in the current dynamic tracking and analysis result in the preset motion direction can be first obtained, wherein a plane rectangular coordinate system is constructed by taking the pixel point of the center tip of the region of interest as the origin of the plane rectangular coordinate system as an example, and the preset motion direction is the positive direction of the X-axis of the plane rectangular coordinate system or the positive direction of the Y-axis of the plane rectangular coordinate system. After obtaining the orthogonal decomposition result of the region of interest in the current dynamic tracking and analysis result in the preset motion direction (still referring to the above example, the partial displacement of the current dynamic tracking and analysis result along the positive direction of the X-axis in the plane rectangular coordinate system, or the partial displacement of the current dynamic tracking and analysis result along the positive direction of the Y-axis in the plane rectangular coordinate system), the current motion curve output result can be generated by combining the time axis as the horizontal axis and the orthogonal decomposition result as the vertical axis. It can be seen that based on the above method, the current motion curve output result corresponding to the region of interest can be quickly generated.

[0132] The user vital sign data acquisition unit 160 is used to acquire user vital sign data uploaded by the smart wearable device that is connected to the server for communication and corresponds to the user information to be evaluated.

[0133] In this embodiment, in order to obtain more dimensional data of the user to be evaluated, the method of obtaining the cardiovascular image set can also be referred to. The user identification unique number corresponding to the user information to be evaluated and the target time period corresponding to the cardiovascular image recognition instruction are also used as search conditions. The user's vital sign data (such as heart rate, blood oxygen concentration, blood pressure, sleep duration and other parameters) collected by the smart wearable device (such as a smart watch, smart bracelet, etc.) worn by the user to be evaluated are obtained from the user feature database of the server and the user's vital sign data of the user to be evaluated is determined. More specifically, the average heart rate value, average blood oxygen concentration value, average blood pressure value and average sleep duration of the user to be evaluated in the target time period can be obtained, and the user's vital sign data corresponding to the user information to be evaluated can be formed. It can be seen that based on the above method, more dimensional user-related data can also be obtained based on the smart wearable device, so as to output the classification results more intelligently.

[0134] The intelligent warning information generation unit 170 is used to combine the structural dimension features, the current motion curve output results and the user's vital signs data into comprehensive input feature data, and input them into a pre-trained classification model to obtain a classification output result, and generate intelligent warning prompt information based on the classification output result and send it to the smart wearable device.

[0135] In this embodiment, after obtaining the structural dimension features, the current motion curve output results and the user vital sign data obtained by processing the user data of each dimension of the user to be evaluated corresponding to the user information to be evaluated within the target time period, the comprehensive input feature data can be composed and input into the pre-trained classification model to obtain the classification output result. The obtained classification output result may include classification output results of multiple dimensions, and it can also determine the intelligent warning prompt sub-information based on the classification output results of each dimension, and finally form the intelligent warning prompt information (which can be understood as information with text characters as components). When the intelligent warning prompt information is obtained in the server, it can also be sent to the intelligent wearable device and viewed in time based on the display screen of the intelligent wearable device. Of course, the intelligent warning prompt information can also be viewed by a user terminal (such as a smart phone, etc.) that has a user binding relationship with the intelligent wearable device.

[0136] In one embodiment, the intelligent early warning information generating unit 170 is specifically used to:

[0137] The structural dimension features and the user vital sign data in the comprehensive input feature data are combined into first-dimensional comprehensive input feature data, and input into the first-dimensional classification sub-model in the classification model to obtain a first-dimensional classification output result;

[0138] Input the current motion curve output result into the second dimension classification sub-model in the classification model, determine the target preset motion curve output result having the greatest similarity with the current motion curve output result from the preset motion curve output result set based on the second dimension classification sub-model, and use the target dimension classification output result corresponding to the target preset motion curve output result as the second dimension classification output result;

[0139] Determine the first dimension risk warning information corresponding to the first dimension classification output result based on the pre-constructed first knowledge graph, and determine the second dimension risk warning information corresponding to the second dimension classification output result based on the pre-constructed second knowledge graph;

[0140] The first dimensional risk warning information and the second dimensional risk warning information are automatically filled into the corresponding filling area of ​​the warning information prompt template, and the intelligent warning prompt information is generated and sent to the smart wearable device.

[0141] In this embodiment, when obtaining the first-dimensional comprehensive input feature data, it can be spliced ​​in the order of structural dimension features first and the user vital signs data later to obtain a first-dimensional comprehensive input feature data (which can be regarded as a row vector or a column vector), and then input it into the first-dimensional classification sub-model (such as a convolutional neural network or a deep neural network) in the classification model to obtain the first-dimensional classification output result (for example, a probability value between 0 and 1).

[0142] When the current motion curve output result is input into the second dimension classification sub-model in the classification model for processing, the target preset motion curve output result with the greatest similarity to the current motion curve output result is determined from the preset motion curve output result set based on the second dimension classification sub-model, and the target dimension classification output result corresponding to the target preset motion curve output result is used as the second dimension classification output result. Among them, taking the calculation of the similarity between the current motion curve output result and the current motion curve output result as an example, the Euclidean distance between the above two motion curves is specifically calculated (the sum of the squares of the corresponding ordinate differences of the two motion curves at each horizontal coordinate point is calculated and then the square root is taken), and the smaller the Euclidean distance between the two motion curves, the greater the similarity.

[0143] Since the first knowledge graph (which includes multiple entities related to the comprehensive input feature data of the first dimension, and each entity can obtain the first dimension risk warning information) and the second knowledge graph (which includes multiple entities related to the comprehensive input feature data of the second dimension, and each entity can obtain the second dimension risk warning information) are also pre-constructed in the server, the first dimension risk warning information corresponding to the first dimension classification output result can be determined based on the first knowledge graph, and the second dimension risk warning information corresponding to the second dimension classification output result can be determined based on the second knowledge graph. Therefore, after obtaining the risk warning information of the above two dimensions, it can be automatically filled in the corresponding filling area of ​​the warning information prompt template, generate the intelligent warning prompt information and send it to the smart wearable device, so that the intelligent warning prompt information of the user to be evaluated within the target time period can be promptly received by the device of the user to be evaluated.

[0144] It can be seen that the implementation example of the device can combine the structural dimensional features extracted from cardiovascular images, the motion curve output results corresponding to the dynamic tracking analysis results, and the user vital signs data collected and uploaded by the smart wearable device to form a multi-dimensional data synthesis, which is then input into the classification model, and combined with the classification output results with higher reliability to intelligently and quickly generate intelligent warning prompt information and send it to the user for viewing in a timely manner.

[0145] The above-mentioned intelligent early warning device based on cardiovascular images can be implemented in the form of a computer program. The computer program can be used in Fig. 9 Runs on the computer device shown.

[0146] See also Fig. 9 , Fig. 9 The schematic block diagram of a computer device provided by an embodiment of the present invention is shown in FIG. The computer device integrates any intelligent early warning device based on cardiovascular images provided by an embodiment of the present invention.

[0147] See also Fig. 9 The computer device 400 includes a processor 402 , a memory and a network interface 405 connected via a system bus 401 , wherein the memory may include a storage medium 403 and an internal memory 404 .

[0148] The storage medium 403 can store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions, and when the program instructions are executed, the processor 402 can execute the above-mentioned intelligent early warning method based on cardiovascular images.

[0149] The processor 402 is used to provide computing and control capabilities to support the operation of the entire computer device.

[0150] The internal memory 404 provides an environment for the operation of the computer program 4032 in the storage medium 403. When the computer program 4032 is executed by the processor 402, the processor 402 can execute the above-mentioned intelligent early warning method based on cardiovascular images.

[0151] The network interface 405 is used to communicate with other devices over the network. Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0152] The processor 402 is used to run the computer program 4032 stored in the memory to implement the above-mentioned intelligent early warning method based on cardiovascular images.

[0153] It should be understood that in the embodiment of the present invention, the processor 402 may be a central processing unit (CPU), and the processor 402 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0154] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer device to implement the process steps of the embodiment of the above method.

[0155] Therefore, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the above-mentioned intelligent early warning method based on cardiovascular images.

[0156] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0157] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0158] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0159] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0160] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0161] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An intelligent early warning method based on cardiovascular images, applied to a server, characterized in that: include: In response to a cardiovascular image recognition instruction, obtaining user information to be evaluated and a cardiovascular image set corresponding to the cardiovascular image recognition instruction; wherein the cardiovascular image set includes multiple frames of ultrasound cardiac images; performing image preprocessing on each cardiovascular image in the cardiovascular image set based on a preset image preprocessing strategy to obtain a preprocessed cardiovascular image set; Based on a preset image segmentation model, the pre-processed cardiovascular image set is subjected to separation of preset target objects and extraction of structural features to obtain a target object separation result set and structural dimension features; Based on the preset centroid optical flow calculation model and ROI dynamic tracking model, optical flow calculation and ROI dynamic tracking are performed on the target object separation result set to obtain the current dynamic tracking analysis result; By performing motion decomposition on the current dynamic tracking analysis result, a corresponding current motion curve output result is obtained; Obtaining user vital sign data uploaded by a smart wearable device that is in communication with a server and corresponds to the user information to be evaluated; The structural dimension features, the current motion curve output results and the user's vital signs data are combined into comprehensive input feature data, and are input into a pre-trained classification model to obtain a classification output result, and intelligent early warning prompt information is generated based on the classification output result and sent to the intelligent wearable device; The image segmentation model is an improved intelligent scissors model; the image segmentation model based on the preset image segmentation model performs separation of preset target objects and structural feature extraction on the preprocessed cardiovascular image set to obtain a target object separation result set and structural dimension features, including: For each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set, a target object separation result of a preset target object is performed on the preprocessed cardiovascular image based on the improved intelligent scissors model corresponding to the image segmentation model; wherein the preset target object is any one of a cardiac chamber and a blood vessel; and the target object separation results corresponding to each preprocessed cardiovascular image constitute the target object separation result set; If it is determined that the preset target object is a cardiac chamber, a ventricular ejection fraction evaluation model is obtained, and a corresponding first-dimensional structural sub-feature is determined based on the ventricular ejection fraction evaluation model for the target object separation result of each frame of preprocessed cardiovascular image, and a first-dimensional structural feature is obtained according to each first-dimensional structural sub-feature and a preset first comprehensive evaluation strategy, and the first-dimensional structural feature is used as the structural dimensional feature; If it is determined that the preset target object is a blood vessel, a blood vessel stenosis rate assessment model is obtained, and a corresponding second-dimensional structural sub-feature is determined based on the target object separation result of each frame of preprocessed cardiovascular image by the blood vessel stenosis rate assessment model, and a second-dimensional structural feature is obtained according to each second-dimensional structural sub-feature and a preset second comprehensive evaluation strategy, and the second-dimensional structural feature is used as the structural dimension feature; The preset centroid optical flow calculation model and ROI dynamic tracking model are used to perform optical flow calculation and ROI dynamic tracking on the target object separation result set to obtain the current dynamic tracking analysis result, including: Based on the centroid optical flow calculation model and the time sequence arrangement order of the image frames corresponding to the target object separation result set, determining a target optical flow field corresponding to the target object separation result ranked first in the target object separation result set; Perform ROI dynamic tracking on the optical flow of each pixel in the region of interest of the target optical flow field to obtain the current dynamic tracking analysis result corresponding to the region of interest of the target optical flow field.

2. The intelligent early warning method based on cardiovascular images according to claim 1 is characterized in that: The performing image preprocessing on each cardiovascular image in the cardiovascular image set based on a preset image preprocessing strategy to obtain a preprocessed cardiovascular image set includes: Sorting the cardiovascular images in the cardiovascular image set in ascending order based on image acquisition time to obtain a sorted cardiovascular image set; Each cardiovascular image in the sorted cardiovascular image set is subjected to image cropping, image noise reduction and normalization processing to obtain a preprocessed cardiovascular image corresponding to each cardiovascular image frame, and the preprocessed cardiovascular image set is formed.

3. The intelligent early warning method based on cardiovascular images according to claim 1 is characterized in that: The target object separation result of the pre-processed cardiovascular image based on the improved intelligent scissors model corresponding to the image segmentation model, which performs the preset target object, includes: Acquire the color feature, grayscale feature, texture feature and gradient feature of each pixel in the preprocessed cardiovascular image, and form a pixel comprehensive feature corresponding to each pixel; Based on the improved intelligent scissors model and the comprehensive features of the pixel points corresponding to each pixel point in the preprocessed cardiovascular image, the preset target object contour is extracted to obtain the target object separation result.

4. The intelligent early warning method based on cardiovascular images according to claim 1, characterized in that: The step of performing motion decomposition on the current dynamic tracking analysis result to obtain a corresponding current motion curve output result includes: The orthogonal decomposition result of the region of interest in the preset motion direction in the current dynamic tracking analysis result is obtained, and the current motion curve output result corresponding to the region of interest is correspondingly determined.

5. The intelligent early warning method based on cardiovascular images according to claim 1, characterized in that: The structural dimension features, the current motion curve output result and the user's vital signs data are combined into comprehensive input feature data, and input into a pre-trained classification model to obtain a classification output result, and intelligent early warning prompt information is generated based on the classification output result and sent to the smart wearable device, including: the structural dimension features and the user's vital signs data in the comprehensive input feature data are combined into first-dimensional comprehensive input feature data, and input into a first-dimensional classification sub-model in the classification model to obtain a first-dimensional classification output result; Input the current motion curve output result into the second dimension classification sub-model in the classification model, determine the target preset motion curve output result having the greatest similarity with the current motion curve output result from the preset motion curve output result set based on the second dimension classification sub-model, and use the target dimension classification output result corresponding to the target preset motion curve output result as the second dimension classification output result; Determine the first dimension risk warning information corresponding to the first dimension classification output result based on the pre-constructed first knowledge graph, and determine the second dimension risk warning information corresponding to the second dimension classification output result based on the pre-constructed second knowledge graph; The first dimensional risk warning information and the second dimensional risk warning information are automatically filled into the corresponding filling area of ​​the warning information prompt template, and the intelligent warning prompt information is generated and sent to the smart wearable device.

6. An intelligent early warning device based on cardiovascular images, configured on a server, characterized in that: include: A cardiovascular image set acquisition unit, configured to acquire, in response to a cardiovascular image recognition instruction, user information to be evaluated and a cardiovascular image set corresponding to the cardiovascular image recognition instruction; wherein the cardiovascular image set includes a plurality of frames of ultrasonic cardiac images; An image preprocessing unit, configured to perform image preprocessing on each cardiovascular image in the cardiovascular image set based on a preset image preprocessing strategy to obtain a preprocessed cardiovascular image set; A structural feature extraction unit, used for separating preset target objects and extracting structural features from the preprocessed cardiovascular image set based on a preset image segmentation model, to obtain a target object separation result set and structural dimension features; A dynamic tracking and analysis unit, used to perform optical flow calculation and ROI dynamic tracking on the target object separation result set based on a preset centroid optical flow calculation model and ROI dynamic tracking model to obtain a current dynamic tracking and analysis result; A motion curve output unit, used for obtaining a corresponding current motion curve output result by performing motion decomposition on the current dynamic tracking analysis result; A user vital sign data acquisition unit, used to acquire user vital sign data uploaded by a smart wearable device that is in communication with the server and corresponds to the user information to be evaluated; An intelligent warning information generating unit, configured to form comprehensive input feature data from the structural dimension features, the current motion curve output result, and the user's vital sign data, and input the data into a pre-trained classification model to obtain a classification output result, and generate intelligent warning prompt information based on the classification output result and send the information to the intelligent wearable device; The image segmentation model is an improved intelligent scissors model; the structural feature extraction unit is used for: For each frame of the preprocessed cardiovascular image in the preprocessed cardiovascular image set, a target object separation result of a preset target object is performed on the preprocessed cardiovascular image based on the improved intelligent scissors model corresponding to the image segmentation model; wherein the preset target object is any one of a cardiac chamber and a blood vessel; and the target object separation results corresponding to each preprocessed cardiovascular image constitute the target object separation result set; If it is determined that the preset target object is a cardiac chamber, a ventricular ejection fraction evaluation model is obtained, and a corresponding first-dimensional structural sub-feature is determined based on the ventricular ejection fraction evaluation model for the target object separation result of each frame of preprocessed cardiovascular image, and a first-dimensional structural feature is obtained according to each first-dimensional structural sub-feature and a preset first comprehensive evaluation strategy, and the first-dimensional structural feature is used as the structural dimensional feature; If it is determined that the preset target object is a blood vessel, a blood vessel stenosis rate assessment model is obtained, and a corresponding second-dimensional structural sub-feature is determined based on the target object separation result of each frame of preprocessed cardiovascular image by the blood vessel stenosis rate assessment model, and a second-dimensional structural feature is obtained according to each second-dimensional structural sub-feature and a preset second comprehensive evaluation strategy, and the second-dimensional structural feature is used as the structural dimension feature; The dynamic tracking and analysis unit is used for: Based on the centroid optical flow calculation model and the time sequence arrangement order of the image frames corresponding to the target object separation result set, determining a target optical flow field corresponding to the target object separation result ranked first in the target object separation result set; Perform ROI dynamic tracking on the optical flow of each pixel in the region of interest of the target optical flow field to obtain the current dynamic tracking analysis result corresponding to the region of interest of the target optical flow field.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer device executes the computer program, the intelligent early warning method based on cardiovascular images as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the intelligent early warning method based on cardiovascular images as described in any one of claims 1 to 5 is implemented.

Images