Cardiovascular image detection method, device and equipment based on multi-mode AI driving

Abstract

The invention discloses a cardiovascular image detection method, device and equipment based on multi-mode AI driving. The method comprises the following steps: firstly, acquiring a cardiovascular image set to be detected, and decrypting and updating the cardiovascular image set; determining a target multi-modal AI model set based on the image type composition proportion in the cardiovascular image set to be detected and a multi-modal AI model determination strategy; and for each to-be-detected cardiovascular image, acquiring a target multi-modal AI model in the target multi-modal AI model set, determining an image detection result corresponding to the to-be-detected cardiovascular image, and performing result fusion to obtain a fusion detection result. Through the method, after the target multi-modal AI model set is determined for the to-be-detected cardiovascular image set comprising multiple types of to-be-detected cardiovascular images based on the image type composition proportion, multi-modal image detection and detection result fusion are correspondingly carried out, and a fusion detection result which can reflect a multi-dimensional evaluation result and has more comprehensive corresponding information is obtained.

Description

Technical Field

[0001] The present invention relates to the field of intelligent decision-making technology of artificial intelligence, and in particular to a cardiovascular image detection method, device and equipment driven by multimodal AI. Background Art

[0002] At present, after acquiring the cardiovascular images of the user, a single-model AI model (such as a convolutional neural network, etc.) is generally used to perform image detection on the cardiovascular images. However, the human cardiovascular system is a complex whole with interrelated parts. Single-modality images such as cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images, cardiovascular MRI images, etc. can often only provide information about a certain aspect of the cardiovascular system. More specifically, cardiovascular X-ray images mainly show the general outline of the heart and the morphology of some large blood vessels. It can be seen that when only a single-modality image is subjected to image detection using a single-model AI model to obtain the test result corresponding to the cardiovascular system, the object to be evaluated, the corresponding information is incomplete, and it is impossible to obtain the corresponding evaluation results of more dimensions and the corresponding test results with more comprehensive information from more dimensions and a more complete image data structure. Summary of the invention

[0003] The embodiments of the present invention provide a cardiovascular image detection method, device and equipment driven by multimodal AI, aiming to solve the problem that when the existing technical method uses a single model AI model image detection on a single modality image to obtain a detection result corresponding to the cardiovascular system to be evaluated, the corresponding information is incomplete, and it is impossible to obtain corresponding evaluation results of more dimensions and detection results with more comprehensive corresponding information from more dimensions and a more complete image data structure.

[0004] In a first aspect, an embodiment of the present invention provides a cardiovascular image detection method based on multimodal AI driving, which includes: In response to an image detection instruction, acquiring a cardiovascular image set to be detected corresponding to the image detection instruction; wherein the cardiovascular image set to be detected includes at least two of cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images, and cardiovascular MRI images; If it is determined that the cardiovascular image set to be detected is an encrypted image set, generating target decryption information based on target user information corresponding to the image detection instruction, and decrypting the cardiovascular image set to be detected based on the target decryption information to update the cardiovascular image set to be detected; Determine a target multimodal AI model set corresponding to the cardiovascular image set to be detected based on the image type composition ratio in the cardiovascular image set to be detected and a preset multimodal AI model determination strategy; For each cardiovascular image to be detected in the cardiovascular image set to be detected, a target multimodal AI model corresponding to the cardiovascular image to be detected is obtained in the target multimodal AI model set, and the cardiovascular image to be detected is input into the corresponding target multimodal AI model to obtain an image detection result corresponding to the cardiovascular image to be detected; Based on a preset detection result fusion model and the image detection result corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected, a fusion detection result is obtained.

[0005] In a second aspect, an embodiment of the present invention provides a cardiovascular image detection device based on multimodal AI driving, comprising: a cardiovascular image set acquisition unit to be detected, used for acquiring a cardiovascular image set to be detected corresponding to the image detection instruction in response to the image detection instruction; wherein the cardiovascular image set to be detected includes at least two of the following: cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images, and cardiovascular MRI images; an image set decryption unit, configured to generate target decryption information based on target user information corresponding to the image detection instruction if it is determined that the cardiovascular image set to be detected is an encrypted image set, and decrypt the cardiovascular image set to be detected based on the target decryption information to update the cardiovascular image set to be detected; a target multimodal AI model set acquisition unit, configured to determine a target multimodal AI model set corresponding to the cardiovascular image set to be detected based on the image type composition ratio in the cardiovascular image set to be detected and a preset multimodal AI model determination strategy; an image detection result acquisition unit, for acquiring, for each cardiovascular image to be detected in the cardiovascular image set, a target multimodal AI model corresponding to the cardiovascular image to be detected in the target multimodal AI model set, and inputting the cardiovascular image to be detected into the corresponding target multimodal AI model to obtain an image detection result corresponding to the cardiovascular image to be detected; The detection result fusion unit is used to obtain a fusion detection result based on a preset detection result fusion model and the image detection result corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected.

[0006] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer device executes the computer program, the multimodal AI-driven cardiovascular image detection method as described in the first aspect above is implemented.

[0007] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the cardiovascular image detection method based on multimodal AI driving as described in the first aspect above is implemented.

[0008] Embodiments of the present invention provide a cardiovascular image detection method, device and apparatus driven by multimodal AI, the method comprising: in response to an image detection instruction, obtaining a cardiovascular image set to be detected corresponding to the image detection instruction; wherein the cardiovascular image set to be detected includes at least two of cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images and cardiovascular MRI images; if it is determined that the cardiovascular image set to be detected is an encrypted image set, generating target decryption information based on target user information corresponding to the image detection instruction, and decrypting the cardiovascular image set to be detected based on the target decryption information to update the cardiovascular image set to be detected; determining a target multimodal AI model set corresponding to the cardiovascular image set to be detected based on the image type composition ratio in the cardiovascular image set to be detected and a preset multimodal AI model determination strategy; for each cardiovascular image to be detected in the cardiovascular image set to be detected, obtaining a target multimodal AI model corresponding to the cardiovascular image to be detected in the target multimodal AI model set, and inputting the cardiovascular image to be detected into the corresponding target multimodal AI model to obtain an image detection result corresponding to the cardiovascular image to be detected; obtaining a fusion detection result based on a preset detection result fusion model and the image detection result corresponding to each vascular image to be detected in the cardiovascular image set to be detected. Through the above method, after determining the target multimodal AI model set based on the image type composition ratio for the cardiovascular image set that includes multiple types of cardiovascular images to be detected, multimodal image detection and detection result fusion can be performed accordingly to obtain a fusion detection result that can reflect the multi-dimensional evaluation results and has more comprehensive corresponding information. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0010] Figure 1 A schematic diagram of an application scenario of a cardiovascular image detection method driven by multimodal AI provided in an embodiment of the present invention; Figure 2 A schematic diagram of a process flow of a cardiovascular image detection method based on multimodal AI drive provided in an embodiment of the present invention; Figure 3A schematic diagram of a sub-process of a cardiovascular image detection method based on multimodal AI drive provided in an embodiment of the present invention; Figure 4 A schematic diagram of a sub-process of a cardiovascular image detection method based on multimodal AI drive provided in an embodiment of the present invention; Figure 5 A schematic diagram of a sub-process of a cardiovascular image detection method based on multimodal AI drive provided in an embodiment of the present invention; Figure 6 A schematic block diagram of a cardiovascular image detection device based on multimodal AI drive provided by an embodiment of the present invention; Figure 7 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0013] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0014] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0015] See also Figure 1 and Figure 2 , Figure 1 The embodiment of the present invention provides Figure 1 A schematic diagram of an application scenario of a cardiovascular image detection method driven by multimodal AI provided in an embodiment of the present invention; Figure 2A schematic flow chart of a cardiovascular image detection method based on multimodal AI driving provided in an embodiment of the present invention; the cardiovascular image detection method based on multimodal AI driving is applied to a server 10, and the server 10 is connected to a user terminal 20 used by a doctor user. Figure 2 As shown, the method includes steps S110 to S170.

[0016] S110 . In response to an image detection instruction, obtain a cardiovascular image set to be detected corresponding to the image detection instruction.

[0017] The cardiovascular image set to be detected includes at least two of the following: cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images, and cardiovascular MRI images.

[0018] In this embodiment, in order to obtain cardiovascular images of the user to be detected from more dimensions, at least two of the following can be obtained for the same user to be detected from cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images (CT stands for Computed Tomography, indicating computer tomography), and cardiovascular MRI images (MRI stands for Nuclear Magnetic Resonance Imaging, indicating nuclear magnetic resonance imaging), and form a cardiovascular image set to be detected. Among them, in order to ensure the validity of the images in the cardiovascular image set to be detected, the image acquisition time of each cardiovascular image to be detected included in the cardiovascular image set to be detected can be further determined, and after determining the maximum image acquisition interval time, it is compared with the preset interval time, and when the maximum image acquisition interval time is less than or equal to the preset interval time, it is determined that the cardiovascular image set to be detected corresponds to the detection result of the valid image set.

[0019] Among them, if each cardiovascular image to be detected in the set of cardiovascular images to be detected is received by the server in the form of an encrypted image, it can be pre-agreed that the image acquisition time of each cardiovascular image to be detected is in plain text information, and other information of each cardiovascular image to be detected can be encrypted.

[0020] S120: If it is determined that the cardiovascular image set to be detected is an encrypted image set, target decryption information is generated based on the target user information corresponding to the image detection instruction, and the cardiovascular image set to be detected is decrypted based on the target decryption information to update the cardiovascular image set to be detected.

[0021] In this embodiment, if the server determines that the cardiovascular image set to be detected is an encrypted image set, it means that the server cannot directly process each cardiovascular image to be detected in the cardiovascular image set to be detected. At this time, it is necessary to first generate target decryption information based on the target user information corresponding to the image detection instruction, that is, generate a decryption key, and decrypt the cardiovascular image set to be detected based on the target decryption information, so as to update the cardiovascular image set to be detected. Moreover, the updated cardiovascular image set to be detected is in plain text and can be further processed by the server.

[0022] In one embodiment, if Figure 3 As shown, step S120 includes: S121, obtaining the last identification digit of the user information corresponding to the target user information; S122, based on the last identification bit of the user information, a target decryption model corresponding to the last identification bit of the user information is obtained from multiple local decryption models, and used as the target decryption information; S123, decrypting the cardiovascular image set to be detected based on the target decryption information to obtain a decrypted cardiovascular image set to be detected, and updating the cardiovascular image set to be detected.

[0023] In this embodiment, in order to improve data security, multiple encryption models can be pre-deployed in the server, and then after each user uses his user terminal to establish a communication connection with the server, when the user terminal uploads each user information, the server can obtain a corresponding encryption model based on the last identification bit of the user information and send it to the user terminal for data encryption, and the server also stores the decryption model corresponding to the encryption model for the user information. Afterwards, when a user terminal uploads the target user information corresponding to it (such as including the user ID, user name, user gender, etc.), the last identification bit of the user information corresponding to the target user information is obtained (such as the last character in the user ID can be used as the last identification bit of the user information); then, the target decryption model with the last identification bit of the user information is obtained from the local multiple decryption models with the last identification bit of the user information, and used as the target decryption information; finally, the cardiovascular image set to be detected is decrypted according to the target decryption information to obtain the decrypted cardiovascular image set to be detected in plain text, and the cardiovascular image set to be detected is updated accordingly.

[0024] S130. Determine a target multimodal AI model set corresponding to the cardiovascular image set to be detected based on the image type composition ratio in the cardiovascular image set to be detected and a preset multimodal AI model determination strategy.

[0025] In this embodiment, after obtaining the decrypted and plaintext cardiovascular image set to be detected, the image type composition ratio can be determined first. For example, the total number of images included in the cardiovascular image set to be detected is recorded as N (where N is a positive integer), and the first total number of vascular ultrasound images is recorded as N1, and the second total number of cardiovascular CT images is recorded as N2 (where N1 and N2 are both positive integers). Then, the image type composition ratio can be determined according to the ratio of the first total number of vascular ultrasound images N1 and the second total number of cardiovascular CT images N2 to the total number of images N included in the cardiovascular image set to be detected. More specifically, the image type ratio of vascular ultrasound images is N1 / N and the image type ratio of cardiovascular CT images is N2 / N. After the image type composition ratio is known, the target multimodal AI model set corresponding to the cardiovascular image set to be detected can be determined in combination with the multimodal AI model determination strategy preset locally by the server, so as to achieve a more flexible way to determine the target multimodal AI model set including the target multimodal AI model.

[0026] In one embodiment, as a first embodiment of step S130, as Figure 4 As shown, step S130 includes: S131, obtaining a first image type ratio, a second image type ratio, a third image type ratio and a fourth image type ratio respectively corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected, and determining an image type composition ratio by using the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio; S132: If it is determined based on the multimodal AI model determination strategy that the first image type ratio in the image type composition ratio exceeds a first preset ratio, and the second image type ratio exceeds a second preset ratio, then obtaining a first pre-trained AI model and a second pre-trained AI model to form the target multimodal AI model set; S133. If it is determined based on the multimodal AI model determination strategy that the proportion of the third image type in the image type composition ratio exceeds the third preset ratio, and the proportion of the fourth image type exceeds the fourth preset ratio, then the third pre-trained AI model and the fourth pre-trained AI model are obtained to form the target multimodal AI model set.

[0027] In this embodiment, as the first embodiment of step S130, after the image type composition ratio in the cardiovascular image set to be detected is determined in the server, the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected can be specifically obtained, wherein when an image type ratio is 0, the number of images of the corresponding image type in the cardiovascular image set to be detected is 0. After determining the above four image type ratios, the image type composition ratio can be specifically determined.

[0028] After determining the image type composition ratio, if it is specifically determined that the first image type ratio in the image type composition ratio exceeds a first preset ratio (such as setting the first preset ratio to 20%), and the second image type ratio exceeds a second preset ratio (such as setting the second preset ratio to 30%), then a first pre-trained AI model (such as a residual network model, i.e., a ResNet model) and a second pre-trained AI model (such as a convolutional neural network model combined with an attention mechanism) are obtained to form the target multimodal AI model set.

[0029] After determining the image type composition ratio, if it is specifically determined that the third image type ratio in the image type composition ratio exceeds the third preset ratio (such as setting the third preset ratio to 30%), and the fourth image type ratio exceeds the fourth preset ratio (such as setting the fourth preset ratio to 40%), then the third pre-trained AI model (such as a 3D U-Net model) and the fourth pre-trained AI model (such as a 3DCNN model, i.e., a three-dimensional convolutional neural network model) are obtained to form the target multimodal AI model set. It can be seen that through the above method, the target multimodal AI model set suitable for the current cardiovascular image set to be tested can be quickly determined based on the image type composition ratio and the preset multimodal AI model determination strategy.

[0030] In one embodiment, after step S131, the method further includes: If it is determined based on the multimodal AI model determination strategy that the first image type ratio in the image type composition ratio exceeds a first preset ratio, and the third image type ratio exceeds a third preset ratio, then obtaining a first pre-trained AI model and a third pre-trained AI model to form the target multimodal AI model set; If it is determined based on the multimodal AI model determination strategy that the proportion of the second image type in the image type composition ratio exceeds the second preset ratio, and the proportion of the fourth image type exceeds the fourth preset ratio, then the second pre-trained AI model and the fourth pre-trained AI model are obtained to form the target multimodal AI model set.

[0031] In this embodiment, similarly, after determining the image type composition ratio, if it is specifically determined that the first image type ratio in the image type composition ratio exceeds the first preset ratio, and the third image type ratio exceeds the third preset ratio, then the first pre-trained AI model (such as a residual network model, i.e., a ResNet model) and the third pre-trained AI model (such as a 3D U-Net model) are obtained to form the target multimodal AI model set.

[0032] After determining the image type composition ratio, if it is specifically determined that the second image type ratio in the image type composition ratio exceeds the second preset ratio, and the fourth image type ratio exceeds the fourth preset ratio, then obtain the second pre-trained AI model (such as a convolutional neural network model combined with an attention mechanism) and the fourth pre-trained AI model (such as a 3DCNN model, i.e., a three-dimensional convolutional neural network model) to form the target multimodal AI model set. It can be seen that through the above method, it is also possible to quickly determine the target multimodal AI model set suitable for the current cardiovascular image set to be tested based on the image type composition ratio and the preset multimodal AI model determination strategy.

[0033] In one embodiment, as a second embodiment of step S130, step S130 includes: Acquire a first image type ratio, a second image type ratio, a third image type ratio and a fourth image type ratio respectively corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected, and determine the image type composition ratio by using the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio; Based on the first image type ratio, the second image type ratio, the third image type ratio, and the fourth image type ratio in the image type composition ratio, determine the output result weight value corresponding to each AI model in the initial preset multimodal AI model set, and determine the target multimodal AI model set based on the output result weight value corresponding to each AI model and the initial preset multimodal AI model set.

[0034] In this embodiment, as a second embodiment of step S130, after the image type composition ratio in the cardiovascular image set to be detected is determined in the server, the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected can also be specifically obtained. At this time, the first image type ratio can be directly used as the first output result weight value corresponding to the first pre-trained AI model (such as the residual network model, i.e., the ResNet model) in the initial preset multimodal AI model set, the second image type ratio can be used as the second output result weight value corresponding to the second pre-trained AI model (such as the convolutional neural network model combined with the attention mechanism) in the initial preset multimodal AI model set, the third image type ratio can be used as the third output result weight value corresponding to the third pre-trained AI model (such as the 3DU-Net model) in the initial preset multimodal AI model set, and the fourth image type ratio can be used as the fourth output result weight value corresponding to the fourth pre-trained AI model (such as the 3DCNN model, i.e., the three-dimensional convolutional neural network model) in the initial preset multimodal AI model set. After determining the output result weight values ​​corresponding to each AI model in the initial preset multimodal AI model set in the above manner, the target multimodal AI model set used for image detection can be determined.

[0035] S140. For each cardiovascular image to be detected in the cardiovascular image set, obtain a target multimodal AI model corresponding to the cardiovascular image to be detected in the target multimodal AI model set, and input the cardiovascular image to be detected into the corresponding target multimodal AI model to obtain an image detection result corresponding to the cardiovascular image to be detected.

[0036] In this embodiment, after the target multimodal AI model set is determined in the server, image detection can be performed for each cardiovascular image to be detected in the cardiovascular image set to be detected to obtain the corresponding image detection result. For example, taking the image detection process of a cardiovascular image to be detected in the cardiovascular image set to be detected as an example, after obtaining the target multimodal AI model corresponding to the cardiovascular image to be detected in the target multimodal AI model set (specifically, the corresponding target multimodal AI model can be selected from the first pre-trained AI model to the fourth pre-trained AI model according to the image type of the cardiovascular image to be detected), the cardiovascular image to be detected is then input into the target multimodal AI model to obtain the image detection result corresponding to the cardiovascular image to be detected. Among them, the image detection result corresponding to the cardiovascular image to be detected is a probability value with a value range of [0, 1], and represents the probability value of the cardiovascular image to be detected corresponding to the corresponding image classification result. The image detection process of other cardiovascular images to be detected in the cardiovascular image set to be detected also refers to the above process. After completing the image detection of each cardiovascular image to be detected in the cardiovascular image set to be detected, subsequent operations can be performed.

[0037] In one embodiment, before step S140, the method further includes: For each cardiovascular image to be detected in the set of cardiovascular images to be detected, image preprocessing is performed on the cardiovascular image to be detected based on a preset set of image preprocessing strategies to update the cardiovascular image to be detected.

[0038] In this embodiment, still referring to the above example, taking the image preprocessing process of a cardiovascular image to be detected in the cardiovascular image set to be detected as an example, after the image type of the cardiovascular image to be detected is obtained, the target image preprocessing strategy corresponding to the cardiovascular image to be detected can also be obtained in the image preprocessing strategy set. For example, if the image type of the cardiovascular image to be detected is a cardiovascular ultrasound image, the corresponding target image preprocessing strategy includes image cropping (which corresponds to a first preset cropped image size), image denoising and normalization processing; if the image type of the cardiovascular image to be detected is a cardiovascular X-ray image, the corresponding target image preprocessing strategy includes image denoising and normalization processing; if the image type of the cardiovascular image to be detected is a cardiovascular CT image, the corresponding target image preprocessing strategy includes image cropping (which corresponds to a second preset cropped image size), image denoising and normalization processing; if the image type of the cardiovascular image to be detected is a cardiovascular MRI image, the corresponding target image preprocessing strategy includes image cropping (which corresponds to a third preset cropped image size), image denoising and normalization processing. After the cardiovascular image to be detected is preprocessed by the target image preprocessing strategy in the image preprocessing strategy set, the image preprocessing of the cardiovascular image to be detected is timely realized, so that subsequent image detection can be performed more accurately.

[0039] S150, obtaining a fusion detection result based on a preset detection result fusion model and the image detection result corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected.

[0040] In this embodiment, after the server obtains the image detection result corresponding to each of the to-be-detected blood vessel images in the to-be-detected cardiovascular image set, it can also fuse the image detection results according to the detection result fusion model to obtain a fused detection result. It can be seen that through the above method, the fused detection result corresponding to the to-be-detected cardiovascular image set can be quickly obtained.

[0041] In one embodiment, if Figure 5 As shown, step S150 includes: S151, arranging the image detection results corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected in the chronological order of image acquisition time based on the detection result fusion model, and forming a multimodal feature vector; S152: Input the multimodal feature vector into the long short-term memory network corresponding to the detection result fusion model to obtain the fusion detection result corresponding to the multimodal feature vector.

[0042] In this embodiment, after obtaining the image detection result corresponding to each vascular image to be detected in the cardiovascular image set to be detected (i.e., corresponding to a probability value between 0 and 1), the above-mentioned image detection results can also be arranged in the chronological order of the image acquisition time, and form a multimodal feature vector. After that, the multimodal feature vector is input into the long short-term memory network corresponding to the detection result fusion model (more specifically, the self-attention mechanism can also be integrated into the long short-term memory network) to obtain the fusion detection result corresponding to the multimodal feature vector (and the fusion detection result is also a probability value between 0 and 1). Finally, the fusion detection result can also be compared with multiple preset value intervals (the union of the above multiple preset value intervals is also [0,1]) to determine the target preset value interval corresponding to the fusion detection result in the multiple preset value intervals, and the classification prompt information corresponding to the target preset value interval (such as the corresponding low risk type) is used as the current classification prompt information sent by the server to the user terminal.

[0043] It can be seen that the embodiment of the method can determine the target multimodal AI model set based on the image type composition ratio for the cardiovascular image set to be detected including multiple types of cardiovascular images to be detected, and then perform multimodal image detection and detection result fusion accordingly to obtain a fusion detection result that can reflect the multi-dimensional evaluation results and has more comprehensive corresponding information.

[0044] The embodiment of the present invention also provides a cardiovascular image detection device based on multimodal AI driving, which can be configured in a server, and is used to execute any embodiment of the aforementioned cardiovascular image detection method based on multimodal AI driving. Specifically, please refer to Figure 6 , Figure 6 A schematic block diagram of a cardiovascular image detection device based on multimodal AI drive provided by an embodiment of the present invention. Figure 6 As shown, the cardiovascular image detection device 100 driven by multimodal AI includes a cardiovascular image set acquisition unit 110 to be detected, an image set decryption unit 120, a target multimodal AI model set acquisition unit 130, an image detection result acquisition unit 140 and a detection result fusion unit 150.

[0045] The cardiovascular image set to be detected acquiring unit 110 is used to acquire the cardiovascular image set to be detected corresponding to the image detection instruction in response to the image detection instruction.

[0046] The cardiovascular image set to be detected includes at least two of the following: cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images, and cardiovascular MRI images.

[0047] In this embodiment, in order to obtain cardiovascular images of the user to be detected from more dimensions, at least two of the following can be obtained for the same user to be detected from cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images (CT stands for Computed Tomography, indicating computer tomography), and cardiovascular MRI images (MRI stands for Nuclear Magnetic Resonance Imaging, indicating nuclear magnetic resonance imaging), and form a cardiovascular image set to be detected. Among them, in order to ensure the validity of the images in the cardiovascular image set to be detected, the image acquisition time of each cardiovascular image to be detected included in the cardiovascular image set to be detected can be further determined, and after determining the maximum image acquisition interval time, it is compared with the preset interval time, and when the maximum image acquisition interval time is less than or equal to the preset interval time, it is determined that the cardiovascular image set to be detected corresponds to the detection result of the valid image set.

[0048] Among them, if each cardiovascular image to be detected in the set of cardiovascular images to be detected is received by the server in the form of an encrypted image, it can be pre-agreed that the image acquisition time of each cardiovascular image to be detected is in plain text information, and other information of each cardiovascular image to be detected can be encrypted.

[0049] The image set decryption unit 120 is used to generate target decryption information based on the target user information corresponding to the image detection instruction if it is determined that the cardiovascular image set to be detected is an encrypted image set, and decrypt the cardiovascular image set to be detected based on the target decryption information to update the cardiovascular image set to be detected.

[0050] In this embodiment, if the server determines that the cardiovascular image set to be detected is an encrypted image set, it means that the server cannot directly process each cardiovascular image to be detected in the cardiovascular image set to be detected. At this time, it is necessary to first generate target decryption information based on the target user information corresponding to the image detection instruction, that is, generate a decryption key, and decrypt the cardiovascular image set to be detected based on the target decryption information, so as to update the cardiovascular image set to be detected. Moreover, the updated cardiovascular image set to be detected is in plain text and can be further processed by the server.

[0051] In one embodiment, the image set decryption unit 120 is specifically configured to: Obtain the last identification digit of the user information corresponding to the target user information; Based on the last identification bit of the user information, a target decryption model corresponding to the last identification bit of the user information is obtained from multiple local decryption models, and used as the target decryption information; The cardiovascular image set to be detected is decrypted based on the target decryption information to obtain the decrypted cardiovascular image set to be detected, and the cardiovascular image set to be detected is updated.

[0052] In this embodiment, in order to improve data security, multiple encryption models can be pre-deployed in the server, and then after each user uses his user terminal to establish a communication connection with the server, when the user terminal uploads each user information, the server can obtain a corresponding encryption model based on the last identification bit of the user information and send it to the user terminal for data encryption, and the server also stores the decryption model corresponding to the encryption model for the user information. Afterwards, when a user terminal uploads the target user information corresponding to it (such as including the user ID, user name, user gender, etc.), the last identification bit of the user information corresponding to the target user information is obtained (such as the last character in the user ID can be used as the last identification bit of the user information); then, the target decryption model with the last identification bit of the user information is obtained from the local multiple decryption models with the last identification bit of the user information, and used as the target decryption information; finally, the cardiovascular image set to be detected is decrypted according to the target decryption information to obtain the decrypted cardiovascular image set to be detected in plain text, and the cardiovascular image set to be detected is updated accordingly.

[0053] The target multimodal AI model set acquisition unit 130 is used to determine the target multimodal AI model set corresponding to the cardiovascular image set to be detected based on the image type composition ratio in the cardiovascular image set to be detected and a preset multimodal AI model determination strategy.

[0054] In this embodiment, after obtaining the decrypted and plaintext cardiovascular image set to be detected, the image type composition ratio can be determined first. For example, the total number of images included in the cardiovascular image set to be detected is recorded as N (where N is a positive integer), and the first total number of vascular ultrasound images is recorded as N1, and the second total number of cardiovascular CT images is recorded as N2 (where N1 and N2 are both positive integers). Then, the image type composition ratio can be determined according to the ratio of the first total number of vascular ultrasound images N1 and the second total number of cardiovascular CT images N2 to the total number of images N included in the cardiovascular image set to be detected. More specifically, the image type ratio of vascular ultrasound images is N1 / N and the image type ratio of cardiovascular CT images is N2 / N. After the image type composition ratio is known, the target multimodal AI model set corresponding to the cardiovascular image set to be detected can be determined in combination with the multimodal AI model determination strategy preset locally by the server, so as to achieve a more flexible way to determine the target multimodal AI model set including the target multimodal AI model.

[0055] In one embodiment, as a first embodiment of the target multimodal AI model set acquisition unit 130, the target multimodal AI model set acquisition unit 130 is specifically used to: Acquire a first image type ratio, a second image type ratio, a third image type ratio and a fourth image type ratio respectively corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected, and determine the image type composition ratio by using the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio; If it is determined based on the multimodal AI model determination strategy that the first image type ratio in the image type composition ratio exceeds a first preset ratio, and the second image type ratio exceeds a second preset ratio, then obtaining a first pre-trained AI model and a second pre-trained AI model to form the target multimodal AI model set; If it is determined based on the multimodal AI model determination strategy that the proportion of the third image type in the image type composition ratio exceeds the third preset ratio, and the proportion of the fourth image type exceeds the fourth preset ratio, then the third pre-trained AI model and the fourth pre-trained AI model are obtained to form the target multimodal AI model set.

[0056] In this embodiment, as the first embodiment of the target multimodal AI model set acquisition unit 130, after determining the image type composition ratio in the cardiovascular image set to be detected in the server, the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio corresponding to the cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images and cardiovascular MRI images in the cardiovascular image set to be detected can be specifically obtained, wherein when an image type ratio is 0, the corresponding image type in the table has a corresponding number of 0 in the cardiovascular image set to be detected. After determining the above four image type ratios, the image type composition ratio can be specifically determined.

[0057] After determining the image type composition ratio, if it is specifically determined that the first image type ratio in the image type composition ratio exceeds a first preset ratio (such as setting the first preset ratio to 20%), and the second image type ratio exceeds a second preset ratio (such as setting the second preset ratio to 30%), then a first pre-trained AI model (such as a residual network model, i.e., a ResNet model) and a second pre-trained AI model (such as a convolutional neural network model combined with an attention mechanism) are obtained to form the target multimodal AI model set.

[0058] After determining the image type composition ratio, if it is specifically determined that the third image type ratio in the image type composition ratio exceeds the third preset ratio (such as setting the third preset ratio to 30%), and the fourth image type ratio exceeds the fourth preset ratio (such as setting the fourth preset ratio to 40%), then the third pre-trained AI model (such as a 3D U-Net model) and the fourth pre-trained AI model (such as a 3DCNN model, i.e., a three-dimensional convolutional neural network model) are obtained to form the target multimodal AI model set. It can be seen that through the above method, the target multimodal AI model set suitable for the current cardiovascular image set to be tested can be quickly determined based on the image type composition ratio and the preset multimodal AI model determination strategy.

[0059] In one embodiment, the target multimodal AI model set acquisition unit 130 is further used to: If it is determined based on the multimodal AI model determination strategy that the first image type ratio in the image type composition ratio exceeds a first preset ratio, and the third image type ratio exceeds a third preset ratio, then obtaining a first pre-trained AI model and a third pre-trained AI model to form the target multimodal AI model set; If it is determined based on the multimodal AI model determination strategy that the proportion of the second image type in the image type composition ratio exceeds the second preset ratio, and the proportion of the fourth image type exceeds the fourth preset ratio, then the second pre-trained AI model and the fourth pre-trained AI model are obtained to form the target multimodal AI model set.

[0060] In this embodiment, similarly, after determining the image type composition ratio, if it is specifically determined that the first image type ratio in the image type composition ratio exceeds the first preset ratio, and the third image type ratio exceeds the third preset ratio, then the first pre-trained AI model (such as a residual network model, i.e., a ResNet model) and the third pre-trained AI model (such as a 3D U-Net model) are obtained to form the target multimodal AI model set.

[0061] After determining the image type composition ratio, if it is specifically determined that the second image type ratio in the image type composition ratio exceeds the second preset ratio, and the fourth image type ratio exceeds the fourth preset ratio, then obtain the second pre-trained AI model (such as a convolutional neural network model combined with an attention mechanism) and the fourth pre-trained AI model (such as a 3DCNN model, i.e., a three-dimensional convolutional neural network model) to form the target multimodal AI model set. It can be seen that through the above method, it is also possible to quickly determine the target multimodal AI model set suitable for the current cardiovascular image set to be tested based on the image type composition ratio and the preset multimodal AI model determination strategy.

[0062] In one embodiment, as a second embodiment of the target multimodal AI model set acquisition unit 130, the target multimodal AI model set acquisition unit 130 is specifically used to: Acquire a first image type ratio, a second image type ratio, a third image type ratio and a fourth image type ratio respectively corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected, and determine the image type composition ratio by using the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio; Based on the first image type ratio, the second image type ratio, the third image type ratio, and the fourth image type ratio in the image type composition ratio, determine the output result weight value corresponding to each AI model in the initial preset multimodal AI model set, and determine the target multimodal AI model set based on the output result weight value corresponding to each AI model and the initial preset multimodal AI model set.

[0063] In this embodiment, as a second embodiment of the target multimodal AI model set acquisition unit 130, after the image type composition ratio of the cardiovascular image set to be detected is determined in the server, the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected can also be specifically obtained. At this time, the first image type ratio can be directly used as the first output result weight value corresponding to the first pre-trained AI model (such as a residual network model, i.e., a ResNet model) in the initial preset multimodal AI model set, the second image type ratio can be used as the second output result weight value corresponding to the second pre-trained AI model (such as a convolutional neural network model combined with an attention mechanism) in the initial preset multimodal AI model set, the third image type ratio can be used as the third output result weight value corresponding to the third pre-trained AI model (such as a 3D U-Net model) in the initial preset multimodal AI model set, and the fourth image type ratio can be used as the fourth output result weight value corresponding to the fourth pre-trained AI model (such as a 3DCNN model, i.e., a three-dimensional convolutional neural network model) in the initial preset multimodal AI model set. After determining the output result weight values ​​corresponding to each AI model in the initial preset multimodal AI model set in the above manner, the target multimodal AI model set used for image detection can be determined.

[0064] The image detection result acquisition unit 140 is used to obtain, for each cardiovascular image to be detected in the cardiovascular image set, a target multimodal AI model corresponding to the cardiovascular image to be detected in the target multimodal AI model set, and input the cardiovascular image to be detected into the corresponding target multimodal AI model to obtain an image detection result corresponding to the cardiovascular image to be detected.

[0065] In this embodiment, after the target multimodal AI model set is determined in the server, image detection can be performed for each cardiovascular image to be detected in the cardiovascular image set to be detected to obtain the corresponding image detection result. For example, taking the image detection process of a cardiovascular image to be detected in the cardiovascular image set to be detected as an example, after obtaining the target multimodal AI model corresponding to the cardiovascular image to be detected in the target multimodal AI model set (specifically, the corresponding target multimodal AI model can be selected from the first pre-trained AI model to the fourth pre-trained AI model according to the image type of the cardiovascular image to be detected), the cardiovascular image to be detected is then input into the target multimodal AI model to obtain the image detection result corresponding to the cardiovascular image to be detected. Among them, the image detection result corresponding to the cardiovascular image to be detected is a probability value with a value range of [0, 1], and represents the probability value of the cardiovascular image to be detected corresponding to the corresponding image classification result. The image detection process of other cardiovascular images to be detected in the cardiovascular image set to be detected also refers to the above process. After completing the image detection of each cardiovascular image to be detected in the cardiovascular image set to be detected, subsequent operations can be performed.

[0066] In one embodiment, the cardiovascular image detection device 100 driven by multimodal AI further includes: The image preprocessing unit is used to perform image preprocessing on each cardiovascular image to be detected in the cardiovascular image set based on a preset image preprocessing strategy set to update the cardiovascular image to be detected.

[0067] In this embodiment, still referring to the above example, taking the image preprocessing process of a cardiovascular image to be detected in the cardiovascular image set to be detected as an example, after the image type of the cardiovascular image to be detected is obtained, the target image preprocessing strategy corresponding to the cardiovascular image to be detected can also be obtained in the image preprocessing strategy set. For example, if the image type of the cardiovascular image to be detected is a cardiovascular ultrasound image, the corresponding target image preprocessing strategy includes image cropping (which corresponds to a first preset cropped image size), image denoising and normalization processing; if the image type of the cardiovascular image to be detected is a cardiovascular X-ray image, the corresponding target image preprocessing strategy includes image denoising and normalization processing; if the image type of the cardiovascular image to be detected is a cardiovascular CT image, the corresponding target image preprocessing strategy includes image cropping (which corresponds to a second preset cropped image size), image denoising and normalization processing; if the image type of the cardiovascular image to be detected is a cardiovascular MRI image, the corresponding target image preprocessing strategy includes image cropping (which corresponds to a third preset cropped image size), image denoising and normalization processing. After the cardiovascular image to be detected is preprocessed by the target image preprocessing strategy in the image preprocessing strategy set, the image preprocessing of the cardiovascular image to be detected is timely realized, so that subsequent image detection can be performed more accurately.

[0068] The detection result fusion unit 150 is used to obtain a fused detection result based on a preset detection result fusion model and the image detection result corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected.

[0069] In this embodiment, after the server obtains the image detection result corresponding to each of the to-be-detected blood vessel images in the to-be-detected cardiovascular image set, it can also fuse the image detection results according to the detection result fusion model to obtain a fused detection result. It can be seen that through the above method, the fused detection result corresponding to the to-be-detected cardiovascular image set can be quickly obtained.

[0070] In one embodiment, the detection result fusion unit 150 is specifically used for: Based on the detection result fusion model, the image detection results corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected are arranged in the temporal order of image acquisition time to form a multimodal feature vector; The multimodal feature vector is input into the long short-term memory network corresponding to the detection result fusion model to obtain the fusion detection result corresponding to the multimodal feature vector.

[0071] In this embodiment, after obtaining the image detection result corresponding to each vascular image to be detected in the cardiovascular image set to be detected (i.e., corresponding to a probability value between 0 and 1), the above-mentioned image detection results can also be arranged in the chronological order of the image acquisition time, and form a multimodal feature vector. After that, the multimodal feature vector is input into the long short-term memory network corresponding to the detection result fusion model (more specifically, the self-attention mechanism can also be integrated into the long short-term memory network) to obtain the fusion detection result corresponding to the multimodal feature vector (and the fusion detection result is also a probability value between 0 and 1). Finally, the fusion detection result can also be compared with multiple preset value intervals (the union of the above multiple preset value intervals is also [0,1]) to determine the target preset value interval corresponding to the fusion detection result in the multiple preset value intervals, and the classification prompt information corresponding to the target preset value interval (such as the corresponding low risk type) is used as the current classification prompt information sent by the server to the user terminal.

[0072] It can be seen that the embodiment of the device can determine the target multimodal AI model set based on the image type composition ratio for a cardiovascular image set to be detected that includes multiple types of cardiovascular images to be detected, and then perform multimodal image detection and detection result fusion accordingly to obtain a fusion detection result that can reflect the multi-dimensional evaluation results and has more comprehensive corresponding information.

[0073] The above-mentioned cardiovascular image detection device based on multimodal AI driving can be implemented in the form of a computer program, which can be used in Figure 7 Runs on the computer device shown.

[0074] See also Figure 7 , Figure 7 is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device integrates any multimodal AI-driven cardiovascular image detection device provided by an embodiment of the present invention.

[0075] See also Figure 7 The computer device 400 includes a processor 402 , a memory and a network interface 405 connected via a system bus 401 , wherein the memory may include a storage medium 403 and an internal memory 404 .

[0076] The storage medium 403 may store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions, which, when executed, may enable the processor 402 to execute the above-mentioned cardiovascular image detection method based on multimodal AI driving.

[0077] The processor 402 is used to provide computing and control capabilities to support the operation of the entire computer device.

[0078] The internal memory 404 provides an environment for the operation of the computer program 4032 in the storage medium 403. When the computer program 4032 is executed by the processor 402, the processor 402 can execute the above-mentioned cardiovascular image detection method based on multimodal AI driving.

[0079] The network interface 405 is used to communicate with other devices over the network. Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0080] The processor 402 is used to run a computer program 4032 stored in the memory to implement the multimodal AI-driven cardiovascular image detection method as described above.

[0081] It should be understood that in the embodiment of the present invention, the processor 402 may be a central processing unit (CPU), and the processor 402 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0082] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer device to implement the process steps of the embodiment of the above method.

[0083] Therefore, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the cardiovascular image detection method based on multimodal AI drive as described above.

[0084] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0085] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0086] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0087] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0088] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0089] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A cardiovascular image detection method based on multimodal AI driving, characterized in that: include: In response to an image detection instruction, acquiring a cardiovascular image set to be detected corresponding to the image detection instruction; wherein the cardiovascular image set to be detected includes at least two of cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images, and cardiovascular MRI images; If it is determined that the cardiovascular image set to be detected is an encrypted image set, generating target decryption information based on target user information corresponding to the image detection instruction, and decrypting the cardiovascular image set to be detected based on the target decryption information to update the cardiovascular image set to be detected; Determine a target multimodal AI model set corresponding to the cardiovascular image set to be detected based on the image type composition ratio in the cardiovascular image set to be detected and a preset multimodal AI model determination strategy; For each cardiovascular image to be detected in the cardiovascular image set to be detected, a target multimodal AI model corresponding to the cardiovascular image to be detected is obtained in the target multimodal AI model set, and the cardiovascular image to be detected is input into the corresponding target multimodal AI model to obtain an image detection result corresponding to the cardiovascular image to be detected; Based on a preset detection result fusion model and the image detection result corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected, a fusion detection result is obtained.

2. The cardiovascular image detection method based on multimodal AI drive according to claim 1, characterized in that: The generating target decryption information based on the target user information corresponding to the image detection instruction, and decrypting the cardiovascular image set to be detected based on the target decryption information to update the cardiovascular image set to be detected, comprises: Obtain the last identification digit of the user information corresponding to the target user information; Based on the last identification bit of the user information, a target decryption model corresponding to the last identification bit of the user information is obtained from multiple local decryption models, and used as the target decryption information; The cardiovascular image set to be detected is decrypted based on the target decryption information to obtain the decrypted cardiovascular image set to be detected, and the cardiovascular image set to be detected is updated.

3. The cardiovascular image detection method based on multimodal AI drive according to claim 1, characterized in that: The step of determining a target multimodal AI model set corresponding to the cardiovascular image set to be detected based on the image type composition ratio in the cardiovascular image set to be detected and a preset multimodal AI model determination strategy includes: Acquire a first image type ratio, a second image type ratio, a third image type ratio and a fourth image type ratio respectively corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected, and determine the image type composition ratio by using the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio; If it is determined based on the multimodal AI model determination strategy that the first image type ratio in the image type composition ratio exceeds a first preset ratio, and the second image type ratio exceeds a second preset ratio, then obtaining a first pre-trained AI model and a second pre-trained AI model to form the target multimodal AI model set; If it is determined based on the multimodal AI model determination strategy that the proportion of the third image type in the image type composition ratio exceeds the third preset ratio, and the proportion of the fourth image type exceeds the fourth preset ratio, then the third pre-trained AI model and the fourth pre-trained AI model are obtained to form the target multimodal AI model set.

4. The cardiovascular image detection method based on multimodal AI drive according to claim 3, characterized in that: After the step of obtaining the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio respectively corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected, and determining the image type composition ratio by using the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio, the method further includes: If it is determined based on the multimodal AI model determination strategy that the first image type ratio in the image type composition ratio exceeds a first preset ratio, and the third image type ratio exceeds a third preset ratio, then obtaining a first pre-trained AI model and a third pre-trained AI model to form the target multimodal AI model set; If it is determined based on the multimodal AI model determination strategy that the proportion of the second image type in the image type composition ratio exceeds the second preset ratio, and the proportion of the fourth image type exceeds the fourth preset ratio, then the second pre-trained AI model and the fourth pre-trained AI model are obtained to form the target multimodal AI model set.

5. The cardiovascular image detection method based on multimodal AI drive according to claim 1, characterized in that: The step of determining a target multimodal AI model set corresponding to the cardiovascular image set to be detected based on the image type composition ratio in the cardiovascular image set to be detected and a preset multimodal AI model determination strategy includes: Acquire a first image type ratio, a second image type ratio, a third image type ratio and a fourth image type ratio respectively corresponding to the cardiovascular ultrasound image, the cardiovascular X-ray image, the cardiovascular CT image and the cardiovascular MRI image in the cardiovascular image set to be detected, and determine the image type composition ratio by using the first image type ratio, the second image type ratio, the third image type ratio and the fourth image type ratio; Based on the first image type ratio, the second image type ratio, the third image type ratio, and the fourth image type ratio in the image type composition ratio, determine the output result weight value corresponding to each AI model in the initial preset multimodal AI model set, and determine the target multimodal AI model set based on the output result weight value corresponding to each AI model and the initial preset multimodal AI model set.

6. The cardiovascular image detection method based on multimodal AI drive according to claim 1, characterized in that: Before the step of acquiring, for each cardiovascular image to be detected in the cardiovascular image set, a target multimodal AI model corresponding to the cardiovascular image to be detected in the target multimodal AI model set, and inputting the cardiovascular image to be detected into the corresponding target multimodal AI model to obtain an image detection result corresponding to the cardiovascular image to be detected, the method further includes: For each cardiovascular image to be detected in the set of cardiovascular images to be detected, image preprocessing is performed on the cardiovascular image to be detected based on a preset set of image preprocessing strategies to update the cardiovascular image to be detected.

7. The cardiovascular image detection method based on multimodal AI drive according to claim 1, characterized in that: The obtaining of the fusion detection result based on the preset detection result fusion model and the image detection result corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected includes: Based on the detection result fusion model, the image detection results corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected are arranged in the temporal order of image acquisition time to form a multimodal feature vector; The multimodal feature vector is input into the long short-term memory network corresponding to the detection result fusion model to obtain the fusion detection result corresponding to the multimodal feature vector.

8. A cardiovascular image detection device based on multimodal AI drive, characterized in that: include: a cardiovascular image set acquisition unit to be detected, used for acquiring a cardiovascular image set to be detected corresponding to the image detection instruction in response to the image detection instruction; wherein the cardiovascular image set to be detected includes at least two of the following: cardiovascular ultrasound images, cardiovascular X-ray images, cardiovascular CT images, and cardiovascular MRI images; an image set decryption unit, configured to generate target decryption information based on target user information corresponding to the image detection instruction if it is determined that the cardiovascular image set to be detected is an encrypted image set, and decrypt the cardiovascular image set to be detected based on the target decryption information to update the cardiovascular image set to be detected; a target multimodal AI model set acquisition unit, configured to determine a target multimodal AI model set corresponding to the cardiovascular image set to be detected based on the image type composition ratio in the cardiovascular image set to be detected and a preset multimodal AI model determination strategy; an image detection result acquisition unit, for acquiring, for each cardiovascular image to be detected in the cardiovascular image set, a target multimodal AI model corresponding to the cardiovascular image to be detected in the target multimodal AI model set, and inputting the cardiovascular image to be detected into the corresponding target multimodal AI model to obtain an image detection result corresponding to the cardiovascular image to be detected; The detection result fusion unit is used to obtain a fusion detection result based on a preset detection result fusion model and the image detection result corresponding to each blood vessel image to be detected in the cardiovascular image set to be detected.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer device executes the computer program, the multimodal AI-driven cardiovascular image detection method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the multimodal AI-driven cardiovascular image detection method as described in any one of claims 1 to 7.

Images