A multimodal image clustering segmentation method and system based on outsourced computing

Abstract

The present invention proposes a multimodal image clustering segmentation method and system based on outsourced computing, which relates to the field of data security technology, and specifically includes: a local computing device obtains multiple images of different modalities to be segmented, and preprocesses them into a multimodal image pixel matrix; based on the multimodal image pixel matrix, the local computing device adopts an improved clustering algorithm to iteratively calculate the classification to which the pixels in the image belong, and obtains the optimal clustering result; according to the classification to which each pixel in the clustering result belongs, the local computing device segments the regions corresponding to the classification from the multiple images of different modalities, and obtains the final result of multimodal image segmentation; on the basis of ensuring the security of outsourced computing of sensitive data, the present invention solves the problem that computing devices of medical institutions with limited resources are difficult to complete the task of multimodal medical image segmentation.

Description

Technical Field

[0001] The present invention belongs to the technical field of data security, and in particular relates to a multimodal image clustering segmentation method and system based on outsourced computing. Background Art

[0002] With the continuous development of medical imaging technology, multimodal medical image segmentation methods have been widely studied and applied in various medical scenarios such as diagnosis, detection, and treatment. Common medical images such as computed tomography (CT) images, magnetic resonance imaging (MRI) images, positron emission tomography (PET) images, and ultrasound (US) images provide physiological information inside the human body from different perspectives. The multimodal medical image segmentation method based on the fuzzy C-means and particle swarm optimization clustering algorithm (FCM-PSO) utilizes the rich complementary information in medical images of different modalities, classifies pixels according to the similarity of the features between each pixel, and automatically and effectively segments the tissue structure or lesion area.

[0003] Since high-resolution medical images take up a lot of storage space, high-precision clustering image segmentation algorithms require a lot of computing resources, and local computing devices in medical institutions have resource-constrained characteristics, multimodal medical image segmentation based on clustering algorithms faces the problem of mismatching demand and resources. In the field of machine learning, when local devices cannot meet the resource requirements of algorithm execution, a common solution is to use outsourced computing technology to deliver complex computing tasks to cloud servers. For specific machine learning algorithms, lightweight calculations such as vector addition and subtraction and threshold comparison are performed locally, and complex calculations such as large-scale matrix multiplication and solving linear equations are performed on cloud servers.

[0004] However, medical images contain patients' private information. When medical institutions use outsourcing methods to perform multimodal image segmentation tasks, sensitive data is exposed to security risks such as illegal eavesdropping and malicious tampering. In the transmission process, multimodal medical image data may be intercepted by unauthorized third parties during the communication between medical institution equipment and cloud servers, and the patient information may be used for malicious activities. In the calculation process, untrusted cloud servers may maliciously spy on private information and deliberately modify calculation data for commercial interests, or randomly return erroneous results to save resources, resulting in deviations in medical analysis and disease diagnosis.

[0005] Therefore, existing multimodal image clustering and segmentation schemes face the trade-off problem between resource mismatch and security risks, and are difficult to be efficiently applied in medical scenarios. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a multimodal image clustering segmentation method and system based on outsourced computing, which solves the problem that computing equipment of medical institutions with limited resources is difficult to complete multimodal medical image segmentation tasks on the basis of ensuring the security of outsourced computing of sensitive data.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] A first aspect of the present invention provides a multimodal image clustering segmentation method based on outsourced computing.

[0009] A multimodal image clustering segmentation method based on outsourced computing, comprising:

[0010] The local computing device obtains a plurality of images of different modalities to be segmented, and pre-processes them into a multi-modal image pixel matrix;

[0011] Based on the multimodal image pixel matrix, the local computing device uses an improved clustering algorithm to iteratively calculate the classification to which the pixels in the image belong, and obtains the optimal clustering result;

[0012] According to the classification to which each pixel point in the clustering result belongs, the local computing device segments the corresponding regions from multiple images of different modalities to obtain the final result of multimodal image segmentation;

[0013] Among them, the improved clustering algorithm is to split the distance calculation method between pixel points and cluster centers into two steps: matrix multiplication operation and distance calculation based on the result of matrix multiplication operation. The matrix multiplication operation is outsourced to cloud computing devices for calculation using a safe outsourcing method based on matrix blinding.

[0014] Furthermore, the preprocessing is to perform grayscale conversion and numerical normalization on the image, and form a multi-modal image pixel matrix from the grayscale values ​​of the same pixel point in different modal images.

[0015] Furthermore, the improved clustering algorithm adopts the clustering algorithm FCM-PSO based on fuzzy C-means and particle swarm optimization, and the specific steps are as follows:

[0016] (1) Given the number of clusters, initialize the cluster center matrix, assign values ​​to each particle, and randomly generate the initial velocity of the particle;

[0017] (2) Calculate the membership degree of each particle, update all cluster centers, calculate the fitness value of each particle, and update the individual extreme value;

[0018] (3) Find the global extreme value and the position of the global extreme value based on the individual extreme values ​​of each particle;

[0019] (4) Update the particle speed according to the speed formula of the particle swarm optimization algorithm and limit it to the maximum speed;

[0020] (5) Update the position of the particle according to the position formula of the particle swarm optimization algorithm;

[0021] (6) If the termination condition is not met, return to step (2) to continue iterative calculation; if the termination condition is met, output the position of the optimal particle as the optimal clustering result;

[0022] The calculation of membership is based on a matrix blinding safety outsourcing method, which calculates the distance between the pixel point and the cluster center, thereby updating the membership matrix between each pixel point and each cluster center.

[0023] Furthermore, the calculating the distance between the pixel point and the cluster center includes:

[0024] The outsourcing method of splitting the Euclidean distance calculation process is adopted to split the distance calculation method between the pixel point and the cluster center into two steps: matrix multiplication operation and distance calculation based on the result of matrix multiplication operation.

[0025] A matrix blinding method based on a sparse key matrix is ​​used to generate encryption keys and decryption keys for securely transmitting data between a local computing device and a cloud computing device;

[0026] A secure outsourcing method based on matrix blinding is adopted to outsource matrix multiplication operations to cloud computing devices through encryption keys, and the blinded calculation results are returned to the local computing device for subsequent distance calculations.

[0027] Furthermore, the distance between the pixel point and the cluster center is calculated as follows:

[0028] Dynamically generate encryption keys and decryption keys based on the generated key matrix and its inverse matrix;

[0029] The local computing device blinds the multimodal image pixel matrix and the cluster center matrix using the encryption key and sends them to the cloud computing device;

[0030] Based on the blinded multimodal image pixel matrix and cluster center matrix, the cloud computing device calculates the matrix multiplication operation and returns the blinded calculation result to the local computing device;

[0031] The local computing device verifies the result returned by the cloud computing device;

[0032] The local computing device uses the decryption key to perform a deblinding operation on the returned result to obtain a matrix multiplication result;

[0033] The local computing device calculates the distance between the pixel and the cluster center based on the multimodal image pixel matrix, the cluster center matrix and the matrix multiplication result.

[0034] Furthermore, the dynamic generation of encryption keys and decryption keys is specifically as follows:

[0035] Randomly generate a preset number of integers to construct a local random matrix and its inverse matrix;

[0036] Based on the randomly selected coordinates and step sizes, the local random matrix and its inverse matrix are embedded into the key matrix and its inverse matrix respectively;

[0037] According to the step size, the positions of the elements in the key matrix and its inverse matrix are changed to obtain the final key matrix and its inverse matrix;

[0038] Based on the final key matrix and its inverse matrix, encryption key and decryption key are generated.

[0039] Furthermore, the optimal clustering result includes a membership matrix and a cluster center matrix. According to the maximum membership principle, each pixel belongs to the class with the largest corresponding membership.

[0040] A second aspect of the present invention provides a multimodal image clustering segmentation system based on outsourced computing.

[0041] A multimodal image clustering and segmentation system based on outsourced computing, comprising:

[0042] The preprocessing module is configured as follows: a local computing device acquires a plurality of images of different modalities to be segmented, and preprocesses them into a multimodal image pixel matrix;

[0043] The clustering module is configured as follows: based on the multimodal image pixel matrix, the local computing device uses an improved clustering algorithm to iteratively calculate the classification to which the pixel points in the image belong, so as to obtain the optimal clustering result;

[0044] The segmentation module is configured to: according to the category to which each pixel point in the clustering result belongs, the local computing device segments the region corresponding to the category from the multiple images of different modalities to obtain the final result of multimodal image segmentation;

[0045] Among them, the improved clustering algorithm is to split the distance calculation method between pixel points and cluster centers into two steps: matrix multiplication operation and distance calculation based on the result of matrix multiplication operation. The matrix multiplication operation is outsourced to cloud computing devices for calculation using a safe outsourcing method based on matrix blinding.

[0046] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a multimodal image clustering segmentation method based on outsourced computing as described in the first aspect of the present invention.

[0047] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a multimodal image clustering segmentation method based on outsourced computing as described in the first aspect of the present invention are implemented.

[0048] One or more of the above technical solutions have the following beneficial effects:

[0049] (1) The present invention proposes an outsourcing method for splitting the Euclidean distance calculation process, which splits the distance calculation method between pixel points and cluster centers into two steps: matrix multiplication operation and distance calculation based on the result of matrix multiplication operation. The Euclidean distance between data points of two different data sets can be efficiently calculated. The method reduces the computational burden of calculating the similarity between data points in the local clustering process, and can be widely used in clustering algorithms based on distance calculation and their optimization algorithms, meeting different practical application requirements, and realizing efficient and high-precision multimodal image segmentation.

[0050] (2) The present invention proposes a matrix blinding method based on a sparse key matrix, in which the construction of the sparse key matrix can be completed quickly locally, a small-scale random local matrix and its inverse matrix are generated, the small-scale matrix is ​​embedded in the large-scale key matrix, and a blinded key is dynamically generated to ensure the security of the outsourced calculation of the FCM-PSO clustering algorithm; this method does not require pre-storage of keys or key generation elements, adapts to the characteristics of multi-modal imaging devices that store a large number of images and have limited remaining space, and reduces the local storage burden.

[0051] (3) The present invention proposes a secure outsourcing method based on matrix blinding, which outsources matrix multiplication operations to cloud computing devices through encryption keys, and returns the blinded calculation results to local computing devices for subsequent distance calculations by FCM-PSO clustering algorithms. This method supports resource-limited multimodal imaging devices to use local computing resources to quickly complete the multimodal image segmentation process and synchronously output multimodal images and image segmentation results.

[0052] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0054] Figure 1 This is a flow chart of the method of the first embodiment. DETAILED DESCRIPTION

[0055] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0056] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0057] Embodiment 1

[0058] In one embodiment of the present disclosure, a multimodal image clustering segmentation method based on outsourced computing is provided, comprising the following steps:

[0059] Step S1: The local computing device obtains a plurality of images of different modalities to be segmented, and pre-processes them into a multi-modal image pixel matrix;

[0060] Furthermore, the preprocessing is to perform grayscale conversion and numerical normalization on the image, and form a multi-modal image pixel matrix from the grayscale values ​​of the same pixel point in different modal images.

[0061] Step S2: Based on the multimodal image pixel matrix, the local computing device uses an improved clustering algorithm to iteratively calculate the classification to which the pixel points in the image belong, and obtain the optimal clustering result;

[0062] Among them, the improved clustering algorithm is to split the distance calculation method between pixel points and cluster centers into two steps: matrix multiplication operation and distance calculation based on the result of matrix multiplication operation. The matrix multiplication operation is outsourced to cloud computing devices for calculation using a safe outsourcing method based on matrix blinding.

[0063] Furthermore, the improved clustering algorithm adopts the clustering algorithm FCM-PSO based on fuzzy C-means and particle swarm optimization, and the specific steps are as follows:

[0064] (1) Given the number of clusters, initialize the cluster center matrix, assign values ​​to each particle, and randomly generate the initial velocity of the particle;

[0065] (2) Calculate the membership degree of each particle, update all cluster centers, calculate the fitness value of each particle, and update the individual extreme value;

[0066] (3) Find the global extreme value and the position of the global extreme value based on the individual extreme values ​​of each particle;

[0067] (4) Update the particle speed according to the speed formula of the particle swarm optimization algorithm and limit it to the maximum speed;

[0068] (5) Update the position of the particle according to the position formula of the particle swarm optimization algorithm;

[0069] (6) If the termination condition is not met, return to step (2) to continue iterative calculation; if the termination condition is met, output the position of the optimal particle as the optimal clustering result;

[0070] The calculation of membership is based on a matrix blinding safety outsourcing method, which calculates the distance between the pixel point and the cluster center, thereby updating the membership matrix between each pixel point and each cluster center.

[0071] Furthermore, the calculating the distance between the pixel point and the cluster center includes:

[0072] The outsourcing method of splitting the Euclidean distance calculation process is adopted to split the distance calculation method between the pixel point and the cluster center into two steps: matrix multiplication operation and distance calculation based on the result of matrix multiplication operation.

[0073] A matrix blinding method based on a sparse key matrix is ​​used to generate encryption keys and decryption keys for securely transmitting data between a local computing device and a cloud computing device;

[0074] A secure outsourcing method based on matrix blinding is adopted to outsource matrix multiplication operations to cloud computing devices through encryption keys, and the blinded calculation results are returned to the local computing device for subsequent distance calculations.

[0075] Furthermore, the distance between the pixel point and the cluster center is calculated as follows:

[0076] Dynamically generate encryption keys and decryption keys based on the generated key matrix and its inverse matrix;

[0077] The local computing device blinds the multimodal image pixel matrix and the cluster center matrix using the encryption key and sends them to the cloud computing device;

[0078] Based on the blinded multimodal image pixel matrix and cluster center matrix, the cloud computing device calculates the matrix multiplication operation and returns the blinded calculation result to the local computing device;

[0079] The local computing device verifies the result returned by the cloud computing device;

[0080] The local computing device uses the decryption key to perform a deblinding operation on the returned result to obtain a matrix multiplication result;

[0081] The local computing device calculates the distance between the pixel and the cluster center based on the multimodal image pixel matrix, the cluster center matrix and the matrix multiplication result.

[0082] Furthermore, the dynamic generation of encryption keys and decryption keys is specifically as follows:

[0083] Randomly generate a preset number of integers to construct a local random matrix and its inverse matrix;

[0084] Based on the randomly selected coordinates and step sizes, the local random matrix and its inverse matrix are embedded into the key matrix and its inverse matrix respectively;

[0085] According to the step size, the positions of the elements in the key matrix and its inverse matrix are changed to obtain the final key matrix and its inverse matrix;

[0086] Based on the final key matrix and its inverse matrix, encryption key and decryption key are generated.

[0087] Furthermore, the optimal clustering result includes a membership matrix and a cluster center matrix. According to the maximum membership principle, each pixel belongs to the class with the largest corresponding membership.

[0088] Step S3: According to the classification to which each pixel point in the clustering result belongs, the local computing device segments the region corresponding to the classification from the multiple images of different modalities to obtain the final result of multimodal image segmentation;

[0089] The following is a detailed description of the implementation process of a multimodal image clustering segmentation method based on outsourced computing in this embodiment.

[0090] In view of the problem that the existing multimodal medical image segmentation technology is difficult to be efficiently applied in medical scenarios, this embodiment provides a multimodal image clustering segmentation method based on outsourced computing, and proposes a method for outsourcing the Euclidean distance calculation process to efficiently execute the distance-based clustering algorithm and its improved algorithm, and reliably complete the multimodal medical image segmentation task locally. The method involves two entities: a local computing device and a cloud computing device. It does not require the use of third-party facilities such as a key management center. By performing lightweight encryption and decryption operations on the local computing device and performing complex matrix multiplication encrypted operations on the cloud computing device, efficient segmentation of multimodal medical images is achieved. The characteristics of the two entities are:

[0091] Local computing device C: refers to multimodal imaging devices with basic computing capabilities such as PET-CT scanning equipment and magnetic resonance imaging equipment in medical institutions. These devices have the need for multimodal medical image segmentation, but have limited computing resources and weak computing capabilities.

[0092] Cloud computing devices S: refers to cloud servers. These devices have abundant computing resources and can efficiently complete computing tasks, but they are in an untrusted network environment and may have malicious behaviors.

[0093] Based on the above two entities, a specific implementation example is provided, such as Figure 1 As shown, the following steps are included:

[0094] Step 1: Image preprocessing

[0095] The local computing device C reads multiple images of different modalities to be segmented, performs grayscale conversion and numerical normalization preprocessing on the image data, and obtains a multimodal image pixel matrix Let Y=(Y1,Y2,…,Y n ), Y i That is, the grayscale value of the i-th pixel in the d images, where n is the number of pixels and d is the number of images.

[0096] Step 2: Algorithm initialization

[0097] Step 2-1: Parameter setting

[0098] The local computing device C sets the basic parameters of the FCM-PSO algorithm, including the number of clusters c, the fuzzy constant u, the particle population size p, the inertia coefficient ω, the learning coefficients c1 and c2, and the maximum number of iterations t.

[0099] Step 2-2: Initialize particle population

[0100] The local computing device C randomly initializes the position matrix of the particle population and the velocity matrix of the particle population Each particle represents a set of candidate cluster centers in the clustering problem, and the position of the i-th particle is X i =(X i1 ,X i2 ,…,X ik ,…,X ic ), X ik That is, the cluster center of the kth class in the ith particle, c is the number of clusters; the velocity of the ith particle is V i =(V i1 ,V i2 ,…,V ik ,…,V ic ), V ik That is, the velocity of the kth cluster center in the ith particle.

[0101] Step 3: Euclidean distance calculation

[0102] Step 3-1: Key generation

[0103] The local computing device C randomly generates a key SK = (P, P -1 ,Q,Q -1 ,S,S -1 ), where (P -1 ,Q,S,S -1 ) is the encryption key, (P,Q -1 ) is the decryption key. and its inverse matrix For example, the generation method is as follows:

[0104] (1) The local computing device C randomly generates four integers a, b, c, d, satisfying e = ad - cd ≠ 0, and constructs a local matrix At the same time, the local computing device C can obtain the inverse matrix of the local matrix A Among them, a′=d / e, b′=-b / e, c′=-c / e, d′=a / e, and satisfy AA -1 =E, the local matrix A and its inverse matrix can be expressed as and

[0105] (2) The local computing device C randomly selects a coordinate i and a random step length len to obtain a coordinate k = i + len. If the coordinate i or the coordinate k has been selected, it is reselected until the coordinate i or the coordinate k is selected for the first time.

[0106] Embed the matrix A into the key matrix P so that P ii =a,P ik =b,P ki =c,P kk=d; Similarly, the matrix A -1 Embedded into the key matrix P -1 In, making The embedded key matrix P and its inverse matrix can be expressed as

[0107] (3) The local computing device C repeats the above process n / 2 times, that is, embedding n / 2 small-scale local matrices into one large-scale key matrix, so that the initially generated key matrix P and P -1 Each row and column of has only two elements. The key matrix P and its inverse matrix can be expressed as

[0108] (4) The local computing device C transforms the position of the element according to the step length len. If the matrix A i With A k Embed the key matrix P with the same step size len, then exchange the elements of their corresponding rows in the key matrix P, and exchange their -1 The elements of the corresponding columns in the exchanged key matrix P and its inverse matrix can be expressed as and

[0109] (5) The local computing device C can obtain the key matrices P and P through the above steps. -1 Using a similar method, by changing the number of repetitions in step 3), the local computing device C can obtain the key matrix and

[0110] Step 3-2: Data blinding

[0111] The local computing device C obtains the cluster center matrix based on the particle population position matrix X Using the encryption key (P -1 ,Q,S,S -1 ) Blind pixel matrix Y and center matrix Z to obtain Y′ and Z′, where Y′=P -1 YS, Z′=S -1 Z T Q, and send Y′ and Z′ to the cloud computing device S.

[0112] Step 3-3, matrix multiplication calculation

[0113] The cloud computing device S performs matrix multiplication according to the blinded matrices Y′ and Z′ according to the algorithm steps, and calculates the blinded result matrix R′ YZ =Y′Z′, and the blinded result matrix R′ YZ Return to the local computing device C.

[0114] Step 3-4: Result verification

[0115] The local computing device C generates a random verification vector r and uses r to verify the result R′ returned by the cloud. YZ to verify.

[0116] The local computing device C calculates R based on the blinded pixel matrix Y′ and the blinded center matrix Z′ v = rY′Z′, and then according to the blinded result matrix R′ YZ Calculate R′ v = rR′ YZ If R v =R′ v , then R′ YZ If the verification is successful, the local computing device C receives the result; otherwise, the local computing device C discards the result.

[0117] Step 3-5: Result recovery

[0118] The local computing device C uses the decryption key (P, Q -1 ) returns the result R′ YZ Perform deblinding and calculate R YZ =PR′ YZ Q -1 =PY′Z′Q -1 =PP -1 YSS -1 Z T QQ -1 =YZ T , and finally get the result matrix R of matrix multiplication YZ .

[0119] Step 3-6: Euclidean distance calculation

[0120] The local computing device C calculates the square sum of the grayscale values ​​of the image pixels and the square sum of the coordinate values ​​of the cluster centers according to the pixel matrix Y and the center matrix Z, and obtains the result R. Y and R Z For Y=(Y1,Y2,…,Y n ), Y i Represents the row vector consisting of the i-th row elements of matrix Y, and calculates For Z=(Z1,Z2,…,Z m ), Z i Represents the row vector consisting of the elements of the i-th row of the matrix Z, and calculates The local computing device C is based on R Y , R Z and R YZ, calculate the Euclidean distance between each pixel and each cluster center, and obtain the Euclidean distance matrix D, where Represents the Euclidean distance between the i-th point and the j-th cluster center; thus, the local computing device C obtains the similarity between each pixel point and each category.

[0121] Step 4: Membership calculation

[0122] The local computing device C calculates the membership matrix μ between each pixel and each category according to the Euclidean distance matrix D, where: Represents the possibility that the ith point belongs to the jth class, that is, the membership of the ith point to the jth class.

[0123] Step 5: Fitness calculation

[0124] The local computing device C calculates the fitness function f of each particle according to the membership matrix μ. Represents the loss caused by the current clustering result. The smaller the value of the fitness function f, the better the clustering effect.

[0125] Step 6: Cluster update

[0126] First, the local computing device C updates the personal best position pbest of each particle and the population best position gbest of the particle population according to the value of the fitness function f; wherein the personal best position pbest of each particle is a set of cluster center positions corresponding to a smaller value of the fitness function f, and the population best position gbest of the particle population is the position of the particle corresponding to the smallest value of the fitness function f.

[0127] Then, the local computing device C updates the particle population position matrix X and the particle population velocity matrix V according to the individual best position pbest of each particle and the population best position gbest of the particle population; where V t+1 =ωV t +c1(pbest t -X t )+c2(gbest t -X t ), represents the velocity matrix of the particle population at the next iteration; X t+1 =X t +V t+1 , represents the position matrix of the particle population at the next iteration.

[0128] Step 7: Iterative execution

[0129] The local computing device C and the cloud computing device S repeatedly execute steps (3) to (6) until the number of iterations reaches a threshold value t; wherein, since the pixel matrix Y does not change during the iterative execution, only one blinding operation is required to blind the matrix Y to Y′.

[0130] Step 8: Output the results

[0131] The local computing device C selects the membership matrix μ corresponding to the optimal clustering result best And the cluster center matrix Z best , that is, the membership matrix μ and cluster center matrix Z with the minimum fitness function f are obtained. According to the maximum membership principle, each pixel in the image belongs to the class with the largest corresponding membership; thus, the area corresponding to the specific classification can be segmented from multiple images of different modalities to obtain the final result of multimodal image segmentation.

[0132] Embodiment 2

[0133] In one embodiment of the present disclosure, a multimodal image clustering and segmentation system based on outsourced computing is provided, comprising:

[0134] The preprocessing module is configured as follows: a local computing device acquires a plurality of images of different modalities to be segmented, and preprocesses them into a multimodal image pixel matrix;

[0135] The clustering module is configured as follows: based on the multimodal image pixel matrix, the local computing device uses an improved clustering algorithm to iteratively calculate the classification to which the pixel points in the image belong, so as to obtain the optimal clustering result;

[0136] The segmentation module is configured to: according to the category to which each pixel point in the clustering result belongs, the local computing device segments the region corresponding to the category from the multiple images of different modalities to obtain the final result of multimodal image segmentation;

[0137] Among them, the improved clustering algorithm is to split the distance calculation method between pixel points and cluster centers into two steps: matrix multiplication operation and distance calculation based on the result of matrix multiplication operation. The matrix multiplication operation is outsourced to cloud computing devices for calculation using a safe outsourcing method based on matrix blinding.

[0138] Embodiment 3

[0139] The purpose of this embodiment is to provide a computer-readable storage medium.

[0140] A computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in a multimodal image clustering and segmentation method based on outsourced computing as described in the first embodiment of the present disclosure.

[0141] Embodiment 4

[0142] The purpose of this embodiment is to provide an electronic device.

[0143] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in a multimodal image clustering segmentation method based on outsourced computing as described in Embodiment 1 of the present disclosure are implemented.

[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multimodal image clustering segmentation method based on outsourced computing, characterized in that: include: The local computing device obtains a plurality of images of different modalities to be segmented, and pre-processes them into a multi-modal image pixel matrix; Based on the multimodal image pixel matrix, the local computing device uses an improved clustering algorithm to iteratively calculate the classification to which the pixels in the image belong, and obtains the optimal clustering result; According to the classification to which each pixel point in the clustering result belongs, the local computing device segments the corresponding regions from multiple images of different modalities to obtain the final result of multimodal image segmentation; The improved clustering algorithm is to split the distance calculation method between the pixel point and the cluster center into two steps: matrix multiplication operation and distance calculation based on the matrix multiplication operation result, and adopt a safe outsourcing method based on matrix blinding to outsource the matrix multiplication operation to a cloud computing device for calculation; The improved clustering algorithm adopts the FCM-PSO clustering algorithm based on fuzzy C-means and particle swarm optimization. The specific steps are as follows: (1) Given the number of clusters, initialize the cluster center matrix, assign values ​​to each particle, and randomly generate the initial velocity of the particle; (2) Calculate the membership degree for each particle, update all cluster centers, calculate the fitness value of each particle, and update the individual extreme value; (3) Find the global extreme value and its position based on the individual extreme values ​​of each particle; (4) Update the particle speed according to the speed formula of the particle swarm optimization algorithm and limit it to the maximum speed; (5) Update the position of the particle according to the position formula of the particle swarm optimization algorithm; (6) If the termination condition is not met, return to step (2) to continue iterative calculation; if the termination condition is met, output the position of the optimal particle as the optimal clustering result; The calculation of membership is based on a matrix blinding safety outsourcing method to calculate the distance between the pixel and the cluster center, thereby updating the membership matrix between each pixel and each cluster center; The calculating the distance between the pixel point and the cluster center includes: The outsourcing method of splitting the Euclidean distance calculation process is adopted to split the distance calculation method between the pixel point and the cluster center into two steps: matrix multiplication operation and distance calculation based on the result of matrix multiplication operation. A matrix blinding method based on a sparse key matrix is ​​used to generate encryption keys and decryption keys for securely transmitting data between a local computing device and a cloud computing device; A secure outsourcing method based on matrix blinding is adopted to outsource matrix multiplication operations to cloud computing devices through encryption keys, and the blinded calculation results are returned to the local computing device for subsequent distance calculations.

2. The multimodal image clustering segmentation method based on outsourced computing according to claim 1, characterized in that: The preprocessing is to perform grayscale conversion and numerical normalization on the image, and form a multi-modal image pixel matrix from the grayscale values ​​of the same pixel point in different modal images.

3. The multimodal image clustering segmentation method based on outsourced computing according to claim 1, characterized in that: The distance between the pixel point and the cluster center is calculated as follows: Dynamically generate encryption keys and decryption keys based on the generated key matrix and its inverse matrix; The local computing device blinds the multimodal image pixel matrix and the cluster center matrix using the encryption key and sends them to the cloud computing device; Based on the blinded multimodal image pixel matrix and cluster center matrix, the cloud computing device calculates the matrix multiplication operation and returns the blinded calculation result to the local computing device; The local computing device verifies the result returned by the cloud computing device; The local computing device uses the decryption key to perform a deblinding operation on the returned result to obtain a matrix multiplication result; The local computing device calculates the distance between the pixel and the cluster center based on the multimodal image pixel matrix, the cluster center matrix and the matrix multiplication result.

4. The multimodal image clustering segmentation method based on outsourced computing according to claim 3, characterized in that: The dynamic generation of encryption keys and decryption keys is specifically as follows: Randomly generate a preset number of integers to construct a local random matrix and its inverse matrix; Based on the randomly selected coordinates and step sizes, the local random matrix and its inverse matrix are embedded into the key matrix and its inverse matrix respectively; According to the step size, the positions of the elements in the key matrix and its inverse matrix are changed to obtain the final key matrix and its inverse matrix; Based on the final key matrix and its inverse matrix, encryption key and decryption key are generated.

5. The multimodal image clustering segmentation method based on outsourced computing according to claim 1, characterized in that: The optimal clustering result includes a membership matrix and a cluster center matrix. According to the maximum membership principle, each pixel belongs to the class with the largest corresponding membership.

6. A multimodal image clustering and segmentation system based on outsourced computing, characterized in that: include: The preprocessing module is configured as follows: a local computing device acquires a plurality of images of different modalities to be segmented, and preprocesses them into a multimodal image pixel matrix; The clustering module is configured as follows: based on the multimodal image pixel matrix, the local computing device uses an improved clustering algorithm to iteratively calculate the classification to which the pixel points in the image belong, so as to obtain the optimal clustering result; The segmentation module is configured to: according to the category to which each pixel point in the clustering result belongs, the local computing device segments the region corresponding to the category from the multiple images of different modalities to obtain the final result of multimodal image segmentation; The improved clustering algorithm is to split the distance calculation method between the pixel point and the cluster center into two steps: matrix multiplication operation and distance calculation based on the matrix multiplication operation result, and adopt a safe outsourcing method based on matrix blinding to outsource the matrix multiplication operation to a cloud computing device for calculation; The improved clustering algorithm adopts the FCM-PSO clustering algorithm based on fuzzy C-means and particle swarm optimization. The specific steps are as follows: (1) Given the number of clusters, initialize the cluster center matrix, assign values ​​to each particle, and randomly generate the initial velocity of the particle; (2) Calculate the membership degree for each particle, update all cluster centers, calculate the fitness value of each particle, and update the individual extreme value; (3) Find the global extreme value and its position based on the individual extreme values ​​of each particle; (4) Update the particle speed according to the speed formula of the particle swarm optimization algorithm and limit it to the maximum speed; (5) Update the position of the particle according to the position formula of the particle swarm optimization algorithm; (6) If the termination condition is not met, return to step (2) to continue iterative calculation; if the termination condition is met, output the position of the optimal particle as the optimal clustering result; The calculation of membership is based on a matrix blinding safety outsourcing method to calculate the distance between the pixel and the cluster center, thereby updating the membership matrix between each pixel and each cluster center; The calculating the distance between the pixel point and the cluster center includes: The outsourcing method of splitting the Euclidean distance calculation process is adopted to split the distance calculation method between the pixel point and the cluster center into two steps: matrix multiplication operation and distance calculation based on the result of matrix multiplication operation. A matrix blinding method based on a sparse key matrix is ​​used to generate encryption keys and decryption keys for securely transmitting data between a local computing device and a cloud computing device; A secure outsourcing method based on matrix blinding is adopted to outsource matrix multiplication operations to cloud computing devices through encryption keys, and the blinded calculation results are returned to the local computing device for subsequent distance calculations.

7. An electronic device, comprising: a memory for non-transitory storage of computer readable instructions; a processor for executing the computer-readable instructions; Wherein, when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 is executed.

8. A storage medium, characterized in that: The computer-readable instructions are non-transitorily stored, wherein when the computer-readable instructions are executed by a computer, the method according to any one of claims 1 to 5 is performed.

Images