Image segmentation method and device based on feature space multi-view analysis

Abstract

The disclosed embodiment provides an image segmentation method and device based on feature space multi-view analysis, which relates to the field of artificial intelligence technology. The image segmentation method based on feature space multi-view analysis includes: obtaining an image target data set; obtaining an image training data set; obtaining an initial neural network model; performing model training processing on the initial neural network model according to the image training data set to obtain a target neural network model; inputting the image target data set into the trained target neural network model; the target neural network model is used for image segmentation of multi-view analysis in feature space; performing image segmentation processing on the image target data set through the target neural network model to obtain a target segmentation result. The technical solution provided by the disclosed embodiment can improve the accuracy of image segmentation.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an image segmentation method and device based on feature space multi-view analysis. Background Art

[0002] Image segmentation algorithms based on deep neural networks can be used to segment medical images. However, the segmentation tasks of current segmentation algorithms are highly complex and are easily affected by factors such as partial volume effects, grayscale inhomogeneity, artifacts, and the proximity of grayscale between different soft tissues, resulting in low accuracy of image segmentation. Summary of the invention

[0003] The main purpose of the embodiments of the present disclosure is to propose an image segmentation method and device based on feature space multi-view analysis, which can improve the accuracy of image segmentation.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present disclosure proposes an image segmentation method based on feature space multi-view analysis, comprising:

[0005] Obtain image target dataset;

[0006] Acquire an image training data set; the image training data set includes image data and segmentation labels corresponding to the image data;

[0007] Acquire an initial neural network model; the initial neural network model includes at least a shared encoder and a target decoder; the target decoder includes at least a first decoder and a second decoder;

[0008] Performing model training processing on the initial neural network model according to the image training data set to obtain a target neural network model;

[0009] Inputting the image target data set into the trained target neural network model; the target neural network model is used for image segmentation of multi-view analysis in feature space;

[0010] The target neural network model is used to perform image segmentation processing on the image target data set to obtain a target segmentation result.

[0011] In some embodiments, performing model training processing on the initial neural network model according to the image training data set to obtain a target neural network model includes:

[0012] Performing feature extraction processing on the image data through the shared encoder to obtain initial image features; the initial image features include instrument position features, contour features, and image saturation features;

[0013] The target decoder performs feature space multi-view analysis on the initial features of the image to obtain the target image prediction result:

[0014] Perform loss calculation according to the target decoder, the target image prediction result, and the segmentation label to obtain a target loss function;

[0015] The model training process is performed on the initial neural network model according to the target loss function to obtain the target neural network model.

[0016] In some embodiments, the performing loss calculation according to the target decoder, the target image prediction result, and the segmentation label to obtain a target loss function includes:

[0017] Obtaining a difference loss function according to the target decoder;

[0018] Obtaining a segmentation loss function according to the target image prediction result and the segmentation label;

[0019] The target loss function is obtained according to the difference loss function and the segmentation loss function.

[0020] In some embodiments, obtaining a difference loss function according to the target decoder includes:

[0021] Acquire a first parameter set of the first decoder, and obtain a first parameter vector according to the first parameter set;

[0022] Acquire a second parameter set of the second decoder, and obtain a second parameter vector according to the second parameter set;

[0023] Perform cosine similarity quantization processing on the first parameter vector and the second parameter vector to obtain the difference loss function.

[0024] In some embodiments, the target image prediction result includes a first image prediction result output by the first decoder and a second image prediction result output by the second decoder; the segmentation loss function includes first segmentation loss information and second segmentation loss information, and obtaining the segmentation loss function according to the target image prediction result and the segmentation label includes:

[0025] Compare the first image prediction result with the segmentation label to obtain the first segmentation loss information;

[0026] The second image prediction result is compared with the segmentation label to obtain the second segmentation loss information.

[0027] In some embodiments, obtaining the target loss function according to the difference loss function and the segmentation loss function includes:

[0028] Obtaining target weight parameters; the target weight parameters include difference weight parameters and segmentation weight parameters;

[0029] The difference loss function and the segmentation loss function are weighted according to the target weight parameter to obtain the target loss function.

[0030] In some embodiments, performing image segmentation processing on the image target dataset by using the target neural network model to obtain a target segmentation result includes:

[0031] Segmenting the image target data set by the first decoder to obtain a first sub-segmentation result;

[0032] Segmenting the image target data set by the second decoder to obtain a second sub-segmentation result;

[0033] Performing probability calculation on the first sub-segmentation result and the second sub-segmentation result to obtain a category probability map;

[0034] A logistic regression calculation is performed according to the category probability map to obtain the target segmentation result.

[0035] To achieve the above-mentioned purpose, the second aspect of the present disclosure proposes an image segmentation device based on feature space multi-view analysis, comprising:

[0036] A target data set acquisition module is used to acquire an image target data set;

[0037] A training data set acquisition module is used to acquire an image training data set; the image training data set includes image data and segmentation labels corresponding to the image data;

[0038] An initial neural network model acquisition module is used to acquire an initial neural network model; the initial neural network model at least includes a shared encoder and a target decoder; the target decoder at least includes a first decoder and a second decoder;

[0039] A model training module, used to perform model training processing on the initial neural network model according to the image training data set to obtain a target neural network model;

[0040] A data input module, used to input the image target data set into the trained target neural network model; the target neural network model is used to perform image segmentation for multi-view analysis in feature space;

[0041] The image segmentation module is used to perform image segmentation processing on the image target data set through the target neural network model to obtain a target segmentation result.

[0042] To achieve the above objective, a third aspect of the present disclosure provides an electronic device, including:

[0043] at least one memory;

[0044] at least one processor;

[0045] at least one program;

[0046] The program is stored in a memory, and the processor executes at least one program to implement the method of the present disclosure as described in the first aspect above.

[0047] To achieve the above objective, a fourth aspect of the present disclosure provides a storage medium, which is a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute:

[0048] As the method according to the first aspect above.

[0049] The image segmentation method and device based on multi-view analysis in feature space proposed in the embodiments of the present disclosure first respectively obtain an image target data set, an image training data set and an initial neural network model, then perform model training processing on the initial neural network model according to the image training data set to obtain a target neural network model, and then input the image target data set into the trained target neural network model, and finally perform image segmentation of multi-view analysis in feature space on the image target data set through the target neural network model to obtain a target segmentation result. The technical solution provided by the embodiments of the present disclosure can improve the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of an image segmentation method based on feature space multi-view analysis provided by an embodiment of the present disclosure.

[0051] Figure 2 yes Figure 1 Flowchart of step S150 in FIG.

[0052] Figure 3 yes Figure 2 Flowchart of step S230 in .

[0053] Figure 4 yes Figure 3 Flowchart of step S310 in .

[0054] Figure 5 yes Figure 3Flowchart of step S320 in .

[0055] Figure 6 yes Figure 1 Flowchart of step S160 in .

[0056] Figure 7 It is a schematic diagram of the principle of the image segmentation method based on feature space multi-view analysis provided by the embodiment of the present disclosure.

[0057] Figure 8 It is a module block diagram of an image segmentation device based on feature space multi-view analysis provided by an embodiment of the present disclosure.

[0058] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present disclosure.

[0059] Figure numerals: target data set acquisition module 810, training data set acquisition module 820, initial neural network model acquisition module 830, model training module 840, data input module 850, image segmentation module 860, processor 901, memory 902, input / output interface 903, communication interface 904, bus 905. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0063] At present, there are many algorithms in the field of image segmentation that can achieve good segmentation effects on natural images, urban street scenes, and medical images. Among them, the segmentation algorithm based on deep neural network is widely used due to its excellent performance.

[0064] However, the complexity of the segmentation task leads to poor results of current image segmentation methods. The effect of the segmentation algorithm is not only affected by factors such as partial volume effect, grayscale inhomogeneity, artifacts, and the proximity of grayscale between different soft tissues, but is also affected by noise, which leads to low image segmentation accuracy and poor segmentation effect.

[0065] Based on this, the embodiments of the present disclosure provide an image segmentation method and device based on multi-perspective analysis in feature space. First, an image target data set, an image training data set and an initial neural network model are respectively obtained. Then, the initial neural network model is trained according to the image training data set to obtain a target neural network model. The image target data set is then input into the trained target neural network model. Finally, the image target data set is subjected to multi-perspective analysis in feature space by the target neural network model to obtain a target segmentation result. The technical solution provided by the embodiments of the present disclosure can improve the accuracy of image segmentation.

[0066] Starting from the principle that "observing things from multiple angles can obtain more accurate conclusions", this application proposes an image segmentation method based on multi-view analysis in feature space, applies a neural network to extract features from images, and uses two classifiers (including decoders) with different parameters to learn the features separately, thereby training a segmentation network that can perform multi-view analysis in feature space, improving the accuracy of image segmentation, and improving the image segmentation effect.

[0067] Images from the same data source can often be described by a set of attributes. Neural networks have the ability to segment images by screening and learning the attributes. Therefore, this application selects different attribute sets for learning to form a comprehensive consideration of the input image, thereby improving the performance of the neural network. Specifically, the features are first extracted through the encoder, and then two decoders with different parameters are used to learn different features, so as to analyze from the perspective of different attribute sets, and finally the image is segmented by combining the capabilities of the two.

[0068] It should be noted that the "feature space multi-perspective" in the image segmentation method based on feature space multi-perspective analysis provided in the embodiment of the present application is not the multi-perspective of ordinary physical space, but the multi-perspective in feature space. Therefore, the image segmentation method based on feature space multi-perspective analysis of the present application is not limited by the dimension of image data and can be applied to the analysis of two-dimensional data, three-dimensional data, time series data and other data.

[0069] The embodiments of the present disclosure provide an image segmentation method and device based on feature space multi-view analysis, which are specifically described through the following embodiments. First, the image segmentation method based on feature space multi-view analysis in the embodiments of the present disclosure is described.

[0070] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0071] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0072] The image segmentation method based on feature space multi-view analysis provided by the embodiment of the present disclosure relates to the field of artificial intelligence technology, and in particular to the field of image segmentation technology. The image segmentation method based on feature space multi-view analysis provided by the embodiment of the present disclosure can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc.; the server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content De livery Network, CDN), and big data and artificial intelligence platforms; the software can be an application that implements an image segmentation method based on feature space multi-view analysis, etc., but is not limited to the above forms.

[0073] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0074] The disclosed embodiment proposes an image segmentation method based on multi-view analysis in feature space, including: obtaining an image target data set; obtaining an image training data set; the image training data set includes image data and segmentation labels corresponding to the image data; obtaining an initial neural network model; the initial neural network model includes at least a shared encoder and a target decoder; the target decoder includes at least a first decoder and a second decoder; performing model training processing on the initial neural network model according to the image training data set to obtain a target neural network model; inputting the image target data set into the trained target neural network model; the target neural network model is used for image segmentation of multi-view analysis in feature space; performing image segmentation processing on the image target data set through the target neural network model to obtain a target segmentation result.

[0075] Figure 1 is an optional flow chart of an image segmentation method based on feature space multi-view analysis provided by an embodiment of the present disclosure, Figure 1 The method may include but is not limited to steps S110 to S160, specifically including:

[0076] S110, obtaining an image target data set;

[0077] S120, obtaining an image training data set;

[0078] S130, obtaining an initial neural network model;

[0079] S140, performing model training processing on the initial neural network model according to the image training data set to obtain a target neural network model;

[0080] S150, inputting the image target data set into the trained target neural network model;

[0081] S160, performing image segmentation processing on the image target data set through the target neural network model to obtain a target segmentation result.

[0082] In step S110 , the image target data set is a data set to be segmented.

[0083] In step S120, the image training data set is a data set used when training the model, including image data and segmentation labels corresponding to the image data, for example, a data set (X, Y), where X is the image data and Y is the segmentation label.

[0084] In step S130, the initial neural network model includes at least a shared encoder and a target decoder; the target decoder includes at least a first decoder and a second decoder, wherein the shared encoder is denoted by E, which is used to extract features in the image data, and the target decoder is denoted by D, which is used to perform multi-view learning on the features in the image data.

[0085] Among them, the sharing of the shared encoder is reflected in the connection with each decoder respectively, and the structure of the shared decoder is not limited to any fully convolutional network model, such as DeepLabV3.

[0086] In a specific embodiment, the structure of each decoder in the target decoder is consistent, but the parameters of each decoder are different, so that different decoders learn different image features, thereby achieving multi-view analysis. For example, the parameters of the first decoder are the first parameter set, and the parameters of the second decoder are the second parameter set.

[0087] In a specific embodiment, the connection between the output of the shared encoder and the input of each decoder in the target decoder is a main connection. In addition, the shared encoder is also connected to each decoder through a branch to improve the performance of the network model.

[0088] In a specific embodiment, each decoder in the target decoder is provided with a fully connected layer, and the fully connected layer is used to output the final segmentation result. Other parts of each decoder in the target decoder include but are not limited to a deconvolution layer.

[0089] It should be noted that at least two decoders in the target decoder are used to learn from different features, thereby realizing multi-perspective analysis in the dimensions of different feature spaces to improve the accuracy of image segmentation. The number of decoders in the target decoder will increase with the increase in the number of perspectives, as long as the purpose of multi-perspective analysis can be achieved.

[0090] In step S140, the training process of the initial neural network model specifically includes: designing a loss function, inputting the image data in the training data set into the model to obtain an output, and comparing the output with the segmentation label, adjusting the parameters of the model through the loss function, and finally obtaining a trained target neural network model.

[0091] In step S150 to step S160, the image target data set is input into the target neural network model to perform image segmentation processing on the image target data set through the target neural network model to obtain the target segmentation result, thereby realizing image segmentation of multi-view analysis in the feature space.

[0092] The disclosed embodiment provides an image segmentation method based on multi-view analysis in feature space. First, an image target data set, an image training data set and an initial neural network model are respectively obtained. Then, the initial neural network model is trained according to the image training data set to obtain a target neural network model. Then, the image target data set is input into the trained target neural network model. Finally, the image target data set is subjected to multi-view analysis in feature space by the target neural network model to obtain a target segmentation result. The technical solution provided by the disclosed embodiment can improve the accuracy of image segmentation.

[0093] In some embodiments, the initial neural network model is subjected to model training processing according to the image training data set to obtain a target neural network model, including: performing feature extraction processing on the image data through a shared encoder to obtain initial image features; the initial image features include instrument position features, contour features, and image saturation features; performing feature space multi-view analysis processing on the initial image features through a target decoder to obtain a target image prediction result: performing loss calculation based on the target decoder, the target image prediction result, and the segmentation label to obtain a target loss function; and performing model training processing on the initial neural network model according to the target loss function to obtain a target neural network model.

[0094] Figure 2 is a flowchart of step S150 in some embodiments, Figure 2 The illustrated step S150 includes but is not limited to steps S210 to S240:

[0095] S210, performing feature extraction processing on the image data through a shared encoder to obtain initial image features;

[0096] S220, performing feature space multi-view analysis processing on the initial features of the image through the target decoder to obtain the target image prediction result:

[0097] S230, performing loss calculation according to the target decoder, the target image prediction result, and the segmentation label to obtain a target loss function;

[0098] S240, performing model training processing on the initial neural network model according to the target loss function to obtain a target neural network model.

[0099] In step S210, the image data is input into the shared encoder, and the attributes in the image are abstractly extracted to obtain the initial image features, wherein the initial image features include instrument position features, contour features, and image saturation features.

[0100] In step S220, the first decoder is denoted as D α , the second decoder is D β, each decoder is responsible for learning the obtained feature vector. Since the first decoder and the second decoder adopt the same structure, the parameters of the two decoders can form two corresponding vectors and output the corresponding target image prediction results, where the target image prediction result is the result of semantic segmentation image prediction.

[0101] In steps S230 to S240, the loss function of the entire target neural network model is the target loss function, which is denoted by L total , the objective loss function is used to train the model.

[0102] In some embodiments, loss calculation is performed based on the target decoder, the target image prediction result, and the segmentation label to obtain a target loss function, including: obtaining a difference loss function based on the target decoder; obtaining a segmentation loss function based on the target image prediction result and the segmentation label; and obtaining a target loss function based on the difference loss function and the segmentation loss function.

[0103] Figure 3 is a flowchart of step S230 in some embodiments, Figure 3 The illustrated step S230 includes but is not limited to steps S310 to S330:

[0104] S310, obtaining a difference loss function according to the target decoder;

[0105] S320, obtaining a segmentation loss function according to the target image prediction result and the segmentation label;

[0106] S330, obtaining a target loss function according to the difference loss function and the segmentation loss function.

[0107] In step S310, in order to encourage different decoders to learn the features of different attribute sets, multi-view analysis is performed in the feature space, and a difference loss function is introduced to constrain the difference of the decoders to ensure that different decoders have different parameters. Specifically, the difference loss function is recorded as L weight .

[0108] In step S320, the segmentation loss function is a loss function of the decoder in the image segmentation process. The segmentation loss function includes first segmentation loss information and second segmentation loss information. Specifically, the first segmentation loss information is recorded as L segα , the second segmentation loss information is recorded as L segβ .

[0109] In step S330, the target loss function is the loss function of the entire target neural network model, and the target loss function is denoted by L total .

[0110] In some embodiments, a difference loss function is obtained according to a target decoder, including: obtaining a first parameter set of a first decoder, and obtaining a first parameter vector according to the first parameter set; obtaining a second parameter set of a second decoder, and obtaining a second parameter vector according to the second parameter set; performing cosine similarity quantization processing on the first parameter vector and the second parameter vector to obtain a difference loss function.

[0111] Figure 4 is a flowchart of step S310 in some embodiments, Figure 4 The illustrated step S310 includes but is not limited to steps S410 to S430:

[0112] S410, acquiring a first parameter set of a first decoder, and obtaining a first parameter vector according to the first parameter set;

[0113] S420, obtaining a second parameter set of a second decoder, and obtaining a second parameter vector according to the second parameter set;

[0114] S430, performing cosine similarity quantization processing on the first parameter vector and the second parameter vector to obtain a difference loss function.

[0115] In step S410, the first parameter set is a parameter of the first decoder, and the first parameter vector is a vector obtained by the first parameter set, which is used to calculate the difference loss function.

[0116] In step S420, the second parameter set is a parameter of the second decoder, and the second parameter vector is a vector obtained by the second parameter set, which is used to calculate the difference loss function.

[0117] In step S430, in order to force the two decoders to learn the features of different attribute sets, multi-view analysis is performed in the feature space, and a difference loss function is introduced to constrain the difference of the decoders to ensure that the two decoders have different parameters. Since the two decoders use the same structure, the parameters of the two decoders can form two corresponding vectors. The difference between the two can be quantified by calculating the cosine similarity of the two vectors as a loss function, where the smaller the cosine similarity, the greater the difference between the decoders, thereby realizing the use of parameters to control the difference between the decoders, improving the flexibility of the model, and improving the efficiency of model training.

[0118] In some embodiments, the target image prediction result includes a first image prediction result output by the first decoder and a second image prediction result output by the second decoder; the segmentation loss function includes first segmentation loss information and second segmentation loss information, and the segmentation loss function is obtained according to the target image prediction result and the segmentation label, including: comparing the first image prediction result with the segmentation label to obtain the first segmentation loss information; comparing the second image prediction result with the segmentation label to obtain the second segmentation loss information.

[0119] Figure 5 is a flowchart of step S320 in some embodiments, Figure 5 The illustrated step S320 includes but is not limited to steps S510 to S520:

[0120] S510, comparing the first image prediction result with the segmentation label to obtain first segmentation loss information;

[0121] S520: Compare the second image prediction result with the segmentation label to obtain second segmentation loss information.

[0122] In steps S510 to S520, the target image prediction result includes a first image prediction result output by the first decoder and a second image prediction result output by the second decoder, wherein the first image prediction result is recorded as The second image prediction result is recorded as

[0123] After obtaining the target image prediction result, the first image prediction result, the second image prediction result and the segmentation label are compared to calculate the segmentation loss function, where the segmentation loss function includes the first segmentation loss information and the second segmentation loss information, thereby guiding the neural network to learn in the correct semantic segmentation direction. Specifically, the first segmentation loss information is recorded as L segα , the second segmentation loss information is recorded as L segα .

[0124] In some embodiments, a target loss function is obtained according to a difference loss function and a segmentation loss function, including: obtaining a target weight parameter; the target weight parameter includes a difference weight parameter and a segmentation weight parameter; and weighting the difference loss function and the segmentation loss function according to the target weight parameter to obtain a target loss function.

[0125] Step S330 also includes: obtaining a target weight parameter, and performing weighted processing on the difference loss function and the segmentation loss function according to the target weight parameter to obtain a target loss function.

[0126] The target weight parameters include difference weight parameters and segmentation weight parameters. The difference weight parameter is denoted as λ1 and the segmentation weight parameter is denoted as λ2. The weighting process is L total =λ1L weight +λ2(L segα +L segβ ), the overall optimization of the model is achieved by minimizing the overall loss function during the training process.

[0127] In some embodiments, image segmentation processing is performed on an image target data set through a target neural network model to obtain a target segmentation result, including: segmenting the image target data set through a first decoder to obtain a first sub-segmentation result; segmenting the image target data set through a second decoder to obtain a second sub-segmentation result; performing probability calculation on the first sub-segmentation result and the second sub-segmentation result to obtain a category probability map; performing logistic regression calculation based on the category probability map to obtain a target segmentation result.

[0128] Figure 6 is a flowchart of step S160 in some embodiments, Figure 6 The illustrated step S160 includes but is not limited to steps S610 to S640:

[0129] S610, segmenting the image target data set through a first decoder to obtain a first sub-segmentation result;

[0130] S620, segmenting the image target data set through a second decoder to obtain a second sub-segmentation result;

[0131] S630, performing probability calculation on the first sub-segmentation result and the second sub-segmentation result to obtain a category probability map;

[0132] S640, performing logistic regression calculation according to the category probability map to obtain a target segmentation result.

[0133] In steps S610 to S620, the first sub-segmentation result is recorded as The second sub-segmentation result is recorded as It should be noted that the recording symbols of the first image prediction result and the first sub-segmentation result are the same, and the recording symbols of the second image prediction result and the second sub-segmentation result are the same, because the target image prediction result is the output of the decoder during the training process, and the target segmentation result is the output of the decoder when the image target data set is segmented. Both are decoder outputs, but the difference is that they are used in the training and segmentation processes of the model respectively. Therefore, the unification of the recording symbols of the two can be understood as different states of the same data in different applications, and should not be understood as ambiguity or limitation on this application.

[0134] In steps S630 to S640, the first sub-segmentation result and the second sub-segmentation result need to be merged, and the feature learning is integrated to obtain the target segmentation result. The target segmentation result is the final semantic segmentation prediction result. The target segmentation result is recorded as Specifically, the merging process does not directly add the final results of the segmentation, but adds the class probability maps of the segmentation output, and then performs a logistic regression calculation on the class probability map to obtain the target segmentation result. The logistic regression calculation includes a softmax calculation.

[0135] Figure 7 is a schematic diagram of the principle of the image segmentation method based on feature space multi-view analysis provided by the embodiment of the present disclosure, and the following reference is made to Figure 7 , an image segmentation method based on feature space multi-view analysis is described in detail with a specific embodiment. It is worth noting that the following description is only an exemplary description and not a specific limitation of the application.

[0136] First, the image dataset (X, Y) required for training, where X is the image data and Y is the segmentation label. Figure 7 "input X" means input affecting data.

[0137] The image data is input into the shared encoder E, and the attributes in the image data, including device position features, contour features, image saturation features, etc., are abstractly extracted to obtain a set of initial image features. The shared encoder E is also called a feature extractor, and any fully convolutional network can be used. Figure 7 "Shared encoder E" means shared encoder.

[0138] Then define the target decoder, the target decoder includes a first decoder D α , the second decoder is D β , each decoder is responsible for learning the obtained feature vector and outputting the corresponding target image prediction result and Figure 7 Decoder D α "Decoder D β " indicates the second decoder, "Prediction " indicates the first image prediction result, "Prediction " represents the second image prediction result. In addition, Figure 7 The connections between the main path and the branch paths between the shared encoder and each decoder are clearly shown.

[0139] In order to force the two decoders to learn the features of different attribute sets, multi-view analysis is performed in the feature space and the difference loss function L is introduced. weight Difference constraints are imposed on the decoders to ensure that the two decoders have different parameters. Figure 7 Discrepancy loss L weight ” represents the difference loss function.

[0140] After obtaining the prediction results of the target image, we compare them with the segmentation labels and calculate the segmentation loss function, thereby guiding the neural network to learn in the correct semantic segmentation direction. Figure 7 Segmentation loss L segα L segβ ” represents the target image prediction result.

[0141] Finally, the segmentation results are merged, the feature learning is integrated, and the target segmentation result is obtained. Figure 7 Final prediction ” indicates the target segmentation result.

[0142] The disclosed embodiment proposes an image segmentation device based on multi-view analysis in feature space, including: a target data set acquisition module, used to acquire an image target data set; a training data set acquisition module, used to acquire an image training data set; the image training data set includes image data and segmentation labels corresponding to the image data; an initial neural network model acquisition module, used to acquire an initial neural network model; the initial neural network model includes at least a shared encoder and a target decoder; the target decoder includes at least a first decoder and a second decoder; a model training module, used to perform model training processing on the initial neural network model according to the image training data set to obtain a target neural network model; a data input module, used to input the image target data set into the trained target neural network model; the target neural network model is used for image segmentation of multi-view analysis in feature space; an image segmentation module, used to perform image segmentation processing on the image target data set through the target neural network model to obtain a target segmentation result.

[0143] See also Figure 8 , Figure 8An image segmentation device based on feature space multi-view analysis according to an embodiment is illustrated. The image segmentation device based on feature space multi-view analysis includes: a target data set acquisition module 810, a training data set acquisition module 820, an initial neural network model acquisition module 830, a model training module 840, a data input module 850, and an image segmentation module 860, wherein the target data set acquisition module 810 is connected to the training data set acquisition module 820, the training data set acquisition module 820 is connected to the initial neural network model acquisition module 830, the initial neural network model acquisition module 830 is connected to the model training module 840, the model training module 840 is connected to the data input module 850, and the data input module 850 is connected to the image segmentation module 860.

[0144] The specific implementation of the image segmentation device based on feature space multi-view analysis in this embodiment is basically consistent with the specific implementation of the image segmentation method based on feature space multi-view analysis mentioned above, belongs to the same inventive concept, and will not be repeated here.

[0145] The present disclosure also provides an electronic device, including:

[0146] at least one memory;

[0147] at least one processor;

[0148] at least one program;

[0149] The program is stored in the memory, and the processor executes the at least one program to implement the image segmentation method based on feature space multi-view analysis in the present disclosure. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0150] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0151] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present disclosure;

[0152] The memory 902 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 902 can store operating systems and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 902, and the processor 901 calls and executes the image segmentation method based on feature space multi-view analysis in the embodiment of the present disclosure;

[0153] Input / output interface 903, used to implement information input and output;

[0154] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); and

[0155] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0156] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0157] The embodiment of the present disclosure also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned image segmentation method based on feature space multi-view analysis.

[0158] The image segmentation method and device based on multi-view analysis in feature space proposed in the embodiments of the present disclosure first respectively obtain an image target data set, an image training data set and an initial neural network model, then perform model training processing on the initial neural network model according to the image training data set to obtain a target neural network model, and then input the image target data set into the trained target neural network model, and finally perform image segmentation of multi-view analysis in feature space on the image target data set through the target neural network model to obtain a target segmentation result, thereby improving the accuracy of image segmentation through the technical solution provided in the embodiments of the present disclosure.

[0159] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0160] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0161] It can be understood by those skilled in the art that Figure 1-6 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present disclosure, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0162] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0164] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0165] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0166] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0169] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0170] The preferred embodiments of the present disclosure are described above with reference to the accompanying drawings, but the scope of the rights of the present disclosure is not limited thereto. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the present disclosure should be within the scope of the rights of the present disclosure.

Claims

1. An image segmentation method based on feature space multi-view analysis, characterized in that: include: Obtain image target dataset; Acquire an image training data set; the image training data set includes image data and segmentation labels corresponding to the image data; Acquire an initial neural network model; the initial neural network model includes at least a shared encoder and a target decoder; the target decoder includes at least a first decoder and a second decoder; the structure of each decoder in the target decoder remains consistent, and the parameters of each decoder are different; Performing model training processing on the initial neural network model according to the image training data set to obtain a target neural network model; Inputting the image target data set into the trained target neural network model; the target neural network model is used for image segmentation of multi-view analysis in feature space; Performing image segmentation processing on the image target data set through the target neural network model to obtain a target segmentation result; The step of performing model training processing on the initial neural network model according to the image training data set to obtain a target neural network model includes: Performing feature extraction processing on the image data through the shared encoder to obtain initial image features; the initial image features include instrument position features, contour features, and image saturation features; Performing feature space multi-view analysis processing on the initial features of the image by the target decoder to obtain a target image prediction result; Perform loss calculation according to the target decoder, the target image prediction result, and the segmentation label to obtain a target loss function; Performing the model training process on the initial neural network model according to the target loss function to obtain the target neural network model; The loss calculation is performed according to the target decoder, the target image prediction result, and the segmentation label to obtain a target loss function, including: Obtaining a difference loss function according to the target decoder; Obtaining a segmentation loss function according to the target image prediction result and the segmentation label; Obtaining the target loss function according to the difference loss function and the segmentation loss function; Wherein, obtaining a difference loss function according to the target decoder includes: Acquire a first parameter set of the first decoder, and obtain a first parameter vector according to the first parameter set; Acquire a second parameter set of the second decoder, and obtain a second parameter vector according to the second parameter set; Performing cosine similarity quantization processing on the first parameter vector and the second parameter vector to obtain the difference loss function; The step of performing image segmentation processing on the image target data set by using the target neural network model to obtain a target segmentation result includes: Segmenting the image target data set by the first decoder to obtain a first sub-segmentation result; Segmenting the image target data set by the second decoder to obtain a second sub-segmentation result; Performing probability calculation on the first sub-segmentation result and the second sub-segmentation result to obtain a category probability map; A logistic regression calculation is performed according to the category probability map to obtain the target segmentation result.

2. The method according to claim 1, characterized in that The target image prediction result includes a first image prediction result output by the first decoder and a second image prediction result output by the second decoder; the segmentation loss function includes first segmentation loss information and second segmentation loss information, and the segmentation loss function is obtained according to the target image prediction result and the segmentation label, including: Compare the first image prediction result with the segmentation label to obtain the first segmentation loss information; The second image prediction result is compared with the segmentation label to obtain the second segmentation loss information.

3. The method according to claim 1, characterized in that The obtaining the target loss function according to the difference loss function and the segmentation loss function includes: Obtaining target weight parameters; the target weight parameters include difference weight parameters and segmentation weight parameters; The difference loss function and the segmentation loss function are weighted according to the target weight parameter to obtain the target loss function.

4. An image segmentation device based on feature space multi-view analysis, characterized in that: include: A target data set acquisition module is used to acquire an image target data set; A training data set acquisition module is used to acquire an image training data set; the image training data set includes image data and segmentation labels corresponding to the image data; An initial neural network model acquisition module is used to acquire an initial neural network model; the initial neural network model at least includes a shared encoder and a target decoder; the target decoder at least includes a first decoder and a second decoder; The structure of each decoder in the target decoder remains consistent, and the parameters of each decoder are different; A model training module, used to perform model training processing on the initial neural network model according to the image training data set to obtain a target neural network model; A data input module, used to input the image target data set into the trained target neural network model; the target neural network model is used to perform image segmentation for multi-view analysis in feature space; An image segmentation module is used to perform image segmentation processing on the image target data set through the target neural network model to obtain a target segmentation result; The model training module is used to perform model training processing on the initial neural network model according to the image training data set to obtain a target neural network model, including: Performing feature extraction processing on the image data through the shared encoder to obtain initial image features; the initial image features include instrument position features, contour features, and image saturation features; Performing feature space multi-view analysis processing on the initial features of the image by the target decoder to obtain a target image prediction result; Perform loss calculation according to the target decoder, the target image prediction result, and the segmentation label to obtain a target loss function; Performing the model training process on the initial neural network model according to the target loss function to obtain the target neural network model; The loss calculation is performed according to the target decoder, the target image prediction result, and the segmentation label to obtain a target loss function, including: Obtaining a difference loss function according to the target decoder; Obtaining a segmentation loss function according to the target image prediction result and the segmentation label; Obtaining the target loss function according to the difference loss function and the segmentation loss function; Wherein, obtaining a difference loss function according to the target decoder includes: Acquire a first parameter set of the first decoder, and obtain a first parameter vector according to the first parameter set; Acquire a second parameter set of the second decoder, and obtain a second parameter vector according to the second parameter set; Performing cosine similarity quantization processing on the first parameter vector and the second parameter vector to obtain the difference loss function; The image segmentation module is used to perform image segmentation processing on the image target data set through the target neural network model to obtain a target segmentation result, including: Segmenting the image target data set by the first decoder to obtain a first sub-segmentation result; Segmenting the image target data set by the second decoder to obtain a second sub-segmentation result; Performing probability calculation on the first sub-segmentation result and the second sub-segmentation result to obtain a category probability map; A logistic regression calculation is performed according to the category probability map to obtain the target segmentation result.

5. An electronic device, characterized in that: include: at least one memory; at least one processor; at least one program; The program is stored in the memory, and the processor executes at least one of the programs to implement: The method according to any one of claims 1 to 3.

6. A storage medium, the storage medium being a computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute: The method according to any one of claims 1 to 3.

Images