Image segmentation method and apparatus therefor

Abstract

The application discloses an image segmentation method and device, and belongs to the field of image processing. The method comprises the following steps: inputting a first image into a base coefficient prediction model to obtain M first base coefficients, wherein the M first base coefficients are in one-to-one correspondence with M first segmentation bases corresponding to the base coefficient prediction model, the base coefficient prediction model is obtained based on a plurality of image segmentation samples, M is a positive integer; performing weighted calculation on a first base feature vector of the corresponding first segmentation base by using the first base coefficient to obtain M target base feature vectors, wherein the first base feature vectors of the M first segmentation bases are extracted from the plurality of image segmentation samples; and segmenting an image subject in the first image according to the M target base feature vectors.

Description

Technical Field

[0001] This application belongs to the field of image processing, and specifically relates to an image segmentation method and apparatus. Background Technology

[0002] With the development of electronic technology, electronic devices are becoming increasingly versatile. For example, electronic devices can perform image segmentation and prediction on images.

[0003] Specifically, in the prior art, electronic devices can first predict the segmentation value corresponding to each pixel based on the pixel information of each pixel in the image, and determine the main body of the image based on the segmentation value corresponding to all pixels in the image.

[0004] However, following the above method, since it is necessary to predict the segmentation value of each pixel in the image separately, the calculation of the segmentation result of the image subject in the image may be large, which may result in a long time to predict the segmentation result of the image subject in the image. Summary of the Invention

[0005] The purpose of this application is to provide an image segmentation method and apparatus that can reduce the time spent segmenting the main body of an image and improve the segmentation speed.

[0006] In a first aspect, embodiments of this application provide an image segmentation method, which includes: inputting a first image into a basis coefficient prediction model to obtain M first basis coefficients, wherein the M first basis coefficients correspond one-to-one with M first segmentation basis corresponding to the basis coefficient prediction model, the basis coefficient prediction model is trained based on multiple image segmentation samples, and M is a positive integer; using the first basis coefficients to perform weighted calculation on the first basis feature vectors of the corresponding first segmentation basis to obtain M target basis feature vectors, wherein the first basis feature vectors of the M first segmentation basis are extracted from multiple image segmentation samples; and segmenting the main image in the first image according to the M target basis feature vectors.

[0007] Secondly, embodiments of this application provide an image segmentation apparatus, which includes: a processing module; the processing module is configured to input a first image into a basis coefficient prediction model to obtain M first basis coefficients, wherein the M first basis coefficients correspond one-to-one with M first segmentation basis corresponding to the basis coefficient prediction model, the basis coefficient prediction model is trained based on multiple image segmentation samples, and M is a positive integer; the processing module is further configured to perform weighted calculation on the first basis feature vectors of the corresponding first segmentation basis using the first basis coefficients to obtain M target basis feature vectors, wherein the first basis feature vectors of the M first segmentation basis are extracted from multiple image segmentation samples; the processing module is further configured to segment the image subject in the first image according to the M target basis feature vectors.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In this embodiment, a first image is input into a basis coefficient prediction model to obtain M first basis coefficients. Each of the M first basis coefficients corresponds one-to-one with one of the M first segmentation basis vectors corresponding to the basis coefficient prediction model. The basis coefficient prediction model is trained based on multiple image segmentation samples, and M is a positive integer. The first basis coefficients are used to weight the first basis feature vectors of their corresponding first segmentation basis vectors to obtain M target basis feature vectors. These first basis feature vectors are extracted from the multiple image segmentation samples. Based on these M target basis feature vectors, the main image in the first image is segmented. This approach, since the basis coefficient prediction model is trained based on multiple image segmentation samples, and the first basis feature vectors of the M first segmentation basis vectors are extracted from multiple image segmentation samples, allows the use of the M first basis coefficients predicted by the basis coefficient prediction model to weight the M first basis feature vectors of the M first segmentation basis vectors, resulting in M ​​target basis feature vectors that characterize the image to be processed. Therefore, the main image in the image to be processed can be directly segmented based on these M target basis feature vectors. Thus, since the image subject in the image to be processed can be segmented using the basis feature vectors of a finite number of preset segmentation basis points and the basis coefficients predicted by the basis prediction model and corresponding one-to-one with the preset segmentation basis points, the image segmentation method provided in this application embodiment can reduce the computational load of segmenting the image subject in the image, thereby reducing the time consumption of segmenting the image subject in the image and improving the segmentation speed. Attached Figure Description

[0013] Figure 1 This is one of the method flowcharts of an image segmentation method provided in the embodiments of this application;

[0014] Figure 2A This is an example schematic diagram of an image to be segmented provided in an embodiment of this application;

[0015] Figure 2B This is an example schematic diagram of the image subject segmentation result corresponding to an image to be segmented, provided in an embodiment of this application;

[0016] Figure 3 This is a second flowchart of an image segmentation method provided in an embodiment of this application;

[0017] Figure 4 This is the third flowchart of an image segmentation method provided in this application embodiment;

[0018] Figure 5This is the fourth flowchart of an image segmentation method provided in the embodiments of this application;

[0019] Figure 6 This is a schematic diagram illustrating an example of a process for cropping an image to be segmented, as provided in an embodiment of this application.

[0020] Figure 7 This is an example diagram illustrating how scale restoration yields the image subject segmentation result corresponding to the image to be segmented, as provided in an embodiment of this application.

[0021] Figure 8 This is a schematic diagram of the structure of an image segmentation device provided in an embodiment of this application;

[0022] Figure 9 This is one of the hardware structure diagrams of an electronic device provided in the embodiments of this application;

[0023] Figure 10 This is a second schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0026] The image segmentation method and apparatus provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0027] In related technologies, when an electronic device predicts the segmentation result of the main subject in an image, such as a single-lens image, it can first predict the segmentation value corresponding to each pixel based on the pixel information (such as color information, depth information, etc.) of each pixel in the image. After predicting the segmentation values ​​corresponding to all pixels in the image, the electronic device can determine the segmentation result of the main subject in the image based on all the predicted segmentation values. However, since it is necessary to predict the segmentation value of each pixel in the image separately, the computational load for predicting the segmentation result of the main subject in the image may be large, resulting in a long time consumption for predicting the segmentation result of the main subject in the image.

[0028] In the image segmentation method and apparatus provided in this application embodiment, M first basis coefficients of the image to be processed can be predicted by a basis coefficient prediction model trained from multiple image segmentation samples. Then, the basis feature vectors of the M first segmentation basis extracted from multiple image segmentation samples are weighted using these M first basis coefficients to obtain M target basis feature vectors that can characterize the image to be processed. Therefore, the main body of the image to be processed can be directly segmented based on the M target basis feature vectors. Thus, since the main body of the image to be processed can be segmented using a finite number of preset basis feature vectors and basis coefficients predicted by the basis prediction model that correspond one-to-one with the preset segmentation basis, the main body of the image to be processed can be segmented. Therefore, compared with the scheme of first predicting the segmentation values ​​of all pixels in the image to be processed and then segmenting the main body of the image based on the segmentation values ​​of all pixels, the image segmentation method provided in this application embodiment can reduce the computational load of segmenting the main body of the image, thereby reducing the time consumption of segmenting the main body of the image and improving the segmentation speed.

[0029] The image segmentation method provided in this application can be implemented by an image segmentation device. Exemplarily, the image segmentation device can be an electronic device, or a component within that electronic device, such as an integrated circuit or a chip. The image segmentation method provided in this application will be described executively below using an image segmentation device as an example.

[0030] This application provides an image segmentation method. Figure 1 A flowchart of an image segmentation method provided in an embodiment of this application is shown. This method can be applied to an image segmentation apparatus. Figure 1 As shown, the image segmentation method provided in this application embodiment may include the following steps 101 to 103.

[0031] Step 101: The image segmentation device inputs the first image into the basis coefficient prediction model to obtain M first basis coefficients.

[0032] Step 102: The image segmentation device uses the first basis coefficient to perform weighted calculation on the first basis feature vector of the corresponding first segmentation basis to obtain M target basis feature vectors.

[0033] Among them, the M first basis coefficients correspond one-to-one with the M first segmentation basis corresponding to the basis coefficient prediction model; the basis coefficient prediction model is trained based on multiple image segmentation samples, and M is a positive integer; the first basis feature vector of the M first segmentation basis is extracted from multiple image segmentation samples.

[0034] In some embodiments of this application, the first image may be an image captured by the imaging device, an image from another device, an image downloaded from the Internet, or an image saved by the imaging device. This application does not limit the image source of the first image.

[0035] In some embodiments of this application, the M first basis coefficients can characterize the weight coefficients corresponding to the M first basis features when the first image is represented by the first basis feature vectors of the M first segmentation basis. It can be understood that the larger the first basis coefficients are, the greater the proportion of the first basis feature vector of the first segmentation basis corresponding to the first basis coefficients in the first image.

[0036] It is understandable that the first segmentation basis can indicate the same type of image features of the image.

[0037] In some embodiments of this application, the first basis feature of the first segmentation basis can be a vector representation of the image features contained in the first segmentation basis. It can be understood that the first segmentation basis can correspond one-to-one with the first basis feature, that is, M first segmentation basis bases correspond to M first basis features.

[0038] For example, a first segmentation basis can indicate the hair color and hair shape features of the main person in the image, and the first basis feature vector of a first segmentation basis can be a comprehensive vector representation of the hair color and hair shape features of the main person in the image.

[0039] In some embodiments of this application, each of the M first segmentation bases can be used to indicate any of the following:

[0040] A type of image feature of the main subject of the image;

[0041] A type of attachment feature of the main body of the image;

[0042] A type of detailed feature of the main body of an image.

[0043] For example, taking a first segmentation basis to indicate a class of image features of the main body of the image, the first segmentation basis can indicate: pupil color features of the eye, size features of the eye, shape features of the eye, contour features of the eye, etc.

[0044] In some embodiments of this application, the attachments to the image subject may include: items, accessories, or pendants attached to the image subject, such as a backpack, earrings, and hat worn by the person.

[0045] In some embodiments of this application, the detailed features of the image subject may include: finer features in the edge region of the image subject, such as the hair of a person or the edges of the leaves of a plant.

[0046] Thus, since each of the M first segmentation bases can indicate a type of image feature of the image subject, a type of attachment feature of the image subject, or a type of detail feature of the image subject, it can be ensured that the M first segmentation bases can indicate various features related to the image subject, thereby ensuring accurate segmentation of the image subject in the image to be processed based on the M first segmentation bases.

[0047] In this embodiment of the application, each target basis feature vector can be a vector representation of a type of image feature of the main body of the image in the first image, that is, each target basis feature vector can correspond to a type of image feature of the main body of the image in the first image.

[0048] It can be understood that M target basis feature vectors can represent the first image. For example, M target basis feature vectors can represent the image features related to the subject in the first image.

[0049] In some embodiments of this application, for the i-th first segmentation basis among M first segmentation basis, the image segmentation device can use formula (1) to perform weighted calculation on the i-th first segmentation basis using the i-th first basis coefficient corresponding to the i-th first segmentation basis to obtain the i-th target basis feature vector.

[0050] P i =w i C i (1)

[0051] Among them, P i w can be the i-th target basis feature vector. i C can represent the i-th first basis coefficient. i It can represent the first basis feature vector of the i-th first segmentation basis, where i is an integer greater than 0 and less than or equal to M.

[0052] It can be understood that the image segmentation device can obtain a target basis feature vector by using a first basis coefficient to perform weighted calculation on the first basis feature vector of the corresponding first segmentation basis through formula (1), and finally obtain M target basis feature vectors.

[0053] In some embodiments of this application, the area of ​​the main image of each image segmentation sample in a plurality of image segmentation samples may be greater than or equal to a preset ratio to the total area of ​​that image segmentation sample.

[0054] For example, the preset ratio can range from 51% to 100%, such as 70%, 80%, 90%, or 95%.

[0055] In some embodiments of this application, when the area of ​​the main body in an image segmentation sample is relatively small, the image segmentation sample can be cropped so that the area of ​​the main body in the cropped image segmentation sample is greater than or equal to a preset ratio. Furthermore, the main body of the cropped image segmentation sample can be located within the target region, thereby facilitating the learning and training of the neural network model.

[0056] For example, the target region can be the central region of the cropped image segmentation sample.

[0057] In some embodiments of this application, the aforementioned plurality of image segmentation samples may include at least one of the following: at least two images with different subject matter, at least two images with different subject matter poses, or at least two images with different subject matter categories. Thus, the basis coefficient prediction model is trained using as many different image segmentation samples as possible, so that the basis coefficient prediction model can accurately predict the M basis coefficients of various images to be processed; and that the subject matter of most images can be represented by the first basis feature vectors of the M first segmentation basis.

[0058] For example, multiple image segmentation samples can include: images with animals as the main subject, images with people as the main subject, images with landscapes as the main subject, images with buildings as the main subject, etc.

[0059] For further descriptions of training the basis coefficient prediction model and extracting the first basis feature vectors of the M first segmentation basis, please refer to the specific descriptions in the following embodiments.

[0060] Step 103: The image segmentation device segments the main image in the first image based on M target basis feature vectors.

[0061] In this embodiment of the application, after segmenting the main body of the image in the first image, a segmentation result of the main body of the image corresponding to the first image can be obtained. This segmentation result can distinguish between the main body of the image in the first image and the image region in the first image other than the main body, that is, segmenting the main body of the image in the first image can segment the main body of the image and the image background.

[0062] In this embodiment of the application, since each target basis feature vector corresponds to a type of image feature of the image subject in the first image, the region where the image subject is located in the first image can be accurately determined by M target basis feature vectors, so that the image subject in the first image can be accurately segmented according to the region where the image subject is located.

[0063] In some embodiments of this application, the image segmentation device can segment the main body of the image in the first image according to M target basis feature vectors using formula (2).

[0064]

[0065] Where S can represent the vector representation of the image subject segmentation result corresponding to the first image, and M can represent the number of the first segmentation basis; It can represent a constant vector with a fixed value.

[0066] For example, The average initial basis feature vector of all initial segmentation basis vectors extracted from multiple image segmentation samples can be selected. See the relevant description in the following embodiments for a description of the initial segmentation basis.

[0067] In some embodiments of this application, S in the above formula (2) can indicate the image subject and image background in the first image using a binary method.

[0068] For example, the image segmentation device can use a "0" value in S to represent the image background and a "1" value in S to represent the image subject.

[0069] For example, suppose the first image is Figure 2A The image shown can be used to determine the subject segmentation result based on the first image. Figure 2B The image segmentation diagram shown. It can be seen that... Figure 2B The black area shown is the background area of ​​the first image, that is, the area indicated by the "0" value in vector S; Figure 2B The white area shown is the main image area of ​​the first image, and the area indicated by the "1" value in vector S.

[0070] This application provides an image segmentation method. Since the basis coefficient prediction model is trained on multiple image segmentation samples, and the first basis feature vectors of the M first segmentation basis are extracted from these samples, the M first basis coefficients of the image to be processed predicted by the basis coefficient prediction model can be used to weight the M first basis feature vectors of the M first segmentation basis to obtain M target basis feature vectors that characterize the image to be processed. Therefore, the main body of the image to be processed can be directly segmented based on these M target basis feature vectors. Thus, since the main body of the image to be processed can be segmented using a finite number of preset basis feature vectors and basis coefficients predicted by the basis prediction model that correspond one-to-one with the preset segmentation basis, compared to the method of first predicting the segmentation values ​​of all pixels in the image to be processed and then segmenting the main body based on these pixel values, the image segmentation method provided in this application reduces the computational load for segmenting the main body of the image, thereby reducing the time spent on segmenting the main body and improving the segmentation speed.

[0071] In some embodiments of this application, before step 101 above, the image segmentation device may first determine the first basis feature vectors of the M first segmentation basis based on the image feature information of multiple image segmentation samples.

[0072] In some embodiments of this application, combined with Figure 1 ,like Figure 3 As shown, prior to step 101 above, the image segmentation method provided in this application embodiment may further include steps 104 and 105 as described below.

[0073] Step 104: The image segmentation device acquires image feature information of multiple image segmentation samples.

[0074] The image feature information of each image segmentation sample in the multiple image segmentation samples may include: the basis feature vector of the initial segmentation basis corresponding to it, and the initial basis coefficients corresponding to the initial segmentation basis.

[0075] For example, an image feature can be represented as: 0.5 * vector a, where vector a is the basis feature vector of the initial segmentation basis corresponding to the image feature, and 0.5 is the initial basis coefficient corresponding to the initial segmentation basis.

[0076] It can be seen that for any image segmentation sample, the number of initial segmentation bases corresponding to the image segmentation sample is the same as the number of image feature information of the image segmentation sample.

[0077] In this embodiment, the image segmentation device can extract feature information from multiple image segmentation samples. Specifically, for each image segmentation sample, the image segmentation device can first reduce the resolution of the image segmentation sample, and then obtain the image feature information of the image segmentation sample through the high semantic feature layer of the image segmentation sample after the resolution reduction.

[0078] For example, suppose an image segmentation sample has a resolution of 512*512. The image segmentation device can first perform sampling processing on the image segmentation sample to obtain a high semantic feature layer with a specific resolution, and then obtain the image feature information of the image segmentation sample through the high semantic feature layer.

[0079] For example, the specific resolution can be any of the following: 128*128, 64*64, 32*32, 16*16.

[0080] Step 105: The image segmentation device determines M first basis feature vectors based on the image feature information of multiple image segmentation samples.

[0081] Among them, the above M first basis feature vectors correspond one-to-one with the M first segmentation basis.

[0082] In some embodiments of this application, the image segmentation apparatus can process the image feature information of multiple image segmentation samples by means of principal component analysis (PCA) or clustering to obtain M first basis feature vectors of M first segmentation basis.

[0083] The following sections will detail the processes of obtaining the first basis feature vectors of the M first segmentation basis using PCA (one possible implementation) and clustering (another possible implementation).

[0084] One possible implementation: Obtain M first basis feature vectors using PCA.

[0085] In some embodiments of this application, the image segmentation apparatus can construct a covariance matrix based on image feature information from multiple image segmentation samples. Then, the image segmentation apparatus can use this covariance matrix to determine the first basis feature vectors of the M first segmentation basis vectors. In other words, the image segmentation apparatus can construct a covariance matrix based on the vector representation of image feature information from multiple image segmentation samples, and then determine the M first basis feature vectors of the M first segmentation basis vectors using this covariance matrix.

[0086] Specifically, 1) For each image segmentation sample among multiple image segmentation samples, a first vector can be constructed based on all image feature information of the image segmentation sample. This first vector consists of the basis feature vector of the initial segmentation basis corresponding to the image segmentation sample and the initial basis coefficients corresponding to the initial segmentation basis. It can be seen that one image segmentation sample corresponds to one first vector, and multiple first vectors can be obtained in the end.

[0087] 2) Calculate the average vector of multiple first vectors that correspond one-to-one with multiple image segmentation samples. This average vector is used to represent the average image feature information of the multiple image segmentation samples, such as a "generic face".

[0088] 3) Determine the difference between each of the multiple first vectors and the average vector to obtain multiple second vectors that correspond one-to-one with the multiple first vectors.

[0089] 4) Use these multiple second vectors to construct the covariance matrix.

[0090] 5) Using the eigenvalue decomposition method, determine multiple eigenvalues ​​and multiple eigenvectors corresponding to the covariance matrix, and there is a one-to-one correspondence between the multiple eigenvalues ​​and the multiple eigenvectors.

[0091] 6) Sort the multiple feature values ​​in descending order and select the feature vectors corresponding to the first M feature values.

[0092] 7) A feature vector P is formed by using each of the M feature vectors corresponding to the M feature values ​​as a row vector, thereby transforming the image feature information of multiple image segmentation samples into a new space of M feature vectors. It should be noted that each row vector in the feature vector P is a first basis feature vector of one of the M first segmentation basis mentioned above.

[0093] It can be understood that any of the above image segmentation samples can be represented by the first basis feature vectors of the M first segmentation basis.

[0094] It should be noted that the larger the eigenvalue, the greater the importance or proportion of the image feature information indicated by the corresponding eigenvector. In other words, the image feature information indicated by the above M eigenvectors is the main image feature information of multiple image segmentation samples.

[0095] Thus, since PCA can preserve the main image feature information of multiple image segmentation samples and filter out the secondary image feature information of multiple image segmentation samples, it can ensure that the image feature information of any image, especially the image feature information of the main body of the image, is accurately represented by the first basis feature vectors of M first segmentation basis. On the other hand, it can reduce the amount of computation when segmenting the main body of the image in any image.

[0096] Another possible implementation: obtain M first basis feature vectors by clustering.

[0097] In some embodiments of this application, the image segmentation apparatus can cluster the image feature information of multiple image segmentation samples using a density-based spatial clustering algorithm (DBSCAN) or a k-means clustering algorithm to determine the first basis feature vectors of M first segmentation basis vectors.

[0098] Specifically, the image feature information of multiple image segmentation samples is clustered using the DBSCAN algorithm or k-means clustering algorithm to obtain vector representations of M centroid segmentation maps, and these M centroid segmentation maps are used as the first basis feature vectors of the aforementioned M first segmentation basis.

[0099] Among them, each of the M centroid segmentation maps can be a cluster of image feature information of the same image subject category.

[0100] It is understandable that all images can be represented by the first basis feature vectors of the M first segmentation basis.

[0101] In some embodiments of this application, for detailed regions (such as hair regions) in the first image, the M first segmentation bases can be supplemented by manual design.

[0102] Thus, since the image segmentation device can determine M first basis feature vectors corresponding one-to-one with the M first segmentation basis vectors based on the image feature information of multiple image segmentation samples, the basis feature vectors of the M first segmentation basis vectors can characterize the features of the image subject in multiple image segmentation samples, thereby ensuring the compatibility of the first basis feature vectors of the M first segmentation basis vectors. Therefore, when segmenting the image subject in any image, since the image subject in the image to be processed can be segmented using the basis feature vectors of a finite number of preset segmentation basis vectors, compared to the scheme of first predicting the segmentation values ​​of all pixels in the image to be processed and then segmenting the image subject based on the segmentation values ​​of all pixels, the image segmentation method provided in this application embodiment can reduce the computational load when segmenting the image subject in the image, thereby reducing the time consumed in segmenting the image subject in the image and improving the segmentation speed.

[0103] In some embodiments of this application, after step 104 above, the image segmentation method provided in this application may further include steps 106 and 107 as described below.

[0104] Step 106: The image segmentation device determines the basis coefficient vector corresponding to each image segmentation sample based on the image feature information in each image segmentation sample and the M first basis feature vectors of the M first segmentation basis.

[0105] Among them, the aforementioned basis coefficient vector may contain M second basis coefficients that correspond one-to-one with the M first partitioning basis.

[0106] It is understandable that each image segmentation sample in multiple image segmentation samples can correspond to a basis coefficient vector, and different image segmentation samples can correspond to different basis coefficient vectors.

[0107] In some embodiments of this application, in one possible implementation described above, where the first basis feature vectors of the M first segmentation basis are obtained via PCA, for each image segmentation sample among multiple image segmentation samples, the image segmentation device can multiply a first vector constructed from all image feature information of an image segmentation sample with the aforementioned feature vector P to obtain the basis coefficient vector corresponding to that image segmentation sample. This basis coefficient vector may include M second basis coefficients corresponding one-to-one with the M first segmentation basis. The first vector is a column vector, and each term of the first vector indicates an image feature information of that image segmentation sample.

[0108] In some embodiments of this application, in another possible implementation, where the first basis feature vectors of the M first segmentation bases are obtained through clustering, for each image segmentation sample among multiple image segmentation samples, the image segmentation device can multiply the first vector constructed from all image feature information of an image segmentation sample by the vector representations of the M centroid segmentation maps to obtain the basis coefficient vector corresponding to that image segmentation sample. This basis coefficient vector may include M second basis coefficients corresponding one-to-one with the M first segmentation bases. The first vector is a column vector, and each term of the first vector indicates an image feature information of the image segmentation sample.

[0109] Step 107: The image segmentation device uses multiple image segmentation samples and the basis coefficient vectors corresponding to the multiple image segmentation samples to train a neural network model to obtain a basis coefficient prediction model.

[0110] In some embodiments of this application, the image segmentation apparatus may use multiple sets of training samples to train a neural network model to obtain a basis coefficient prediction model. Each set of training samples may include an image segmentation sample and the basis coefficient vector corresponding to that image segmentation sample.

[0111] During the training of the neural network model using the aforementioned multiple sets of training samples, the neural network model can fit and output a predicted basis coefficient vector containing M predicted basis coefficients. In the process of fitting the predicted basis coefficient vector, the difference between the predicted basis coefficient vector output by the neural network model and the basis coefficient vector in the training samples can be continuously reduced through optimizer iteration and loss function calculation (such as L1 loss, L2 loss, etc.) until it is infinitely close to the basis coefficient vector. At this point, the neural network model can be used as a basis coefficient prediction model.

[0112] Thus, since the basis coefficient prediction model is trained on the neural network model using multiple image segmentation samples and the basis coefficient vectors corresponding to those multiple image segmentation samples, and the M basis coefficients in the basis coefficient vectors corresponding to those multiple image segmentation samples are the true basis coefficients corresponding to the image segmentation sample represented by the basis feature vectors of the M first segmentation basis, the accuracy of the basis coefficient prediction model in predicting basis coefficients can be improved.

[0113] In some embodiments of this application, using a basic shape segmentation basis may result in poor fitting of images with large or extreme movements. Therefore, different deformations based on the average segmentation can be added as motion basis. In this way, even if the image subject in the subsequent image to be processed is severely deformed, the image subject in the image to be processed can still be well fitted by the basis feature vectors of the M first segmentation basis.

[0114] In some embodiments of this application, combined with Figure 1 ,like Figure 4 As shown, prior to step 101 above, the image segmentation method provided in this application embodiment may further include step 108 as described below.

[0115] Step 108: The image segmentation device performs pose deformation processing on the main body of N initial image segmentation samples from multiple initial image segmentation samples to obtain N deformed image segmentation samples that correspond one-to-one with the N initial image segmentation samples.

[0116] The aforementioned multiple image segmentation samples may include N deformed image segmentation samples and multiple initial image segmentation samples, where N is a positive integer.

[0117] In some embodiments of this application, the above-mentioned posture deformation processing may include at least one of the following: face offset, hand offset, body twisting, etc.

[0118] In some embodiments of this application, when multiple image segmentation samples include N deformed image segmentation samples and multiple initial image segmentation samples, the image segmentation device can first determine the first basis feature vectors of some of the M first segmentation bases using the image feature information of the multiple initial image segmentation samples, and then determine the first basis feature vectors of the remaining M first segmentation bases based on the image feature information of the N deformed image segmentation samples. Alternatively, the image feature information of the multiple initial image segmentation samples and the image feature information of the N deformed image segmentation samples can be directly mixed together, and then the first basis feature vectors of the M first segmentation bases can be determined according to steps 104 to 107 above. The specific method can be determined according to actual usage requirements.

[0119] It should be noted that after obtaining N deformed image segmentation samples, a neural network model needs to be trained based on these N deformed image segmentation samples and the basis coefficient vector corresponding to each of the N deformed image segmentation samples. In other words, the neural network model needs to be trained using N deformed image segmentation samples and their corresponding basis coefficient vectors, as well as multiple image segmentation samples and their corresponding basis coefficient vectors, to obtain a basis coefficient prediction model. This ensures that the basis coefficient prediction model can accurately predict the basis coefficients corresponding to images with significant subject deformation, such as the M first basis coefficients mentioned above.

[0120] In some embodiments of this application, when the above-mentioned multiple image segmentation samples include N deformed image segmentation samples and multiple initial image segmentation samples, the image segmentation device can segment the image subject in the first image according to M target basis feature vectors by formula (3).

[0121]

[0122] Among them, B d w can represent the basis feature vector of the d-th deformable segmentation basis. d It can represent the coefficient of the d-th deformed basis corresponding to the d-th deformed segmentation basis, where d, l, i, and k are all positive integers, and l+k=M.

[0123] It's understandable, B d It can be a segmentation basis determined based on the image feature information of N deformed image segmentation samples; w i It can be a segmentation basis determined based on image information of segmentation samples from multiple initial images.

[0124] Thus, since the image segmentation device can perform pose deformation processing on the image subject of N initial image segmentation samples out of multiple initial image segmentation samples to obtain N deformed image segmentation samples that correspond one-to-one with the N initial image segmentation samples, the first basis features of the M first segmentation basis can better fit the pose deformation image subject of the image subject, thereby further improving the accuracy of segmenting the image subject in the image.

[0125] In some embodiments of this application, combined with Figure 1 ,like Figure 5 As shown, before step 101 above, the image segmentation method provided in this application embodiment may further include step 109 below; after step 103 above, the image segmentation method provided in this application embodiment may further include step 110 below.

[0126] Step 109: The image segmentation device crops the second image to obtain the first image.

[0127] The second image contains the subject of the first image, which is located in the target area of ​​the first image.

[0128] In some embodiments of this application, the target region may be the middle region of the first image.

[0129] For example, the subject of the first image may be located in the center of the first image.

[0130] In some embodiments of this application, the image segmentation apparatus can obtain a first image by performing subject detection on a second image and cropping the second image based on the detected subject.

[0131] For example, such as Figure 6 As shown, the image segmentation device can perform subject detection on the image subject 61 in image 60. Then, the image segmentation device can crop image 60 with the image subject 61 as the center to obtain image 62, i.e., the first image, which only retains the image subject 61 and a small number of other image areas, and crop out the other image areas.

[0132] In some embodiments of this application, the image segmentation apparatus may also perform key point detection on the subject of the first image in the second image to prevent the extension area of ​​the subject of the first image, such as the outstretched fingers of a person in the image, from being ignored during subject detection.

[0133] Furthermore, in some embodiments of this application, the image segmentation device can align the detected first image subject with the cropping frame, thereby ensuring that the first image subject can be located in the middle area of ​​the first image.

[0134] Step 110: The image segmentation device performs scale restoration processing on the image subject segmentation result corresponding to the first image based on the cropping coordinates in the second image and the second image, to obtain the image subject segmentation result corresponding to the second image.

[0135] The image subject segmentation result corresponding to the first image is obtained by segmenting the image subject in the first image based on M target basis feature vectors. In other words, after the image processing device segments the image subject in the first image based on M target basis feature vectors, the image subject segmentation result corresponding to the first image can be obtained.

[0136] In some embodiments of this application, when the image segmentation device crops the second image, it can simultaneously record the cropping coordinates at the time of cropping, so as to perform scale restoration processing on the image subject segmentation result corresponding to the first image to obtain the image subject segmentation result corresponding to the second image.

[0137] For example, when cropping the second image, the image segmentation device can record the coordinates of the four vertices of the cropping frame and the coordinates of the four vertices of the second image. Thus, after obtaining the image subject segmentation result corresponding to the first image, the image subject segmentation result corresponding to the first image can be scaled back according to the recorded coordinates to obtain the image subject segmentation result corresponding to the second image.

[0138] It is understood that, in the embodiments of this application, the image subject segmentation result corresponding to the second image can be the result of the image processing device segmenting the image subject in the second image.

[0139] For example, suppose the second image is Figure 6 Image 60 shown, the first image is Figure 6 Image 62 is shown. Therefore, the image subject segmentation result corresponding to the first image is as follows: Figure 7 The image subject segmentation result shown is 70, and the image subject segmentation result corresponding to the second image is as follows: Figure 7 The image subject segmentation result shown is 71.

[0140] Thus, since the image segmentation device can crop the second image with the main body of the first image as the target area to obtain the first image; and after obtaining the image main body segmentation result corresponding to the first image, it can directly restore the image main body segmentation result corresponding to the second image based on the cropping coordinates of the second image and the image main body segmentation result corresponding to the first image. That is, in the process of segmenting the image main body in the image, the image background can be ignored and more attention can be paid to the image main body. Therefore, on the one hand, the amount of calculation when segmenting the image main body in the image can be reduced, and on the other hand, the accuracy of segmenting the image main body can be improved.

[0141] In some embodiments of this application, after step 103 described above, the image segmentation method provided in this application may further include step 111 as described below.

[0142] Step 111: The image segmentation device blurs the image region in the first image other than the image subject based on the image subject segmentation result corresponding to the first image.

[0143] In some embodiments of this application, the image segmentation apparatus can perform blurring processing on the image region other than the image subject in the first image based on the image subject segmentation result indicated by the image subject segmentation result corresponding to the first image.

[0144] Thus, since the image segmentation device can obtain the image segmentation result of the image subject corresponding to the first image and blur the image area in the first image other than the image subject, it can quickly obtain the image subject segmentation result of the image subject corresponding to the image and blur the image area in the image other than the image subject, reducing the time spent on image blurring.

[0145] Understandably, in actual implementation, the image region other than the image subject in the image subject corresponding to the second image can also be blurred based on the image subject segmentation result corresponding to the second image.

[0146] In some embodiments of this application, after obtaining the first segmentation result, enhancement processing can be performed on the main image in the first image. Of course, any possible operation can be performed on the first image based on the image main image segmentation result corresponding to the first image, and this application embodiment does not limit this.

[0147] The image segmentation method provided in this application can be executed by an image segmentation device. This application uses an image segmentation device executing the image segmentation method as an example to illustrate the image segmentation device provided in this application.

[0148] Figure 8 A schematic diagram of a possible structure of the image segmentation apparatus involved in an embodiment of this application is shown. For example... Figure 8 As shown, the image segmentation device 80 may include a processing module 81.

[0149] The processing module 81 is configured to input the first image into the basis coefficient prediction model to obtain M first basis coefficients, wherein the M first basis coefficients correspond one-to-one with the M first segmentation basis corresponding to the basis coefficient prediction model, and the basis coefficient prediction model is trained based on multiple image segmentation samples, where M is a positive integer; and to perform weighted calculation on the first basis feature vectors of the corresponding first segmentation basis using the first basis coefficients to obtain M target basis feature vectors, wherein the first basis feature vectors of the M first segmentation basis are extracted from multiple image segmentation samples; and to segment the main image in the first image based on the M target basis feature vectors.

[0150] In one possible implementation, the above-mentioned apparatus further includes: an acquisition module and a determination module;

[0151] The above-mentioned acquisition module is used to acquire image feature information of multiple image segmentation samples before the processing module 81 inputs the first image into the basis coefficient prediction model to obtain M first basis coefficients. The image feature information of each image segmentation sample in the multiple image segmentation samples includes: the basis feature vector of the initial segmentation basis corresponding to it, and the initial basis coefficient corresponding to the initial segmentation basis.

[0152] The aforementioned determining module is used to determine M first basis feature vectors based on the image feature information of multiple image segmentation samples obtained by the acquisition module, wherein the M first basis feature vectors correspond one-to-one with the M first segmentation basis vectors.

[0153] In one possible implementation, the above-mentioned device further includes a training module;

[0154] The aforementioned determining module is further configured to, after the acquisition module acquires the image feature information of multiple image segmentation samples, determine the basis coefficient vector corresponding to each image segmentation sample based on the image feature information in each image segmentation sample and the M first basis feature vectors of the M first segmentation basis, wherein the basis coefficient vector contains M second basis coefficients that correspond one-to-one with the M first segmentation basis.

[0155] The training module described above is used to train a neural network model using multiple image segmentation samples and the basis coefficient vectors corresponding to the multiple image segmentation samples determined by the determination module, so as to obtain a basis coefficient prediction model.

[0156] In one possible implementation, the above-mentioned processing module 81 is further configured to perform pose deformation processing on the image subjects of N initial image segmentation samples among multiple initial image segmentation samples before inputting the first image into the basis coefficient prediction model to obtain M first basis coefficients, so as to obtain N deformed image segmentation samples that correspond one-to-one with the N initial image segmentation samples.

[0157] The multiple image segmentation samples include N deformed image segmentation samples and multiple initial image segmentation samples, where N is a positive integer.

[0158] In one possible implementation, the processing module 81 is further configured to crop the second image to obtain the first image before inputting the first image into the basis coefficient prediction model to obtain M first basis coefficients, the second image containing the main body of the first image, the main body of the first image being located in the middle region of the first image; and to perform scale restoration processing on the image main body segmentation result corresponding to the first image based on the cropping coordinates in the second image and the second image after segmenting the image main body in the first image according to the M target basis feature vectors, to obtain the image main body segmentation result corresponding to the second image;

[0159] The image subject segmentation result corresponding to the first image is obtained by segmenting the image subject in the first image based on M target basis feature vectors.

[0160] This application provides an image segmentation apparatus. Since the basis coefficient prediction model is trained based on multiple image segmentation samples, and the first basis feature vectors of the M first segmentation basis are extracted from these samples, the M first basis coefficients of the image to be processed predicted by the basis coefficient prediction model can be used to weight the M first basis feature vectors of the M first segmentation basis to obtain M target basis feature vectors that characterize the image to be processed. Therefore, the main body of the image to be processed can be directly segmented based on these M target basis feature vectors. Thus, since the main body of the image to be processed can be segmented using a finite number of preset segmentation basis feature vectors and basis coefficients predicted by the basis prediction model that correspond one-to-one with the preset segmentation basis, compared to the scheme of first predicting the segmentation values ​​of all pixels in the image to be processed and then segmenting the main body based on these pixel values, the image segmentation method provided in this application reduces the computational load for segmenting the main body of the image, thereby reducing the time spent segmenting the main body and improving the segmentation speed.

[0161] The image segmentation device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0162] The image segmentation device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0163] The image segmentation apparatus provided in this application embodiment can implement the various processes implemented in the above-described image segmentation method embodiment, achieving the same technical effect. To avoid repetition, it will not be described again here.

[0164] In some embodiments of this application, such as Figure 9 As shown, this application embodiment also provides an electronic device 900, including a processor 901 and a memory 902. The memory 902 stores a program or instructions that can run on the processor 901. When the program or instructions are executed by the processor 901, they implement the various steps of the above-described image segmentation method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0165] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0166] Figure 10 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0167] The electronic device 1000 includes, but is not limited to, components such as: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, and processor 1010.

[0168] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 10 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0169] The processor 1010 is configured to input the first image into the basis coefficient prediction model to obtain M first basis coefficients, wherein the M first basis coefficients correspond one-to-one with the M first segmentation basis corresponding to the basis coefficient prediction model, and the basis coefficient prediction model is trained based on multiple image segmentation samples, where M is a positive integer; and to perform weighted calculations on the first basis feature vectors of the corresponding first segmentation basis using the first basis coefficients to obtain M target basis feature vectors, wherein the first basis feature vectors of the M first segmentation basis are extracted from multiple image segmentation samples; and to segment the main image in the first image based on the M target basis feature vectors.

[0170] In some embodiments of this application, the processor 1010 is configured to acquire image feature information of multiple image segmentation samples before inputting the first image into the basis coefficient prediction model to obtain M first basis coefficients. The image feature information of each image segmentation sample in the multiple image segmentation samples includes: the basis feature vector of the corresponding initial segmentation basis and the initial basis coefficient corresponding to the initial segmentation basis. The processor is also configured to determine M first basis feature vectors based on the acquired image feature information of the multiple image segmentation samples, wherein the M first basis feature vectors correspond one-to-one with the M first segmentation basis.

[0171] In some embodiments of this application, the processor 1010 is further configured to, after acquiring image feature information of multiple image segmentation samples, determine the basis coefficient vector corresponding to each image segmentation sample based on the image feature information in each image segmentation sample and the M first basis feature vectors of the M first segmentation basis, wherein the basis coefficient vector contains M second basis coefficients that correspond one-to-one with the M first segmentation basis; and to train a neural network model using multiple image segmentation samples and the basis coefficient vectors corresponding to the multiple image segmentation samples to obtain a basis coefficient prediction model.

[0172] In some embodiments of this application, the processor 1010 is further configured to perform pose deformation processing on the image subjects of N initial image segmentation samples among multiple initial image segmentation samples before inputting the first image into the basis coefficient prediction model to obtain M first basis coefficients, so as to obtain N deformed image segmentation samples that correspond one-to-one with the N initial image segmentation samples.

[0173] The multiple image segmentation samples include N deformed image segmentation samples and multiple initial image segmentation samples, where N is a positive integer.

[0174] In some embodiments of this application, the processor 1010 is further configured to crop the second image to obtain the first image before inputting the first image into the basis coefficient prediction model to obtain M first basis coefficients, the second image containing the main body of the first image and the main body of the first image being located in the middle region of the first image; and to perform scale restoration processing on the image main body segmentation result corresponding to the first image based on the cropping coordinates in the second image and the second image after segmenting the image main body in the first image according to the M target basis feature vectors, to obtain the image main body segmentation result corresponding to the second image;

[0175] The image subject segmentation result corresponding to the first image is obtained by segmenting the image subject in the first image based on M target basis feature vectors.

[0176] This application provides an electronic device. Since the basis coefficient prediction model is trained based on multiple image segmentation samples, and the first basis feature vectors of the M first segmentation basis are extracted from the multiple image segmentation samples, the M first basis coefficients of the image to be processed predicted by the basis coefficient prediction model can be used to weight the M first basis feature vectors of the M first segmentation basis to obtain M target basis feature vectors that can characterize the image to be processed. Therefore, the main body of the image to be processed can be directly segmented based on the M target basis feature vectors. Thus, since the main body of the image to be processed can be segmented using the basis feature vectors of a finite number of preset segmentation basis and the basis coefficients predicted by the basis prediction model that correspond one-to-one with the preset segmentation basis, the main body of the image to be processed can be segmented. Therefore, compared with the scheme of first predicting the segmentation values ​​of all pixels in the image to be processed and then segmenting the main body of the image based on the segmentation values ​​of all pixels, the image segmentation method provided by this application can reduce the computational amount of segmenting the main body of the image, thereby reducing the time consumption of segmenting the main body of the image and improving the segmentation speed.

[0177] It should be understood that, in this embodiment, the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042. The GPU 10041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1007 includes a touch panel 10071 and at least one of other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0178] The memory 1009 can be used to store software programs and various data. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0179] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor 1010.

[0180] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image segmentation method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0181] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0182] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image segmentation method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0183] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0184] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the image segmentation method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0185] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0186] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0187] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image segmentation method characterized by, The method comprises: inputting a first image into a base coefficient prediction model to obtain M first base coefficients, wherein the M first base coefficients correspond one-to-one to M first segmentation bases corresponding to the base coefficient prediction model, the base coefficient prediction model is obtained based on a plurality of image segmentation samples, and M is a positive integer; performing weighted calculation on first base feature vectors of the M first segmentation bases by using the first base coefficients to obtain M target base feature vectors, wherein the first base feature vectors of the M first segmentation bases are extracted from the plurality of image segmentation samples; segmenting an image subject in the first image according to the M target base feature vectors.

2. The method of claim 1, wherein, Before the step of inputting the first image into the base coefficient prediction model to obtain the M first base coefficients, the method further comprises: obtaining image feature information of the plurality of image segmentation samples, wherein the image feature information of each image segmentation sample in the plurality of image segmentation samples comprises base feature vectors of initial segmentation bases corresponding to the image segmentation sample and initial base coefficients corresponding to the initial segmentation bases; determining M first base feature vectors based on the image feature information of the plurality of image segmentation samples, wherein the M first base feature vectors correspond one-to-one to the M first segmentation bases.

3. The method of claim 2, wherein, After the step of obtaining the image feature information of the plurality of image segmentation samples, the method further comprises: determining a base coefficient vector corresponding to each image segmentation sample according to the image feature information in the image segmentation sample and M first base feature vectors of the M first segmentation bases, wherein the base coefficient vector comprises M second base coefficients corresponding one-to-one to the M first segmentation bases; training a neural network model by using the plurality of image segmentation samples and the base coefficient vectors corresponding to the plurality of image segmentation samples to obtain the base coefficient prediction model.

4. The method according to claim 1 or 2, characterized in that, Before the step of inputting the first image into the base coefficient prediction model to obtain the M first base coefficients, the method further comprises: performing pose deformation processing on image subjects of N initial image segmentation samples in a plurality of initial image segmentation samples to obtain N deformed image segmentation samples corresponding one-to-one to the N initial image segmentation samples; wherein the plurality of image segmentation samples comprise the N deformed image segmentation samples and the plurality of initial image segmentation samples, and N is a positive integer.

5. The method of claim 1, wherein, Before the step of inputting the first image into the base coefficient prediction model to obtain the M first base coefficients, the method further comprises: cropping a second image to obtain the first image, wherein the second image comprises a first image subject, and the first image subject is located in a target region of the first image; After the step of segmenting the image subject in the first image according to the M target base feature vectors, the method further comprises: performing scale restoration processing on an image subject segmentation result corresponding to the first image based on a cropping coordinate in the second image and the second image to obtain an image subject segmentation result corresponding to the second image. The image subject segmentation result corresponding to the first image is obtained by segmenting an image subject in the first image according to the M target basis feature vectors.

6. An image segmentation apparatus characterized by comprising: The device comprises a processing module; The processing module is configured to input a first image into a basis coefficient prediction model to obtain M first basis coefficients, wherein the M first basis coefficients correspond to M first segmentation bases corresponding to the basis coefficient prediction model one by one, the basis coefficient prediction model is obtained by training based on a plurality of image segmentation samples, and M is a positive integer. The processing module is further configured to perform weighted calculation on first basis feature vectors of the first segmentation bases corresponding to the first basis coefficients to obtain M target basis feature vectors, wherein the first basis feature vectors of the M first segmentation bases are extracted from the plurality of image segmentation samples. The processing module is further configured to segment an image subject in the first image according to the M target basis feature vectors.

7. The apparatus of claim 6, wherein, The device further comprises an obtaining module and a determining module. The obtaining module is configured to obtain image feature information of the plurality of image segmentation samples before the processing module inputs the first image into the basis coefficient prediction model to obtain the M first basis coefficients, wherein the image feature information of each image segmentation sample in the plurality of image segmentation samples comprises basis feature vectors of initial segmentation bases corresponding to the initial segmentation bases and initial basis coefficients corresponding to the initial segmentation bases. The determining module is configured to determine M first basis feature vectors based on the image feature information of the plurality of image segmentation samples obtained by the obtaining module, wherein the M first basis feature vectors correspond to M first segmentation bases one by one.

8. The apparatus of claim 7, wherein, The device further comprises a training module. The determining module is further configured to determine a basis coefficient vector corresponding to each image segmentation sample according to the image feature information in the each image segmentation sample and M first basis feature vectors of the M first segmentation bases after the obtaining module obtains the image feature information of the plurality of image segmentation samples, wherein the basis coefficient vector comprises M second basis coefficients corresponding to the M first segmentation bases one by one. The training module is configured to train a neural network model by using the plurality of image segmentation samples and the basis coefficient vectors corresponding to the plurality of image segmentation samples determined by the determining module to obtain the basis coefficient prediction model.

9. The apparatus of claim 6 or 7, wherein, The processing module is further configured to perform pose deformation processing on image subjects of N initial image segmentation samples in a plurality of initial image segmentation samples to obtain N deformed image segmentation samples corresponding to the N initial image segmentation samples one by one before inputting the first image into the basis coefficient prediction model to obtain the M first basis coefficients. The plurality of image segmentation samples comprise the N deformed image segmentation samples and the plurality of initial image segmentation samples, and N is a positive integer.

10. The apparatus of claim 6, wherein, The processing module is further configured to crop the second image to obtain the first image before inputting the first image into the basis coefficient prediction model to obtain M first basis coefficients, wherein the second image contains a first image subject, and the first image subject is located in a middle region in the first image. The processing module is further configured to perform scale restoration processing on the image subject segmentation result corresponding to the first image based on the cropping coordinates in the second image and the second image after segmenting the image subject in the first image according to the M target basis feature vectors to obtain an image subject segmentation result corresponding to the second image. The image subject segmentation result corresponding to the first image is obtained by segmenting the image subject in the first image according to the M target basis feature vectors.

Images