A generalized convolution-based image semantic segmentation method and system

Abstract

The application provides a kind of image semantic segmentation method and system based on generalized convolution, and the information of feature layer is restored at pixel level by down-sampling convolution layer and deep feature layer, which is based on the mathematical basis of generalized inverse matrix, and can realize pixel-level recovery of information by transforming convolution matrix. On the one hand, generalized convolution can restore the size of feature map, and can construct a new up-sampling method for image semantic segmentation field. On the other hand, the up-sampling method of accurate matrix can make the up-sampling and down-sampling of the network symmetrical, so that only the parameters of the prediction part need to be fine-tuned when deploying.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of semantic segmentation in computer vision, and particularly relates to an image semantic segmentation method and system based on generalized convolution. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] With the deepening of computer vision research, the high-speed propagation and large-scale application of the Internet make computers play an increasingly important role in life. Technologies such as autonomous driving, medical image segmentation, and farmland segmentation rely on computers, not only liberating part of human work, but also promoting the development of productivity. In this case, the accuracy and interpretability of computer vision become indispensable.

[0004] The semantic segmentation problem meets the requirement of more accurate analysis and understanding of images. The fundamental purpose of image semantic segmentation is to determine the semantic class of each pixel in the image by training the content of the image. For example, using a fully convolutional neural network to realize FCN for pixel-level classification of images, thereby solving the image segmentation problem at the semantic level. FCN can accept input images of any size, and uses deconvolution to up-sample the feature map of the last convolutional layer to restore it to the size of the input image.

[0005] Currently, existing technologies mostly focus on generating new data using various functions, such as bilinear interpolation and transposed convolution, using self-constructed functions or computer-generated functions to extract additional information from deep feature layers to achieve the effect of upsampling, but lack of finding lost information from the downsampling process of the network. Among them, the low-pass filtering property of bilinear interpolation is easy to damage high-frequency components in the calculation process, resulting in blurred contours and loss of small object information; while transposed convolution can only restore the new feature map to the size before downsampling in the mathematical theory level, but cannot make up for the information lost in the downsampling process. That is, the current upsampling method cannot restore the information lost in the downsampling process, affecting the final output result. SUMMARY

[0006] To solve the above problems, the first aspect of the present application proposes a semantic segmentation method for restoring the down-sampled image using generalized convolution. This method is based on the mathematical basis of generalized inverse matrix and can realize pixel-level restoration of image information by transforming the convolution matrix. On the one hand, generalized convolution can restore the size of the feature map, which can construct a new upsampling method for the field of image semantic segmentation; on the other hand, the accurate matrix upsampling method can make the upsampling and downsampling of the network symmetrical, so that only the parameters of the prediction part need to be fine-tuned when deploying.

[0007] To achieve the above object, the present application mainly includes the following aspects:

[0008] In a first aspect, the embodiment one of the present application provides a generalized convolution-based image semantic segmentation method.

[0009] A generalized convolution-based image semantic segmentation method, comprising:

[0010] Step 1: extracting the initial feature map of the input image;

[0011] Step 2: performing down-sampling operation on the initial feature map;

[0012] Step 3: repeating the convolution operation of step 2 for k times to obtain k expressions;

[0013] Step 4: splicing the k expressions to obtain a splicing matrix;

[0014] Step 5: restoring the initial feature map by using the generalized inverse matrix on the splicing matrix;

[0015] Wherein, k is the number of convolution operations.

[0016] In a possible implementation, the down-sampling operation uses stride convolution to perform down-sampling operation on the feature layer.

[0017] In a possible implementation, the stride convolution is used to perform down-sampling operation on the feature layer, comprising the following steps:

[0018] (1) performing flattening operation on the original feature map with size (a, a) to obtain a row vector A (1*n) with length n=a 2 ;

[0019] (2) expanding the convolution kernel with size (b, b) into a convolution matrix P, P has a shape of (n, (a-b+1) 2 ), denoted as P (n, m) ;

[0020] (3) performing AP=B calculation on the original feature map, where A is the row vector after flattening the original feature map, and B is the row vector after flattening the new feature map;

[0021] (4) due to the particularity of convolution operation, n≥m without adding padding;

[0022] Wherein, a represents the size value of the original feature map; b represents the size value of the convolution kernel; n represents the number of information points in the original feature map, i.e.a 2 ; m represents the number of columns of the convolution matrix after expanding the convolution kernel into the convolution matrix.

[0023] In one possible implementation, the value of k is calculated as follows:

[0024] (1) Starting from 1, calculate the value of k*m;

[0025] (2) Compare the value of k*m with n;

[0026] (3) If n > k*m, increment k by 1 and return to step (1);

[0027] (4) If n≤k*m, obtain the required value of k, and let q=k*m, where q is the number of columns of the new matrix obtained by concatenating k convolution matrices, and represents the total number of information points in all new feature maps.

[0028] In one possible implementation, repeating step 2 k times for the convolution operation to obtain k expressions is as follows: performing k stride convolutions on row vector A yields k expressions:

[0029] AP1 = B1

[0030] AP2 = B2

[0031] …

[0032] AP k =B k ;

[0033] Among them, P1, P2, ..., P k Let B1, B2, ..., B be the convolution matrices corresponding to the (1-k)th convolution operations. k This is the row vector after flattening the new feature map corresponding to the (1-k)th convolution operation.

[0034] In one possible implementation, k expressions are concatenated to obtain:

[0035] A[P1 P2…P k ] = [B1 B2…B k Let A(1,n)Q(n,q)=C(1,q), where C represents the flattened vector of all new feature maps obtained in this round of calculation, and Q represents the matrix obtained by concatenating k step convolution matrices.

[0036] In one possible implementation, obtaining the initial feature map by restoring the concatenated matrix through the generalized inverse matrix includes the following steps:

[0037] (1) Transpose both sides of the expression A(1,n)Q(n,q)=C(1,q) to get Q T (q,n)A T (n,1)=C T (q,1);

[0038] (2) Matrix Q is obtained by splicing k-step convolution matrix, so QQ T is reversible.

[0039] (3) According to the knowledge of generalized inverse matrix, a unique solution A=CQ is obtained by Q and C T (QQ T ) -T .

[0040] In a second aspect, the embodiment of the present application provides an image semantic segmentation system based on generalized convolution.

[0041] An image semantic segmentation system based on generalized convolution comprises:

[0042] A feature extraction module extracts initial feature maps of an input image.

[0043] A down-sampling module performs a down-sampling operation on the initial feature maps.

[0044] A convolution module performs a convolution operation k times on the down-sampling process to obtain k expressions.

[0045] A splicing module splices the k expressions to obtain a spliced matrix.

[0046] A restoration module restores the initial feature maps from the spliced matrix through a generalized inverse matrix.

[0047] Wherein, k is the number of convolution operations.

[0048] In a third aspect, the embodiment of the present application provides a computer device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the image semantic segmentation method based on generalized convolution as described in any possible implementation manner of the first aspect.

[0049] In a fourth aspect, the embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the image semantic segmentation method based on generalized convolution as described in any possible implementation manner of the first aspect.

[0050] The present application has the following beneficial effects:

[0051] (1) The application proposes a generalized convolution method for upsampling in various segmentation scenes, which introduces the concept of generalized inverse matrix in the matrix, restores the information of the feature layer of the image at the pixel level through the convolution layer and the deep feature layer of the down-sampling, and solves the problem that the transpose convolution method used in the traditional restoration process only realizes the restoration of the original feature map size through the transpose operation of the convolution matrix, without considering the information loss in the network forward propagation process.

[0052] (2) The basic design idea of the generalized convolution comes from the transpose convolution which changes the shape of the feature map by matrix on demand, and on this basis, the generalized convolution uses the stride convolution in the down-sampling to find the information lost in the initial down-sampling process, and considers the restoration and size reduction of the image information, so that various segmentation technologies can realize accurate upsampling, and the method has excellent semantic segmentation effect for portrait, landscape and building images. BRIEF DESCRIPTION OF DRAWINGS

[0053] The drawings accompanying the specification of the application form a part of the application and serve to further understand the application, the illustrative embodiments of the application and the description thereof serve to explain the application and do not constitute an improper limitation of the application.

[0054] Figure 1 is a flowchart of the method of the embodiment of the application;

[0055] Figure 2 is the original feature map of the embodiment of the application;

[0056] Figure 3 is the convolution kernel diagram of the embodiment of the application;

[0057] Figure 4 is a schematic diagram of the extension of the convolution kernel of the embodiment of the application;

[0058] Figure 5 is a flattened schematic diagram of the extended convolution kernel of the embodiment of the application;

[0059] Figure 6 is a schematic diagram of the down-sampling processing of the original feature map of the embodiment of the application;

[0060] Figure 7 is a schematic diagram of the expression splicing of the embodiment of the application;

[0061] Figure 8 is a schematic diagram of the expression splicing of the embodiment of the application

[0062] Figure 9 is a neural network diagram built in the application scenario of the embodiment of the application.

[0063] Figure 10is an image semantic segmentation effect display diagram of an embodiment of the present application. DETAILED DESCRIPTION

[0064] The present application will be further described below in conjunction with the drawings and embodiments.

[0065] Embodiment one

[0066] Origin of the idea of generalized convolution

[0067] The present application relates to feature restoration in the three fields of image semantic segmentation, instance segmentation and panorama segmentation, so as to realize accurate up-sampling at the pixel level in the classification process. In the three fields of semantic segmentation, instance segmentation and panorama segmentation, the basic operation is to first down-sample the image through maximum pooling and extract the content and edge information of the image, then after the prediction process without changing the size, the deep feature map at the end of the prediction is geometrically restored, so that the final output image has the same size as the input image, and the result after each pixel classification is displayed in the output image.

[0068] However, the maximum pooling down-sampling will inevitably lose most of the pixel information, and in the segmentation task, due to the "spatial insensitivity", it is easy to reduce the accuracy of the final result. In the traditional restoration process, the most commonly used transposed convolution method only realizes the restoration of the size of the original feature map through the transposition operation of the convolution matrix, and does not consider the information loss caused by the decrease of the information storage amount due to the decrease of the size of the feature map in the forward propagation process of the network, and such information loss cannot be recovered by a simple linear interpolation algorithm; and the basic design idea of generalized convolution comes from the change of the shape of the feature map by the matrix in the transposed convolution, and on this basis, the generalized convolution simultaneously utilizes the stride convolution in the down-sampling to find the information lost in the down-sampling process, and considers the restoration of the information and the restoration of the size, so that various segmentation techniques can realize accurate up-sampling.

[0069] The algorithm proposed in the present application is based on the mathematical basis of the generalized inverse matrix, and can realize pixel-level restoration of information by transforming the convolution matrix. On the one hand, the generalized convolution can restore the size of the feature map, and can construct a new up-sampling method for the field of image semantic segmentation; on the other hand, since the up-sampling method proposed in the present application utilizes the information in the down-sampling process, it can realize the mutual symmetry of the up-sampling and down-sampling of the network structure (as shown in Figure 9 ), so that when deployed, only the parameters of the transformation part (i.e. the prediction part) of the deepest feature map can be adjusted, and the parameters of the up-sampling and down-sampling stages on both sides can be frozen, realizing the lightweight of the network in application.

[0070] Please refer to Figure 1 , Figure 1This is a flowchart of an image semantic segmentation method based on generalized convolution, according to an embodiment of the present invention. Figure 1 As shown, to achieve semantic segmentation of an image based on generalized convolution, the image is first input into the network, and a conventional single-step convolution is performed to extract feature maps. Then, generalized convolution is used to restore the downsampled image, including the following steps:

[0071] Step 1: Extract the initial feature map of the input image;

[0072] Step 2: Perform a downsampling operation on the initial feature map;

[0073] Step 3: Repeat the convolution operation from step 2 k times to obtain k expressions;

[0074] Step 4: Concatenate the k expressions to obtain the concatenation matrix;

[0075] Step 5: Obtain the initial feature map by restoring the spliced ​​matrix using the generalized inverse matrix.

[0076] In this embodiment of the invention, as an optional embodiment, the image in step 1 can be an image of a person, landscape, or building.

[0077] Step 2 involves downsampling the feature layer using strided convolution, specifically including the following steps:

[0078] (1) Flatten the feature map of size (a, a) as follows: Figure 2 As shown, the length is obtained as

[0079] n = a 2 The row vector A(1*n);

[0080] (2) The convolution kernel of size (b, b) is processed according to... Figure 4 The result is expanded into a convolution matrix P, where the shape of matrix P is (n, (a-b+1)). 2 ), denoted as P(n,m);

[0081] (3) Figure 6 As shown, downsampling the original feature map is equivalent to calculating AP = B, where A is the flattened row vector of the original feature map and B is the flattened row vector of the new feature map, with a shape of (1, m).

[0082] (4) Due to the special nature of convolution operation, when the edges of the original image features are not numerically filled, n≥m;

[0083] Where 'a' represents the length and width dimensions of the original feature map; 'b' represents the length and width dimensions of the convolution kernel; and 'n' represents the number of information points in the original feature map, i.e., a0. 2; m represents: after the convolution kernel is expanded into a convolution matrix, the column number of the convolution matrix, that is, (a-b+1) 2 The numerical value physically represents the number of information points in the new feature map.

[0084] In the embodiment of the application, as an optional embodiment, the step 3 repeats the step 2 k times to obtain k expressions, wherein the calculation method of the value of k is as follows:

[0085] (1) k starts from 1, and the value of k*m is calculated;

[0086] (2) the value of k*m is compared with n;

[0087] (3) if n>k*m, the k is operated by 1 and step (1) is returned again;

[0088] (4) if n≤k*m, the value of k is obtained, and q=k*m.

[0089] Wherein, q represents: after the k convolution matrices are spliced, the column number of the new matrix obtained, that is, k*m, and the numerical value physically represents the total number of information points in all new feature maps.

[0090] The row vector A is subjected to k-step convolution to obtain k expressions:

[0091] AP1=B1

[0092] AP2=B2

[0093] …

[0094] AP k =B k ;

[0095] Wherein, P1, P2, …, P k are the corresponding convolution matrices in the 1-kth convolution operation; B1, B2, …, B k are the corresponding row vectors of the flattened new feature maps in the 1-kth convolution operation.

[0096] In the embodiment of the application, as an optional embodiment, the step 4 splices the k expressions, as shown in Figure 7 , Figure 8 to obtain:

[0097] A[P1 P2…P k ]=[B1 B2…B k ], denoted as A(1,n)Q(n,q)=C(1,q), wherein C represents: the vector of the flattened new feature map obtained in this round of calculation.

[0098] As an optional embodiment in the embodiment of the present application, the step 5 reduces the initial feature map by the generalized inverse matrix, comprising the following steps:

[0099] (1) transpose both sides of the expression A(1,n)Q(n,q)=C(1,q) to obtain Q T (q,n)A T (n,1)=C T (q,1);

[0100] (2) the matrix Q is obtained by splicing k-step convolution matrices, so that QQ T is reversible;

[0101] (3) according to the knowledge of the generalized inverse matrix, the unique solution A=CQ T is obtained by Q and C; T -T .

[0102] Wherein, the proof process of the generalized inverse matrix is as follows:

[0103] (1) the original convolution calculation method is Q T (q,n)A T (n,1)=C T (q,1);

[0104] (2) minimize the distance between two vectors, that is, let J=min||Q T A T -C T ||;

[0105] (3) the to-be-solved vector is A T , so let

[0106] (4) solve to obtain QQ T A T =QC T ;

[0107] (5) since each column of the convolution matrix Q is not all 0, so that QQ T is reversible;

[0108] (6) that is, the unique solution A=CQ T (QQ T ) -T can be obtained by Q and C.

[0109] Wherein, J represents: divergence, used to describe the difference between the vector Q T A T and the vector C T , so as to perform subsequent calculation according to the derivative of J. DETAILED DESCRIPTION​

[0111] Take the original feature map size as 5*5, the convolution kernel size as 3*3, and the step as 2 for example, and the new feature map size obtained is 2*2.

[0112] (1) The original feature map is as shown in Figure 2 , the convolution kernel is as shown in Figure 3 , and n = 25 at this time.

[0113] (2) The convolution kernel is extended as shown in Figure 4 to the same size as the original feature map, which facilitates subsequent calculation.

[0114] (3) As shown in Figure 5 , the extended convolution kernel is flattened to complete the construction of the convolution matrix P1, and m = 4 at this time.

[0115] (4) As shown in Figure 6 , the extended A is multiplied by the convolution matrix P1 to obtain the extended B; where A is the row vector after the original feature map is flattened, and B is the row vector after the new feature map is flattened.

[0116] (5) The value of k is calculated, and k = 7.

[0117] (6) Steps (2) to (4) are repeated six times, as shown in Figure 7 , to obtain the convolution kernel splicing matrix Q and the splicing matrix C of the flattened new feature map, and q = 28 at this time.

[0118] (7) According to the knowledge of generalized inverse matrix, the unique solution A = CQ T is obtained through Q and C. T -T

[0119] In this embodiment, in the process of applying generalized convolution to the neural network, P1, P2, …, P k represent k different convolution kernels, and after the feature map is subjected to k different convolution operations, B1, B2, …, B k These different new feature maps are spliced in the depth direction to form a feature layer with a depth of k. The traditional convolution operation takes the spliced feature layer B1, B2, …, B k as the starting point to perform the next round of convolution operation. However, in the generalized convolution, the method splices B1, B2, …, B k after being flattened respectively to form a single one-dimensional row vector, and then uses the knowledge of generalized inverse matrix to recover the data during upsampling.

[0120] Application scenarios

[0121] ​The generalized convolution as an upsampling method instead of the transpose convolution can be directly deployed in any segmentation network to replace the feature recovery method in the original network.

[0122] Taking the application of the generalized convolution to the UNet neural network as an example:

[0123] 1. First, according to the description in step 2 above, change the max-pooling downsampling to a convolution kernel with a stride distance equal to the window size of the convolution kernel, so as to ensure that no information is directly discarded in the downsampling process.

[0124] 2. Build a neural network as shown in Figure 9

[0125] (1) Each box has undergone two traditional convolutions to achieve feature extraction;

[0126] (2) The four left-side downsampling processes all use the stride convolution defined in step 2;

[0127] (3) The two orange feature layers use the point-by-point convolution method to achieve individual pixel classification result prediction;

[0128] (4) Use the convolution kernel of the stride convolution in the downsampling process and the generated feature map to construct a generalized inverse matrix, and in each calculation, the same group of stride convolution, feature map and generalized inverse matrix are represented by a dashed box and a dashed arrow in Figure 9 , wherein:

[0129] The downward arrow in the left dashed box represents the convolution matrix in the downsampling stage,

[0130] The square in the left dashed box represents the feature map obtained after downsampling,

[0131] The upward arrow in the right dashed box represents the convolution matrix of the generalized convolution in the upsampling stage,

[0132] The right arrow in the middle part represents that the generalized convolution calculates the content of the right dashed box from the content of the left dashed box;

[0133] (5) In the right-side upsampling process, the generalized inverse matrix is used to perform upsampling calculation on the feature layer.

[0134] 3. The feature map in the upper right corner is the output obtained, that is, the segmentation result obtained after replacing the transpose convolution with the generalized convolution.

[0135] 4. Through the loss calculation of the segmentation result and the true label, update the convolution parameters in the downsampling stage according to the backpropagation algorithm, and at this time, the parameters in the upsampling stage will be adaptively changed in the next round of training.

[0136] ​5. Multiple training-validation training steps are performed until the output of the network is stable and reasonable, and the training is considered to be completed.

[0137] In the process of training the network, due to the existence of stride convolution, the historical information can be preserved when downsampling, and the existence of generalized convolution makes it possible to restore the historical information.

[0138] In fact, when applying generalized convolution to the neural network, the stride convolution of the downsampling can be an equivalent pooling layer with a stride distance equal to the size of the convolution kernel window, or a traditional convolution layer with a stride distance smaller than the size of the convolution kernel window, as long as the details of the previous layer are preserved in the new feature map. The details can be restored through generalized convolution.

[0139] On the other hand, since the convolution kernel of generalized convolution comes from the convolution kernel and feature layer during downsampling, only the downsampling process needs to be trained when applying it, which can greatly reduce the parameter amount of the network; and due to the symmetric relationship between upsampling and downsampling, the parameters on both sides can be frozen, and only the prediction part of the deepest network is fine-tuned to realize transfer learning.

[0140] Figure 10 The effect diagram of the image semantic segmentation method based on generalized convolution according to the embodiment is shown, which respectively performs semantic segmentation on images of portraits, landscapes and buildings. The first column is the original image, the second column is the image after downsampling, and the third column is the comparison result of bilinear interpolation upsampling and generalized convolution upsampling. As can be seen from the comparison result, the scheme according to the embodiment can better recover the details lost in the downsampling through generalized convolution, and at the same time realize the recovery and size reduction of image information, realize accurate upsampling, and the effect is better than that of the bilinear interpolation upsampling method.

[0141] Embodiment two

[0142] The embodiment two of the application provides an image semantic segmentation system based on generalized convolution.

[0143] An image semantic segmentation system based on generalized convolution comprises:

[0144] A feature extraction module extracts initial feature maps of an input image;

[0145] A downsampling module performs a downsampling operation on the initial feature maps;

[0146] A convolution module performs a convolution operation on the downsampling process for k times to obtain k expressions;

[0147] A splicing module splices the k expressions to obtain a splicing matrix;

[0148] The reduction module reduces the splicing matrix by a generalized inverse matrix to obtain an initial feature map.

[0149] Embodiment three

[0150] Embodiment three of the present application provides a computer device, comprising: a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the image semantic segmentation method based on generalized convolution as described in any possible implementation manner of the first aspect.

[0151] Embodiment four

[0152] Embodiment four of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to perform the steps of the image semantic segmentation method based on generalized convolution as described in any possible implementation manner of the first aspect.

[0153] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An image semantic segmentation method based on generalized convolution, characterized in that, include: Step 1: Extract the initial feature map of the input image; Step 2: Perform a downsampling operation on the initial feature map; Step 3: Repeat the convolution operation from step 2 k times to obtain k expressions; Step 4: Concatenate the k expressions to obtain the concatenation matrix; Step 5: Obtain the initial feature map by restoring the concatenated matrix using the generalized inverse matrix; Where k is the number of convolution operations; The downsampling operation on the feature layer using strided convolution includes the following steps: (1) Flatten the feature map of the original size (a,a) to obtain a row vector A(1*n) of length n=a²; (2) Expand the convolution kernel of size (b,b) into a convolution matrix P, with shape (n,(a-b+1)²), denoted as P(n,m); (3) Calculate AP=B for the original feature map, where A is the row vector of the original feature map after flattening and B is the row vector of the new feature map after flattening. (4) Due to the special nature of convolution operations, n ≥ m when no padding is added; Where a represents the size of the original feature map; b represents the size of the convolution kernel; and n represents the number of information points in the original feature map, i.e., a 2 ; m represents the number of columns in the convolution matrix after the convolution kernel is expanded into a convolution matrix; The initial feature map is obtained by restoring the concatenated matrix using the generalized inverse matrix, including the following steps: (1) Transpose both sides of the expression A(1,n)Q(n,q)=C(1,q) to get Q T (q,n)A T (n,1)=C T (q,1); (2) Matrix Q is obtained by concatenating k-step convolution matrices, therefore QQ T reversible; (3) Based on the knowledge of generalized inverse matrices, the unique solution A=CQ can be obtained through Q and C. T (QQ T ) -T ; Where q is the number of columns of the new matrix obtained by concatenating k convolutional matrices, represents the total number of information points in all new feature maps, Q represents the matrix obtained by concatenating k step convolutional matrices, and C represents the flattened vector of all new feature maps obtained in this round of calculation.

2. The image semantic segmentation method based on generalized convolution as described in claim 1, characterized in that, The method for calculating the value of k is as follows: (1) Starting from 1, calculate the value of k*m; (2) Compare the value of k*m with n; (3) If n > k*m, increment k by 1 and return to step (1); (4) If n≤k*m, obtain the required value of k and let q=k*m.

3. The image semantic segmentation method based on generalized convolution as described in claim 1, characterized in that, The step 2 is repeated k times to obtain k expressions, which are: performing k stride convolutions on row vector A to obtain k expressions: AP1=B1 AP2=B2 … AP k =B k Among them, P1, P2, ..., P k Let B1, B2, ..., B be the convolution matrices corresponding to the (1-k)th convolution operations. k This is the row vector flattened from the new feature map corresponding to the (1-k)th convolution operation.

4. The image semantic segmentation method based on generalized convolution as described in claim 1, characterized in that, By concatenating k expressions, we obtain: A[P1 P2 … P k ]=[B1 B2 …B k ], denoted as A(1,n)Q(n,q)=C(1,q).

5. An image semantic segmentation system based on generalized convolution, characterized in that, include: Feature extraction module: Extracts the initial feature map of the input image; The downsampling module performs downsampling operations on the initial feature map; The convolution module repeats the convolution operation k times during the downsampling process to obtain k expressions; The concatenation module concatenates k expressions to obtain a concatenation matrix; The reconstruction module uses a generalized inverse matrix to reconstruct the concatenated matrix to obtain the initial feature map. Where k is the number of convolution operations; The downsampling operation on the feature layer using strided convolution includes the following steps: (1) Flatten the feature map of the original size (a,a) to obtain a row vector A(1*n) of length n=a²; (2) Expand the convolution kernel of size (b,b) into a convolution matrix P, with shape (n,(a-b+1)²), denoted as P(n,m); (3) Calculate AP=B for the original feature map, where A is the row vector of the original feature map after flattening and B is the row vector of the new feature map after flattening. (4) Due to the special nature of convolution operations, n ≥ m when no padding is added; Where a represents the size of the original feature map; b represents the size of the convolution kernel; and n represents the number of information points in the original feature map, i.e., a 2 ; m represents the number of columns in the convolution matrix after the convolution kernel is expanded into a convolution matrix; The initial feature map is obtained by restoring the concatenated matrix using the generalized inverse matrix, including the following steps: (1) Transpose both sides of the expression A(1,n)Q(n,q)=C(1,q) to get Q T (q,n)A T (n,1)=C T (q,1); (2) Matrix Q is obtained by concatenating k-step convolution matrices, therefore QQ T reversible; (3) Based on the knowledge of generalized inverse matrices, the unique solution A=CQ can be obtained through Q and C. T (QQ T ) -T ; Where q is the number of columns of the new matrix obtained by concatenating k convolutional matrices, represents the total number of information points in all new feature maps, Q represents the matrix obtained by concatenating k step convolutional matrices, and C represents the flattened vector of all new feature maps obtained in this round of calculation.

6. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the image semantic segmentation method based on generalized convolution as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the image semantic segmentation method based on generalized convolution as described in any one of claims 1 to 4.

Images