Image preprocessing method and device, image processing method, electronic equipment and medium

Abstract

An image preprocessing method, an image processing apparatus, an electronic device, and a computer-readable storage medium are disclosed. The image preprocessing method includes: acquiring an input image; performing preprocessing calculations on the input image to obtain a calculation result; comparing the calculation result with the input image to obtain transformation parameters; and performing preprocessing on the input image based on the transformation parameters. This method preprocesses the image based on the transformation parameters, which can omit hardware preprocessing steps, save hardware computing power, and improve the efficiency of image processing.

Description

Technical Field

[0001] Embodiments of this disclosure relate to an image preprocessing method, an image preprocessing apparatus, an image processing method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Residual neural networks (RNNs) are widely used in image classification and object recognition. They are characterized by ease of optimization and the ability to improve accuracy by increasing network depth. The ResNet architecture is one of the most successful RNN algorithms for image recognition in recent years. In RNNs, preprocessing is often required before performing convolution operations on the input image data, resulting in more efficient data processing. Summary of the Invention

[0003] At least one embodiment of this disclosure provides an image preprocessing method, comprising: acquiring an input image; performing preprocessing calculations on the input image to obtain a calculation result; comparing the calculation result with the input image to obtain transformation parameters; and performing preprocessing on the input image according to the transformation parameters.

[0004] For example, in an embodiment of the image preprocessing method provided in this disclosure, the calculation results include n first output image matrices and n second output image matrices, where n is a positive integer. The transformation parameters include a first set of transformation parameters and a second set of transformation parameters. Preprocessing the input image to obtain the calculation results includes: performing a filling operation on the input image to obtain a filled image matrix; performing a splitting operation on the data in the filled image matrix to obtain n split image matrices; performing a folding operation on the n split image matrices to obtain n first output image matrices and n second output image matrices; and comparing the calculation results with the input image to obtain the transformation parameters, including: comparing the n first output image matrices with the n split image matrices to obtain a first set of transformation parameters; and comparing the n second output image matrices with the n split image matrices to obtain a second set of transformation parameters.

[0005] For example, in an image preprocessing method provided in an embodiment of this disclosure, comparing n first output image matrices with n split image matrices to obtain a first set of transformation parameters includes: comparing the dimensions of the n first output image matrices with the dimensions of the n split image matrices to obtain a first set of transformation parameters; comparing n second output image matrices with n split image matrices to obtain a second set of transformation parameters includes: comparing the dimensions of the n second output image matrices with the n split image matrices to obtain a second set of transformation parameters.

[0006] For example, in an image preprocessing method provided in one embodiment of this disclosure, the input image is preprocessed according to transformation parameters, including: splitting the data in the input image to obtain n split data matrices, where n is a positive integer; and performing a filling and folding operation on the n split data matrices to obtain multiple output data matrices. The filling and folding operation includes a filling operation and a folding operation. Performing the filling and folding operation on the n split data matrices to obtain multiple output data matrices includes: loading the n split data matrices, performing a folding operation on the loaded n split data matrices to obtain 2n folded data matrices, and performing a filling operation on the 2n folded data matrices according to transformation parameters to obtain multiple output data matrices.

[0007] For example, in an image preprocessing method provided in an embodiment of this disclosure, each folded data matrix includes four sides, and the filling operation includes multiple sub-filling operations. A sub-filling operation is performed on a folded data matrix. The sub-filling operation performed on each folded data matrix includes at least one of the following operations: not performing any operation on at least one side of each folded data matrix, adding data of a corresponding size to at least one side of each folded data matrix, and truncating data of a corresponding size to at least one side of each folded data matrix.

[0008] For example, in an image preprocessing method provided in an embodiment of this disclosure, the data in each folded data matrix is ​​arranged in multiple rows and columns in a first direction and a second direction. The four sides of each folded data matrix include a first side and a second side that are opposite to each other in the first direction and a third side and a fourth side that are opposite to each other in the second direction.

[0009] For example, in an image preprocessing method provided in one embodiment of this disclosure, 2n folded data matrices include n first folded data matrices and n second folded data matrices. A filling operation is performed on the 2n folded data matrices to obtain multiple output data matrices. This includes: performing a first set of sub-filling operations on the n first folded data matrices to obtain n first output data matrices; and performing a second set of sub-filling operations on the n second folded data matrices to obtain n second output data matrices. The first set of sub-filling operations includes n sub-filling operations performed on the n first folded data matrices respectively, and the second set of sub-filling operations includes n sub-filling operations performed on the n second folded data matrices respectively. The multiple output data matrices include n first output data matrices and n second output data matrices.

[0010] For example, in an image preprocessing method provided in an embodiment of this disclosure, performing a first set of sub-filling operations on n first folded data matrices includes: performing a sub-filling operation in the first set of sub-filling operations on each of the n first folded data matrices to obtain a first output data matrix; performing a second set of sub-filling operations on n second folded data matrices includes: performing a sub-filling operation in the second set of sub-filling operations on each of the n second folded data matrices to obtain a second output data matrix.

[0011] For example, in an embodiment of the image preprocessing method provided in this disclosure, 2n folded data matrices include n first folded data matrices and n second folded data matrices. Each split data matrix corresponds to one first folded data matrix and one second folded data matrix. Folding operations are performed on the loaded n split data matrices to obtain 2n folded data matrices, including: for each split data matrix in the n split data matrices: adding at least one channel to each data in the split data matrix to obtain a first folded data matrix corresponding to the split data matrix; and obtaining a second folded data matrix corresponding to the split data matrix based on the first folded data matrix; the first folded data matrix and the second folded data matrix are the same.

[0012] For example, in an image preprocessing method provided in one embodiment of this disclosure, adding at least one channel to each data in a split data matrix includes: filling the data to be filled into at least one channel.

[0013] For example, in an image preprocessing method provided in one embodiment of this disclosure, the data to be filled into at least one channel is 0.

[0014] For example, in an embodiment of the image preprocessing method provided in this disclosure, the data in the input image is arranged to form multiple data rows, and the data in the input image is split to obtain n split data matrices, including: reading the odd-numbered rows of the multiple data rows to obtain a first intermediate data matrix, the data in the first intermediate data matrix is ​​arranged to form multiple first intermediate data columns, each first intermediate data column including one data from each odd-numbered row of the multiple data rows; reading the even-numbered rows of the multiple data rows to obtain a second intermediate data matrix, the data in the second intermediate data matrix is ​​arranged to form multiple second intermediate data columns, each second intermediate data column including one data from each even-numbered row of the multiple data rows; reading the odd-numbered columns of the multiple first intermediate data columns to obtain a first split data matrix, reading the even-numbered columns of the multiple first intermediate data columns to obtain a second split data matrix, reading the odd-numbered columns of the multiple second intermediate data columns to obtain a third split data matrix, and reading the even-numbered columns of the multiple second intermediate data columns to obtain a fourth split data matrix, the n split data matrices including the first split data matrix, the second split data matrix, the third split data matrix, and the fourth split data matrix.

[0015] For example, an image preprocessing method provided in one embodiment of this disclosure further includes: normalizing the data in the input image before performing a splitting operation on the data in the input image.

[0016] At least one embodiment of this disclosure also provides an image processing method, including: acquiring an input image; processing data in the input image using an image preprocessing method provided in at least one embodiment of this disclosure to obtain multiple output data matrices corresponding to the input image; and processing the multiple output data matrices using a neural network to obtain the output of the neural network.

[0017] For example, in an image processing method provided in one embodiment of this disclosure, the neural network is a 50-layer residual neural network.

[0018] At least one embodiment of this disclosure also provides an image preprocessing apparatus, comprising: an acquisition unit configured to acquire an input image; a preprocessing calculation unit configured to perform preprocessing calculations on the input image to obtain a calculation result; a comparison unit configured to compare the calculation result with the input image to obtain transformation parameters; and a preprocessing unit configured to perform preprocessing on the input image according to the transformation parameters.

[0019] At least one embodiment of this disclosure also provides an electronic device, including: a processor; and a memory storing computer-executable instructions, which, when executed by the processor, implement the image preprocessing method or the image processing method provided in at least one embodiment of this disclosure.

[0020] At least one embodiment of this disclosure also provides a computer-readable storage medium for non-transitory storage of computer-executable instructions that, when executed by a processor, implement the image preprocessing method or the image processing method provided in at least one embodiment of this disclosure. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0022] Figure 1 A schematic flowchart of an image preprocessing method provided in at least one embodiment of the present disclosure is shown;

[0023] Figure 2 A flowchart illustrating an example of the preprocessing calculation steps provided in an embodiment of this disclosure is shown;

[0024] Figure 3 A schematic diagram illustrating an example of fill and split operations in preprocessing computation is shown;

[0025] Figure 4 A schematic flowchart illustrating the steps of splitting the padded image matrix is ​​shown;

[0026] Figure 5 A schematic diagram illustrating an example of a folding operation in preprocessing computation is shown;

[0027] Figure 6 It shows Figure 1 A schematic flowchart of step S140 in the image preprocessing method shown;

[0028] Figure 7 It shows Figure 6 A schematic diagram of an example of step S601 in the image preprocessing method shown;

[0029] Figure 8 It shows Figure 6 A schematic flowchart of step S602 in the image preprocessing method shown;

[0030] Figure 9 A schematic diagram illustrating an example of filling a first folded data matrix and a second folded data matrix according to transformation parameters is shown.

[0031] Figure 10 It shows Figure 8 A schematic flowchart of step S803 in the image preprocessing method shown;

[0032] Figure 11A flowchart illustrating the steps of preprocessing an input image according to transformation parameters provided in at least one embodiment of this disclosure is shown;

[0033] Figure 12 A schematic flowchart of an image processing method provided in at least one embodiment of the present disclosure is shown;

[0034] Figure 13 A schematic block diagram of an image preprocessing apparatus provided in at least one embodiment of the present disclosure is shown;

[0035] Figure 14 A schematic block diagram of an electronic device provided for some embodiments of this disclosure;

[0036] Figure 15 A schematic block diagram of another electronic device provided for some embodiments of this disclosure; and

[0037] Figure 16 This is a schematic diagram of a computer-readable storage medium provided for some embodiments of this disclosure. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0039] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0040] Figure 1A schematic flowchart of an image preprocessing method provided in at least one embodiment of the present disclosure is shown.

[0041] like Figure 1 As shown, the image preprocessing method includes the following steps S110-S130.

[0042] Step S110: Obtain the input image.

[0043] Step S120: Perform preprocessing calculations on the input image to obtain the calculation results.

[0044] Step S130: Compare the calculation results with the input image to obtain the transformation parameters.

[0045] Step S140: Preprocess the input image according to the transformation parameters.

[0046] For example, the size of the input image is n*c*h*w, where n is the number of input images, c is the number of channels in the input image, h is the height of the input image, and w is the width of the input image. For example, the size of the input image could be 2*3*224*224, meaning the input image data is arranged in a 2*3*224*224 matrix.

[0047] Preprocessing the input image involves calculating potential changes before the convolution operation. This preprocessing is typically considered from two perspectives: the neural network algorithm itself and the combination of hardware and software. For example, with an input data size of 2*3*224*224 and a hardware channel granularity of 8 (meaning the hardware processes at least 8 channels at a time), performing a convolution operation with a 3*3 kernel and a stride of 2 requires not only preprocessing the data to improve computational efficiency but also considering the combination of hardware and software to ensure the preprocessed data meets the hardware's computational requirements.

[0048] For example, the calculation result includes n first output image matrices and n second output image matrices, where n is a positive integer. In some embodiments of this disclosure, step S120 may include: performing a filling operation on the input image to obtain a filled image matrix; performing a splitting operation on the data in the filled image matrix to obtain n split image matrices; and performing a folding operation on the n split image matrices to obtain n first output image matrices and n second output image matrices.

[0049] Figure 2 A flowchart illustrating an example of the preprocessing calculation steps provided in an embodiment of this disclosure is shown.

[0050] like Figure 2As shown, taking a 50-layer residual neural network (ResNet50) as an example, the preprocessing computation generally follows this order: padding operation – splitting operation – folding operation. After this preprocessing computation, n first output image matrices and n second output image matrices suitable for convolution operations are obtained, where n is a positive integer. Before the convolution operation, the input image matrix is ​​usually padded to ensure that the convolution computation retains as much information as possible from the input image. Performing the splitting operation after padding allows the subsequent convolution operation to retain more effective data, saving hardware computing power. The folding operation aligns the number of channels of the image data with the hardware, resulting in higher processing efficiency.

[0051] Preprocessing calculations can be performed by computer or manually; this disclosure does not restrict the method of preprocessing calculations.

[0052] It should be noted that, Figure 2 This is merely an example of preprocessing computation. The steps involved in preprocessing computation can be determined based on factors such as the type of neural network, the size of the input image, the size of the convolutional kernel, and the number of hardware channels. This disclosure does not impose any limitations on these steps. For example, only padding and splitting operations can be performed on the image without folding operations, or only padding and folding operations can be performed without splitting operations. Alternatively, other operations besides padding, splitting, and folding operations, such as normalization, can be performed.

[0053] The normalization operation involves multiplying each component of the data in the input image by its corresponding scale and adding the corresponding bias. The values ​​of scale and bias are obtained through iterative calculations.

[0054] For example, the image format of the input image data can be RGB, where R represents the red component, G represents the green component, and B represents the blue component. Each data point in the input image includes a red component, a green component, and a blue component. The scale corresponding to the red component is scale_R, and the corresponding bias is bias_R; the scale corresponding to the green component is scale_G, and the corresponding bias is bias_G; the scale corresponding to the blue component is scale_B, and the corresponding bias is bias_B. Then, the normalized value of the red component is expressed as normalized_R = R0 * scale_R + bias_R, the normalized value of the green component is expressed as normalized_G = G0 * scale_G + bias_G, and the normalized value of the blue component is expressed as normalized_B = B0 * scale_B + bias_B, where R0 represents the initial value of the red component, G0 represents the initial value of the green component, and B0 represents the initial value of the blue component.

[0055] For example, the proportions of the red components in all data in the input image are the same, and the deviations of the red components in all data in the input image are also the same; similarly, the proportions of the green components in all data in the input image are the same, and the deviations of the green components in all data in the input image are also the same, and the proportions of the blue components in all data in the input image are the same, and the deviations of the blue components in all data in the input image are also the same.

[0056] Below, in conjunction with Figure 3 Explain the process of filling and splitting operations in the preprocessing calculation.

[0057] The input image has dimensions of 2*3*224*224, meaning it has two channels, each with a width and height of 224 pixels. The data arrangement in the input image forms matrix 101, with a size of 3*224*224 for a single input image data matrix 101 (the number of channels is not shown). First, a padding operation is performed on the input image matrix 101. In subsequent convolution operations using a 3*3 convolution kernel with a stride of 2, the padding can be performed by padding the left and top edges of matrix 101 with 3 pixels each, and the bottom and right edges with 2 pixels each, resulting in a padded image matrix 102. The padding data can be 0 or other values; this disclosure does not impose any restrictions. The padded image matrix 102 obtained after the padding operation has dimensions of 3*229*229.

[0058] For example, the data in the image matrix 102 is arranged to form multiple data rows.

[0059] After obtaining the padded image matrix 102, it can be split to improve the efficiency of subsequent convolution operations. The padded image matrix 102 can be split into multiple split image matrices. For example, as... Figure 3 As shown, for the data in matrices 101 and 102, the data to be split into matrix Split_ee1 is marked as "ee", the data to be split into matrix Split_eo1 is marked as "eo", the data to be split into matrix Split_oe1 is marked as "oe", and the data to be split into matrix Split_oo1 is marked as "oo". For the padding data in matrix 102, the padding pixels to be split into matrix Split_ee1 are marked as "1", the padding pixels to be split into matrix Split_eo1 are marked as "2", the padding pixels to be split into matrix Split_oe1 are marked as "3", and the padding pixels to be split into matrix Split_oo1 are marked as "4".

[0060] The splitting operation steps for the filled image matrix 102 can be combined with Figure 4 To illustrate, the splitting operation may include the following steps S401 to S406.

[0061] Step S401: Read the odd-numbered rows from multiple data rows to obtain the first intermediate image matrix.

[0062] Step S402: Read the even-numbered rows from multiple data rows to obtain the second intermediate image matrix.

[0063] Step S403: Read the odd columns from multiple first intermediate data columns to obtain the first split image matrix.

[0064] Step S404: Read the even-numbered columns from multiple first intermediate data columns to obtain the second split image matrix.

[0065] Step S405: Read the odd columns from multiple second intermediate data columns to obtain the third split image matrix.

[0066] Step S406: Read the even-numbered columns from multiple second intermediate data columns to obtain the fourth split image matrix.

[0067] For example, in some embodiments, the size of the filling image matrix is ​​H*W. When both H and W are even, the sizes of the first, second, third, and fourth split image matrices are all the same, and all are H / 2*W / 2. When H is even and W is odd, the sizes of the first and third split image matrices are the same, and both are H / 2*((W+1) / 2), and the sizes of the second and fourth split image matrices are the same, and both are H / 2*((W-1) / 2). When H is odd and W is even, the size of the first split image matrix... The size of the first and second split image matrices is the same, and both are ((H+1) / 2)*W / 2. The size of the third and fourth split image matrices is the same, and both are ((H-1) / 2)*W / 2. When H and W are both odd, the size of the first split image matrix is ​​((H+1) / 2)*((W+1) / 2), the size of the second split image matrix is ​​((H+1) / 2)*((W-1) / 2), the size of the third split image matrix is ​​((H-1) / 2)*((W+1) / 2), and the size of the fourth split image matrix is ​​((H-1) / 2)*((W-1) / 2).

[0068] Therefore, with the size of the filled image matrix being 2*3*229*229, the first split image matrix is ​​denoted as Split_ee1, with a size of 2*3*115*115. The second split image matrix is ​​denoted as Split_eo1, with a size of 2*3*115*114. The third split image matrix is ​​denoted as Split_oe1, with a size of 2*3*114*115. The fourth split image matrix is ​​denoted as Split_oo1, with a size of 2*3*114*114.

[0069] It should be noted that after the input image is split, the convolution kernel also needs to be processed accordingly. Since this is not the subject of this invention, it will not be elaborated here.

[0070] Below, in conjunction with Figure 5 This section describes the folding operation process in the preprocessing calculation of the input image.

[0071] After the splitting operation, the split image matrix can be folded according to the hardware characteristics. For example, in a residual neural network, the input image can be an RGB image with 3 channels. If the hardware computing unit has an 8-channel computational granularity, meaning the hardware processes 8 channels of data at a time, the image matrix can be folded along the channel dimension to align the number of channels in the image matrix with the number of channels in the hardware.

[0072] In some embodiments, if the image format of the data in the input image is not RGB, a format conversion operation can be performed on the data in the input image before the normalization operation. For example, if the image format of the data in the input image is YUV, a format conversion operation can be performed on the data in the input image first to make the image format of the data in the input image become RGB, and then subsequent operations can be performed on the RGB format data in the input image.

[0073] For example, if the three channels of a split image matrix are denoted as channels 0-2, we can first add a channel with all data set to 0, calling it channel 3. Taking Split_ee1 as an example, after adding this channel, the size becomes 2*4*115*115. While ensuring the equivalence of subsequent convolution operations, we can perform a folding operation on the split image matrix to increase the number of channels and decrease the width or height, thus meeting the hardware processing requirements. That is, the folding operation represents the process of transforming k channels of the split image matrix into 2k channels. The data of the original k channels and the newly added k channels are the same. However, to ensure the equivalence of convolution operations, the data of different channels needs to be truncated in different ways. Therefore, the data of the first k channels and the last k channels of the final output image matrix that can be directly used for convolution operations by the hardware are not completely identical.

[0074] The folding operation can be performed horizontally or vertically. After folding, each split image matrix yields its corresponding folded image matrix. The first split image matrix, after folding, yields the first folded image matrix Fold_ee; the second split image matrix, after folding, yields the second folded image matrix Fold_eo; the third split image matrix, after folding, yields the third folded image matrix Fold_oe; and the fourth split image matrix, after folding, yields the fourth folded image matrix Fold_oo.

[0075] Depend on Figure 5As can be seen, after the first split image matrix Split_ee1 is folded horizontally, to ensure the equivalence of convolution operations before and after folding, the last two columns of data for channel 0 of the first folded image matrix Fold_ee are truncated. The data for channel 4 of Fold_ee is obtained by truncating the last two columns of data from channel 4 of Split_ee1, and then stacking them with channels 0-3 of Fold_ee (the gray area represents the truncated portion). That is, channel 0 is folded to produce channel 4, channel 1 is folded to produce channel 5, and so on. It should be noted that only channels 0 and 4 are shown in the figure; the folding operations for channels 1-3 are the same as for channel 0, and the formation process for channels 5-7 is the same as for channel 4. In other words, after horizontal folding, the last two columns of channels 0-3 are truncated, and the first two columns of channels 4-7 are truncated. This transforms a 3-channel matrix into an 8-channel matrix that can be directly processed by the hardware. It is important to note that the folding operation requires a corresponding folding operation on the convolution kernel in subsequent processing to ensure the equivalence of convolution operations before and after the operation. Since the processing of convolution kernels is not the subject of this invention, it will not be described in detail here.

[0076] For example, the transformation parameters include a first set of transformation parameters and a second set of transformation parameters. In some embodiments of this disclosure, step S130 may include: comparing n first output image matrices with the n split image matrices to obtain a first set of transformation parameters; and comparing n second output image matrices with the n split image matrices to obtain a second set of transformation parameters.

[0077] For example, in some embodiments of this disclosure, comparing n first output image matrices with the n split image matrices to obtain a first set of transformation parameters may include: comparing the dimensions of the n first output image matrices with the dimensions of the n split image matrices to obtain a first set of transformation parameters; comparing n second output image matrices with the n split image matrices to obtain a second set of transformation parameters may include: comparing the dimensions of the n second output image matrices with the n split image matrices to obtain a second set of transformation parameters.

[0078] For example, a padding operation is performed on the input image to obtain a padding data matrix. The data in this padding data matrix is ​​then split to obtain four split image matrices. These four split image matrices are then folded to obtain four first output image matrices and four second output image matrices. Each of the four split image matrices corresponds one-to-one with both the four first and four second output image matrices. Each of the four first output image matrices is compared with its corresponding split image matrix to obtain a first set of transformation parameters. Similarly, each of the four second output image matrices is compared with its corresponding split image matrix to obtain a second set of transformation parameters. For instance, the matrix Fold_ee is compared with the split data matrix Split_ee1 to obtain the transformation parameters.

[0079] If the preprocessing calculations described above are performed directly in hardware, the hardware computation and storage relationships are complex, and the data from channels 4-7 depends on the prior processing of data from channels 0-3. Therefore, the entire process involves a large amount of computation and has low parallelism. Therefore, it is advisable to first calculate the transformation parameters using preprocessing, and then preprocess the input image based on these parameters.

[0080] Figure 6 A schematic flowchart of step S140 provided in at least one embodiment of the present disclosure is shown.

[0081] like Figure 6 As shown, step S140 includes the following steps S601 to S602.

[0082] Step S601: Perform a splitting operation on the data in the input image to obtain n split data matrices.

[0083] For example, the data in the input image is arranged to form multiple rows of data.

[0084] For example, n is a positive integer, and in some embodiments, n can be 4.

[0085] For example, in some embodiments, step S601 may include steps S701 to S706.

[0086] Step S701: Read the odd-numbered rows from multiple data rows to obtain the first intermediate data matrix.

[0087] Step S702: Read the even-numbered rows from multiple data rows to obtain the second intermediate data matrix.

[0088] Step S703: Read the odd columns from multiple first intermediate data columns to obtain the first split data matrix.

[0089] Step S704: Read the even-numbered columns from multiple first intermediate data columns to obtain the second split data matrix.

[0090] Step S705: Read the odd columns from multiple second intermediate data columns to obtain the third split data matrix.

[0091] Step S706: Read the even-numbered columns from multiple second intermediate data columns to obtain the fourth split data matrix.

[0092] Step S602: Perform a filling and folding operation on the n split data matrices to obtain multiple output data matrices.

[0093] Figure 7 A schematic diagram of an example of step S601 is shown.

[0094] like Figure 7As shown, the input image has dimensions of 2*3*224*224. The data arrangement in the input image forms a matrix 700. After a splitting operation, matrix 700 is split into four split data matrices: split_ee2, split_eo2, split_oe2, and split_oo2. Figure 7 The text omits some data from each split data matrix, although... Figure 7 Although the matrix sizes shown are different, as can be seen from steps S701 to S706, the sizes of each split data matrix are the same. Specifically, the size of the split data matrix split_ee2 is 2*3*112*112, the size of the split data matrix split_eo2 is 2*3*112*112, the size of the split data matrix split_oe2 is 2*3*112*112, and the size of the split data matrix split_oo2 is 2*3*112*112.

[0095] For example, such as Figure 7 As shown, for the data in matrix 700, the data split into matrix Split_ee2 is labeled "ee", the data split into matrix Split_eo2 is labeled "eo", the data split into matrix Split_oe2 is labeled "oe", and the data split into matrix Split_oo2 is labeled "oo".

[0096] Then, fill and fold operations can be performed on the n split data matrices. For example, fill and fold operations include fill operations and fold operations. A fill operation means filling or truncating at least one side of the matrix to a certain size, or leaving it unchanged, so that the shape of the filled matrix meets the requirements. A fold operation means transforming the k channels of the split data matrix into 2k channels. In the embodiments of this disclosure, by combining fill and fold operations, both operations can be implemented by loading the split data matrix only once, effectively reducing the number of data accesses and increasing data reuse rate.

[0097] like Figure 8 As shown, in some embodiments, step S602 may include steps S801 to S803.

[0098] Step S801: Load n split data matrices.

[0099] Step S802: Perform a folding operation on the loaded n split data matrices to obtain 2n folded data matrices.

[0100] The 2n folded data matrices include n first-folded data matrices and n second-folded data matrices. Each split data matrix corresponds to one first-folded data matrix and one second-folded data matrix. The four channels of the first-folded data matrix are represented as channels 0 to 3, and the four channels of the second-folded data matrix are represented as channels 4 to 7. The folding operation here is the same as the folding operation in the preprocessing calculation. For example, to transform a 3-channel matrix into an 8-channel matrix, a layer of channels with all data set to 0 needs to be added to the 3-channel matrix before folding the channels.

[0101] For example, the split data matrix Split_ee2 corresponds to the first folded data matrix Fold_ee1 and the second folded data matrix Fold_ee2; the split data matrix Split_eo2 corresponds to the first folded data matrix Fold_eo1 and the second folded data matrix Fold_eo2; the split data matrix Split_oe2 corresponds to the first folded data matrix Fold_oe1 and the second folded data matrix Fold_oe2; and the split data matrix Split_oo2 corresponds to the first folded data matrix Fold_oo1 and the second folded data matrix Fold_oo2.

[0102] For example, in some embodiments, step S802 may include: for each of the n split data matrices: adding at least one channel to each data in the split data matrix to obtain a first folded data matrix corresponding to the split data matrix; and obtaining a second folded data matrix corresponding to the split data matrix based on the first folded data matrix. For example, the first folded data matrix and the second folded data matrix are the same. It should be noted that, in the embodiments of this disclosure, the "folding operation" includes the operation of adding at least one channel to each data and the operation of obtaining the second folded data matrix based on the first folded data matrix.

[0103] For example, in some embodiments, adding at least one channel to each data in a split data matrix may include filling the data to be filled into at least one channel.

[0104] For example, in some embodiments, the data to be filled into at least one channel is 0. It should be noted that the data to be filled can be set according to actual needs, and the embodiments of this disclosure do not impose specific limitations on the value of the data to be filled.

[0105] Step S803: Fill the 2n folded data matrices according to the transformation parameters to obtain multiple output data matrices.

[0106] For example, in some embodiments, each folded data matrix includes four sides, and the data in each folded data matrix is ​​arrayed in multiple rows and columns in a first direction and a second direction. The four sides of each folded data matrix include a first side and a second side opposite to each other in the first direction and a third side and a fourth side opposite to each other in the second direction.

[0107] For example, the first direction can be the row direction of the folded data matrix, and the second direction can be the column direction of the folded data matrix. For instance, the row direction of the folded data matrix is ​​parallel to the row direction of the input image, and the column direction of the folded data matrix is ​​parallel to the column direction of the input image. For instance, the first side can be the left side, and the second side can be the right side, with the first and second sides opposite to each other in the row direction of the folded data matrix. The third side can be the top side, and the fourth side can be the bottom side, with the third and fourth sides opposite to each other in the column direction of the folded data matrix.

[0108] For example, in some embodiments of this disclosure, the filling operation includes multiple sub-filling operations. A sub-filling operation is performed on a folded data matrix. The sub-filling operation performed on each folded data matrix includes at least one of the following operations: doing nothing on at least one side of each folded data matrix, adding data of a corresponding size to at least one side of each folded data matrix, and truncating data of a corresponding size to at least one side of each folded data matrix.

[0109] Table 1 shows an example of the transformation parameters used for the first folded data matrix. The data in Table 1 are the transformation parameters, and the specific method for obtaining the transformation parameters can be found in the relevant description above, which will not be repeated here. In Table 1, 0 indicates that no operation is performed on a side, positive numbers indicate that data of the corresponding size is filled on a side, and negative numbers indicate that data of the corresponding size is truncated on a side.

[0110] Table 1: An example of transformation parameters

[0111] Left side right side Top side bottom side Fold_oo1 1 0 1 1 Fold_oe1 2 -1 1 1 Fold_eo1 1 1 2 -1 Fold_ee1 2 1 2 -1

[0112] Table 2 shows an example of the transformation parameters used for the second folded data matrix. The data in Table 2 are the transformation parameters. The specific method for obtaining the transformation parameters can be found in the relevant description above, and will not be repeated here. In Table 2, 0 indicates that no operation is performed on a side, positive numbers indicate that data of the corresponding size is filled on a side, and negative numbers indicate that data of the corresponding size is truncated on a side.

[0113] Table 2: Another example of transformation parameters

[0114] Left side right side Top side bottom side Fold_oo2 -1 2 1 1 Fold_oe2 0 1 1 1 Fold_eo2 1 1 0 1 Fold_ee2 2 1 0 1

[0115] For example, in Tables 1 and 2 above, "X" means filling the corresponding side with X rows or X columns of data, and "-X" means truncating the corresponding side with X rows or X columns of data. For example, X is a positive integer, and X can be 1, 2, etc.

[0116] The following example illustrates the filling operation of the first folded data matrix Fold_ee1 and the second folded data matrix Fold_ee2 according to the transformation parameters.

[0117] like Figure 9 As shown, the transformation parameters represent the addition or truncation of data on the four sides of the matrix after the input data has been split and transformed, compared to the preprocessed output data matrix. The transformation parameters can be used to fill in the split data matrix. For the matrix Fold_ee, dashed squares marked with the number 1 represent added data, and gray shaded squares represent truncated data.

[0118] For example, such as Figure 9 As shown in Table 1, the sub-filling operations for the first folded data matrix Fold_ee1 are as follows: add 2 columns of data to its left, add 1 column of data to its right, add 2 rows of data to its top, and truncate 1 row of data to its bottom. Based on the filling operations shown in Table 2, the sub-filling operations for the second folded data matrix Fold_ee2 are as follows: add 2 columns of data to its left, add 1 column of data to its right, do not perform any operation on its top, and add 1 row of data to its bottom. It should be noted that the order of the sub-filling operations on the four sides of the first or second folded data matrix affects the number of data added to each side.

[0119] It should be noted that the transformation parameters for filling the folded data matrix are obtained through steps S120 and S130. The specific calculation process can be found in the description above and will not be repeated here.

[0120] like Figure 10 As shown, in some embodiments, step S803 may include steps S1001 to S1002.

[0121] For example, 2n folded data matrices include n first folded data matrices and n second folded data matrices.

[0122] Step S1001: Perform the first set of sub-filling operations on the n first folded data matrices to obtain n first output data matrices.

[0123] Step S1002: Perform a second set of sub-filling operations on the n second folded data matrices to obtain n second output data matrices.

[0124] For example, there is a one-to-one correspondence between n first-fold data matrices and n split data matrices, and a one-to-one correspondence between n second-fold data matrices and n split data matrices. That is, each split data matrix corresponds to one first-fold data matrix and one second-fold data matrix.

[0125] For example, the first set of sub-fill operations includes n sub-fill operations performed on n first folded data matrices respectively, the second set of sub-fill operations includes n sub-fill operations performed on n second folded data matrices respectively, and the multiple output data matrices include n first output data matrices and n second output data matrices.

[0126] For example, a fill operation includes n sub-fill operations in the first group of sub-fill operations and n sub-fill operations in the second group of sub-fill operations. In other words, a fill operation can include 2n sub-fill operations.

[0127] For example, the first set of sub-fill operations and the second set of sub-fill operations can be executed in parallel and simultaneously.

[0128] It should be noted that, depending on actual needs, at least some of the sub-filling operations in the first group of sub-filling operations can be the same, and similarly, at least some of the sub-filling operations in the second group of sub-filling operations can be the same. The embodiments of this disclosure do not impose specific limitations on the n sub-filling operations in the first group of sub-filling operations and the n sub-filling operations in the second group of sub-filling operations, as long as the size of the data matrix after filling the two folded data matrices corresponding to each split data matrix is ​​the same.

[0129] For example, in some embodiments, step S1001 may include: performing a sub-fill operation in a first set of sub-fill operations on each of the n first folded data matrices to obtain a first output data matrix; step S1002 may include: performing a sub-fill operation in a second set of sub-fill operations on each of the n second folded data matrices to obtain a second output data matrix.

[0130] Figure 11 A flowchart illustrating the steps of preprocessing an input image according to transformation parameters provided in at least one embodiment of the present disclosure is shown.

[0131] like Figure 11As shown, for an input image of size 2*3*224*224, the input image is first split. After the splitting operation, four split data matrices are obtained: sp_ee, sp_eo, sp_oe, and sp_oo. The dimensions of the split data matrix sp_ee are 2*3*112*112, the dimensions of the split data matrix sp_eo are 2*3*112*112, the dimensions of the split data matrix sp_oe are 2*3*112*112, and the dimensions of the split data matrix sp_oo are 2*3*112*112.

[0132] Next, fill and fold operations are performed on the four split data matrices sp_ee, sp_eo, sp_oe, and sp_oo to obtain eight output data matrices, namely pad_fold_ee1, pad_fold_eo1, pad_fold_oe1, pad_fold_oo1, pad_fold_ee2, pad_fold_eo2, pad_fold_oe2, and pad_fold_oo2. Among them, the size of output data matrix pad_fold_ee1 is 2*4*113*115, the size of output data matrix pad_fold_eo1 is 2*4*113*114, the size of output data matrix pad_fold_oe1 is 2*4*114*113, and the size of output data matrix pad_fold_oo1 is 2*4*114*113. The dimensions of the output data matrix pad_fold_ee2 are 2*4*113*115, the dimensions of the output data matrix pad_fold_eo2 are 2*4*113*114, the dimensions of the output data matrix pad_fold_oe2 are 2*4*114*113, and the dimensions of the output data matrix pad_fold_oo2 are 2*4*114*113.

[0133] The output data matrices pad_fold_ee1, pad_fold_eo1, pad_fold_oe1, and pad_fold_oo1 correspond to the four sub-fill operations shown in Table 1. The output data matrices pad_fold_ee2, pad_fold_eo2, pad_fold_oe2, and pad_fold_oo2 correspond to the four sub-fill operations shown in Table 2.

[0134] It should be noted that the two output data matrices corresponding to the same split data matrix have the same size. For example, for the split data matrix sp_ee, the size of the first output data matrix pad_fold_ee1 obtained by performing a padding and folding process on the split data matrix sp_ee based on a sub-padding operation in the first group of sub-padding operations is the same as the size of the second output data matrix pad_fold_ee2 obtained by performing a padding and folding process on the split data matrix sp_ee based on a sub-padding operation in the second group of sub-padding operations.

[0135] The eight output data matrices pad_fold_ee1, pad_fold_eo1, pad_fold_oe1, pad_fold_oo1, pad_fold_ee2, pad_fold_eo2, pad_fold_oe2, and pad_fold_oo2 can be input into the neural network for subsequent operations such as convolution, batch normalization, and max pooling.

[0136] It should be noted that, Figure 11 This is merely an example of preprocessing, in which the input image is split into four split data matrices, and the number of channels in the input image is increased from 3 to 8. However, this disclosure does not impose any restrictions on the number of split data matrices or the number of channels, only requiring consistency with the preprocessing calculation. For example, if the input image is split into four split data matrices in the preprocessing calculation, then the input image must also be split into four split data matrices in the preprocessing; or, for example, if the number of channels in the input image is increased from 3 to 16 in the preprocessing calculation, then the number of channels in the input image must also be increased from 3 to 16 in the preprocessing.

[0137] It should be noted that, Figure 11 The preprocessing shown employs the following steps: split operation – fold operation – fill operation. However, this disclosure does not limit the operations included in the preprocessing, only requiring them to be consistent with the operations included in the preprocessing calculation. For example, if the preprocessing calculation includes split, fold, and fill operations, the preprocessing also includes split, fold, and fill operations; conversely, if the preprocessing calculation does not include fold operations, the preprocessing also does not include fold operations. However, the order of the operations included in the preprocessing can differ from the order of the operations included in the preprocessing calculation.

[0138] Figure 12 A schematic flowchart of an image processing method provided by at least one embodiment of the present disclosure is shown.

[0139] like Figure 12 As shown, the image processing method includes the following steps S1201 to S1203.

[0140] Step S1201: Obtain the input image.

[0141] Step S1202: The data in the input image is processed using the image preprocessing method provided in at least one embodiment of this disclosure to obtain multiple output data matrices corresponding to the input image.

[0142] Step S1202 is a preprocessing operation performed on the data in the input image.

[0143] Step S1203: Use a neural network to process multiple output data matrices to obtain the output of the neural network.

[0144] For example, neural networks can be used to perform convolution and other processing on multiple output data matrices.

[0145] For example, in some embodiments, the neural network is a 50-layer residual neural network (ResNet50). The image processing method provided by the embodiments of this disclosure can be applied to the application scenarios of convolutional neural networks ResNet50 in artificial intelligence. In this image processing method, image preprocessing is achieved through an efficient hardware and software co-processing image preprocessing method provided by the embodiments of this disclosure, realizing the conversion of image data to convolutional input data, reducing the computational burden of image preprocessing, greatly reducing the computation time, increasing the flexibility of image data reading, reducing the amount of computation, and improving computational efficiency.

[0146] The technical effects of this image processing method and Figure 1 The image preprocessing methods shown have the same technical effect, and will not be described in detail here.

[0147] Figure 13 A schematic block diagram of an image preprocessing apparatus 1300 provided in at least one embodiment of the present disclosure is shown. This image preprocessing apparatus can be used to perform... Figure 1 The image preprocessing method shown.

[0148] like Figure 13 As shown, the image preprocessing apparatus 1300 includes an acquisition unit 1301, a preprocessing calculation unit 1302, a comparison unit 1303, and a preprocessing unit 1304.

[0149] The acquisition unit 1301 is configured to acquire the input image.

[0150] The preprocessing calculation unit 1302 is configured to perform preprocessing calculations on the input image to obtain the calculation result.

[0151] The comparison unit 1303 is configured to compare the calculation result with the input image to obtain the transformation parameters.

[0152] The preprocessing unit 1304 is configured to preprocess the input image according to the transformation parameters.

[0153] The technical effects of the above image preprocessing device and Figure 1 The image preprocessing methods shown have the same technical effect, and will not be described in detail here.

[0154] At least one embodiment of this disclosure also provides an electronic device, which includes a processor and a memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the image preprocessing method or image processing method provided in at least one embodiment of this disclosure.

[0155] Figure 14 This is a schematic block diagram of an electronic device 1400 provided for some embodiments of this disclosure. For example... Figure 14 As shown, the electronic device 1400 includes a processor 1410 and a memory 1420. The memory 1420 stores computer-executable instructions (e.g., one or more computer program modules). The processor 1410 executes the computer-executable instructions, which, when run by the processor 1410, can perform one or more steps of the image preprocessing method or image processing method described above. The memory 1420 and the processor 1410 can be interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0156] For example, processor 1410 may be a central processing unit (CPU), a graphics processing unit (GPU), or other form of processing unit with data processing and / or program execution capabilities. For example, the central processing unit (CPU) may be an x86 or ARM architecture. Processor 1410 may be a general-purpose processor or a special-purpose processor, capable of controlling other components in electronic device 1400 to perform desired functions.

[0157] For example, memory 1420 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and processor 1410 may run one or more computer program modules to implement various functions of electronic device 1400. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0158] It should be noted that, in the embodiments of this disclosure, the specific functions and technical effects of the electronic device 1400 can be referred to the description of the image preprocessing method and image processing method above, and will not be repeated here.

[0159] Figure 15 This is a schematic block diagram of another electronic device provided in some embodiments of this disclosure. The electronic device 1000 is, for example, suitable for implementing the image preprocessing method and / or image processing method provided in the embodiments of this disclosure. The electronic device 1000 may be a terminal device, etc. It should be noted that... Figure 15 The illustrated electronic device 1000 is merely an example and does not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0160] like Figure 15 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1010, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1020 or a program loaded from a storage device 1080 into a random access memory (RAM) 1030. The RAM 1030 also stores various programs and data required for the operation of the electronic device 1000. The processing device 1010, ROM 1020, and RAM 1030 are interconnected via a bus 1040. An input / output (I / O) interface 1050 is also connected to the bus 1040.

[0161] Typically, the following devices can be connected to the I / O interface 1050: input devices 1060 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1070 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1080 including, for example, magnetic tape, hard disk, etc.; and communication devices 1090. Communication device 1090 allows electronic device 1000 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 10 An electronic device 1000 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 1000 may alternatively implement or have more or fewer devices.

[0162] For example, according to embodiments of this disclosure, the above-described image preprocessing method or image processing method can be implemented as a computer software program. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the above-described image preprocessing method or image processing method. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1090, or installed from a storage device 1080, or installed from a ROM 1020. When the computer program is executed by the processing device 1010, the functions defined in the image preprocessing method or image processing method provided in the embodiments of this disclosure can be implemented.

[0163] At least one embodiment of this disclosure provides a computer-readable storage medium for non-temporarily storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement an image preprocessing method or image processing method provided in at least one embodiment of this disclosure.

[0164] Figure 16 This is a schematic diagram of a computer-readable storage medium provided for some embodiments of this disclosure. For example... Figure 16 As shown, the computer-readable storage medium 1100 is used to store computer-executable instructions 1110. For example, when the computer-executable instructions 1110 are executed by a computer, one or more steps in the image preprocessing method or image processing method described above can be performed.

[0165] For example, computer-readable storage medium 1100 may be a non-transitory computer-readable storage medium.

[0166] For example, the computer-readable storage medium 1100 can be applied to the aforementioned electronic device, and may include, for example, the memory 1420 in the electronic device.

[0167] For example, the description of the computer-readable storage medium 1100 can be found in the description of the memory 1420 in the embodiments of the electronic device, and will not be repeated here.

[0168] The following points need to be explained:

[0169] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0170] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0171] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. An image preprocessing method, comprising: Obtain the input image; The input image is preprocessed to obtain the calculation result; The transformation parameters are obtained by comparing the calculation results with the input image. The input image is preprocessed according to the transformation parameters. The calculation results include n first output image matrices and n second output image matrices, where n is a positive integer. The transformation parameters include a first set of transformation parameters and a second set of transformation parameters. The calculation results obtained by preprocessing the input image include: The input image is filled to obtain a filled image matrix; The data in the filled image matrix is ​​split to obtain n split image matrices; The n split image matrices are folded to obtain the n first output image matrices and the n second output image matrices; The transformation parameters are obtained by comparing the calculation result with the input image, including: The first set of transformation parameters is obtained by comparing the n first output image matrices with the n split image matrices; The second set of transformation parameters is obtained by comparing the n second output image matrices with the n split image matrices.

2. The image preprocessing method according to claim 1, wherein, The first set of transformation parameters is obtained by comparing the n first output image matrices with the n split image matrices, including: comparing the size of the n first output image matrices with the size of the n split image matrices to obtain the first set of transformation parameters; The second set of transformation parameters is obtained by comparing the n second output image matrices with the n split image matrices, including: comparing the dimensions of the n second output image matrices with the n split image matrices to obtain the second set of transformation parameters.

3. The image preprocessing method according to claim 1, wherein, Preprocessing the input image according to the transformation parameters includes: The data in the input image is split into n split data matrices, where n is a positive integer; and The n split data matrices are padded and folded to obtain multiple output data matrices. The fill-and-fold operation includes a fill operation and a fold operation. The fill-and-fold operation is performed on the n split data matrices to obtain multiple output data matrices, including: Load the n split data matrices, The folding operation is performed on the loaded n split data matrices to obtain 2n folded data matrices. The filling operation is performed on the 2n folded data matrices according to the transformation parameters to obtain the plurality of output data matrices.

4. The image preprocessing method according to claim 3, wherein, Each folded data matrix includes four sides, and the padding operation comprises multiple sub-padding operations, with one sub-padding operation being performed on a folded data matrix. The sub-filling operation performed on each folded data matrix includes at least one of the following operations: doing nothing on at least one side of each folded data matrix, adding data of a corresponding size to at least one side of each folded data matrix, and truncating data of a corresponding size to at least one side of each folded data matrix.

5. The image preprocessing method according to claim 4, wherein, The data in each folded data matrix is ​​arranged in multiple rows and columns in the first and second directions. The four sides of each folded data matrix include a first side and a second side that are opposite to each other in the first direction, and a third side and a fourth side that are opposite to each other in the second direction.

6. The image preprocessing method according to claim 4, wherein, The 2n folded data matrices include n first folded data matrices and n second folded data matrices. The filling operation is performed on the 2n folded data matrices to obtain the plurality of output data matrices, including: Perform a first set of sub-filling operations on the n first folded data matrices to obtain n first output data matrices; Perform a second set of sub-filling operations on the n second folded data matrices to obtain n second output data matrices; The first set of sub-filling operations includes n sub-filling operations performed on the n first folded data matrices respectively, the second set of sub-filling operations includes n sub-filling operations performed on the n second folded data matrices respectively, and the plurality of output data matrices includes the n first output data matrices and the n second output data matrices.

7. The image preprocessing method according to claim 6, wherein, Perform a first set of sub-filling operations on the n first folded data matrices, including: Perform a sub-fill operation in the first group of sub-fill operations on each of the n first folded data matrices to obtain a first output data matrix; Perform a second set of sub-filling operations on the n second-folded data matrices, including: Perform a sub-fill operation in the second group of sub-fill operations on each of the n second folded data matrices to obtain a second output data matrix.

8. The image preprocessing method according to claim 3, wherein, The 2n folded data matrices include n first folded data matrices and n second folded data matrices, with each split data matrix corresponding to one first folded data matrix and one second folded data matrix. The folding operation is performed on the loaded n split data matrices to obtain 2n folded data matrices, including: For each of the n split data matrices: Add at least one channel to each data in the split data matrix to obtain a first folded data matrix corresponding to the split data matrix; Based on the first folded data matrix, a second folded data matrix corresponding to the split data matrix is ​​obtained; The first folded data matrix is ​​the same as the second folded data matrix.

9. The image preprocessing method according to claim 8, wherein, Adding the at least one channel to each data point in the split data matrix includes: The data to be filled is then filled into the at least one channel.

10. The image preprocessing method according to claim 9, wherein, The data to be filled into the at least one channel is 0.

11. The image preprocessing method according to any one of claims 3-10, wherein, The data in the input image is arranged to form multiple data rows. The data in the input image is split to obtain the n split data matrices, including: Read the odd-numbered rows from the plurality of data rows to obtain a first intermediate data matrix, wherein the data in the first intermediate data matrix are arranged to form a plurality of first intermediate data columns, and each first intermediate data column includes one data from each odd-numbered row of the plurality of data rows; A second intermediate data matrix is ​​obtained by reading even-numbered rows from the plurality of data rows, wherein the data in the second intermediate data matrix is ​​arranged to form a plurality of second intermediate data columns, and each second intermediate data column includes one data from each even-numbered row of the plurality of data rows; The first split data matrix is ​​obtained by reading the odd-numbered columns from the plurality of first intermediate data columns. The second split data matrix is ​​obtained by reading the even-numbered columns from the plurality of first intermediate data columns. The third split data matrix is ​​obtained by reading the odd-numbered columns from the plurality of second intermediate data columns. The fourth split data matrix is ​​obtained by reading the even-numbered columns from the plurality of second intermediate data columns. The n split data matrices include the first split data matrix, the second split data matrix, the third split data matrix, and the fourth split data matrix.

12. The image preprocessing method according to any one of claims 1-10, further comprising: Before splitting the data in the input image, the data in the input image is normalized.

13. An image processing method, comprising: Obtain the input image; The input image is processed using the image preprocessing method according to any one of claims 1-12 to obtain a plurality of output data matrices corresponding to the input image; The multiple output data matrices are processed using a neural network to obtain the output of the neural network.

14. The image processing method according to claim 13, wherein, The neural network is a 50-layer residual neural network.

15. An image preprocessing apparatus, comprising: The acquisition unit is configured to acquire the input image; The preprocessing calculation unit is configured to perform preprocessing calculations on the input image to obtain the calculation result; The comparison unit is configured to compare the calculation result with the input image to obtain the transformation parameters; The preprocessing unit is configured to preprocess the input image according to the transformation parameters. The calculation results include n first output image matrices and n second output image matrices, where n is a positive integer. The transformation parameters include a first set of transformation parameters and a second set of transformation parameters. The calculation results obtained by preprocessing the input image include: The input image is filled to obtain a filled image matrix; The data in the filled image matrix is ​​split to obtain n split image matrices; The n split image matrices are folded to obtain the n first output image matrices and the n second output image matrices; The transformation parameters are obtained by comparing the calculation result with the input image, including: The first set of transformation parameters is obtained by comparing the n first output image matrices with the n split image matrices; The second set of transformation parameters is obtained by comparing the n second output image matrices with the n split image matrices.

16. An electronic device comprising: processor; as well as Memory, which stores computer-executable instructions. The computer-executable instructions, when executed by the processor, implement the image preprocessing method according to any one of claims 1-12 or the image processing method according to claim 13 or 14.

17. A computer-readable storage medium for non-transitory storage of computer-executable instructions, in, The computer-executable instructions, when executed by a processor, implement the image preprocessing method according to any one of claims 1-12 or the image processing method according to claim 13 or 14.

Images