Image feature extraction method, device, electronic device and storage medium

Abstract

The present application relates to the field of image processing, and provides an image feature extraction method, device, electronic device and storage medium. The method comprises: converting an image into an image matrix, dividing the image matrix to obtain image sub-blocks, dividing the weight matrix to obtain a weight vector, dividing each weight vector to obtain a weight sub-block, dividing the offset matrix to obtain an offset sub-block, multiplying the weight sub-block of the weight vector with the image sub-block and summing to obtain a feature sub-block, adding the feature sub-block to the offset sub-block to obtain an image feature sub-block, and the image feature sub-block constitutes an image feature. The present application can reduce the amount of convolution operations in a neural network during image feature extraction, thereby improving the efficiency of image feature extraction.

Description

Technical Field

[0001] The present application relates to the field of image processing, and specifically to an image feature extraction method, device, electronic device and storage medium. Background Art

[0002] Before classifying, identifying and detecting an image, it is usually necessary to extract the features of the image first. Existing image feature extraction methods mainly use the method of directly multiplying the image with a weight vector to achieve feature extraction, which makes the image feature extraction inefficient. Summary of the invention

[0003] In view of the above, it is necessary to propose an image feature extraction method, device, electronic device and storage medium to improve the efficiency of image feature extraction.

[0004] A first aspect of the present application provides an image feature extraction method, the method comprising:

[0005] Acquire an image, convert the image into an image matrix, divide the image matrix to obtain image vectors, and divide the image vectors to obtain a first number of image sub-blocks;

[0006] Obtain a weight matrix, divide the weight matrix to obtain a second number of weight vectors, and divide each of the weight vectors to obtain a first number of weight sub-blocks;

[0007] Obtaining an offset matrix, and dividing the offset matrix to obtain a second number of offset sub-blocks;

[0008] Vector multiplication is performed on a first number of image sub-blocks and a first number of weight sub-blocks of each of the weight vectors to obtain a second number of feature sub-blocks, and the second number of feature sub-blocks and the second number of offset sub-blocks are added to obtain image feature sub-blocks, wherein the image feature sub-blocks constitute image features.

[0009] Preferably, each of the image sub-block, the weight sub-block and the offset sub-block has a preset number of rows and a preset number of columns, and the preset number of columns is the same as the preset number of rows.

[0010] Preferably, dividing the image matrix to obtain an image vector comprises:

[0011] Dividing the image matrix according to the preset number of rows to obtain a preliminary image vector set, wherein the preliminary image vector set includes at least one preliminary image vector;

[0012] Determining whether there are any preliminary image vectors in the preliminary image vector set whose number of rows is less than the preset number of rows;

[0013] When there exists an image vector whose number of rows is less than the preset number of rows in the prepared image vector, the preset value is used to fill the prepared image vector whose number of rows is less than the preset number of rows to obtain a prepared image vector whose number of rows is equal to the preset number of rows, and the prepared image vector is used as the image vector; or when there does not exist an image vector whose number of rows is less than the preset number of rows in the prepared image vector, the prepared image vector is used as the image vector.

[0014] Preferably, dividing the image vector to obtain a first number of image sub-blocks comprises:

[0015] Dividing the image vector according to the preset number of columns to obtain a first number of preliminary image sub-blocks;

[0016] Determining whether there is a prepared image sub-block having a column number less than the preset column number in the prepared image sub-block;

[0017] When there are prepared image sub-blocks whose number of columns is less than the preset number of columns among the prepared image sub-blocks, preset values ​​are used to fill the prepared image sub-blocks whose number of columns is less than the preset number of columns to obtain prepared image sub-blocks whose number of columns is equal to the preset number of columns, and the first number of prepared image sub-blocks are used as the first number of image sub-blocks; or when there are no prepared image sub-blocks whose number of columns is less than the preset number of columns among the prepared image sub-blocks, the first number of prepared image sub-blocks are used as the first number of image sub-blocks.

[0018] Preferably, dividing the weight matrix to obtain a second number of weight vectors comprises:

[0019] Dividing the weight matrix according to the number of columns to obtain a second number of preliminary weight vectors;

[0020] Determining whether there is a preliminary weight vector having a column number less than the preset column number in the preliminary weight vectors;

[0021] When there are weight vectors with a number of columns less than the preset number of columns in the prepared weight vectors, the preset value is used to fill the prepared weight vectors with a number of columns less than the preset number of columns to obtain prepared weight vectors with a number of columns equal to the preset number of columns, and the second number of prepared weight vectors are used as the second number of weight vectors; or when there are no weight vectors with a number of columns less than the preset number of columns in the prepared weight vectors, the second number of prepared weight vectors are used as the second number of weight vectors.

[0022] Preferably, dividing each of the weight vectors to obtain a first number of weight sub-blocks comprises:

[0023] Dividing each of the weight vectors according to the number of rows to obtain a fourth number of preliminary weight sub-blocks;

[0024] Determine whether the fourth number is equal to the first number, and when the fourth number is less than the first number, calculate a first difference between the fourth number and the first number, and increase the number of preliminary weight sub-blocks according to the first difference until the first number of preliminary weight sub-blocks are obtained; or when the fourth number is greater than the first number, select the first number of preliminary weight sub-blocks from the fourth number of preliminary weight sub-blocks;

[0025] Determining whether there is a preliminary weight sub-block having a column number less than the preset column number among the first number of preliminary weight sub-blocks;

[0026] When there are preparation weight sub-blocks with a number of columns less than the preset number of columns among the first number of preparation weight sub-blocks, preset values ​​are used to fill the preparation weight sub-blocks with a number of columns less than the preset number of columns to obtain preparation weight sub-blocks with a number of columns equal to the preset number of columns, and the first number of preparation weight sub-blocks are used as the first number of weight sub-blocks; or when there are no preparation weight sub-blocks with a number of columns less than the preset number of columns among the first number of preparation weight sub-blocks, the first number of preparation weight sub-blocks are used as the first number of weight sub-blocks.

[0027] Preferably, dividing the offset matrix to obtain a second number of offset sub-blocks comprises:

[0028] Dividing the offset matrix according to the preset number of rows to obtain prepared offset vectors, and selecting an offset vector from the prepared offset vectors;

[0029] Dividing the offset vector according to the preset number of columns to obtain a fifth number of preliminary offset sub-blocks;

[0030] Determine whether the fifth number is equal to the second number, and when the fifth number is less than the second number, calculate a second difference between the fifth number and the second number, and increase the number of the preliminary offset quantum blocks according to the second difference until the second number of preliminary offset quantum blocks is obtained; or when the fifth number is greater than the second number, select the second number of preliminary offset quantum blocks from the fifth number of preliminary offset quantum blocks;

[0031] Determining whether there are any prepared offset quantum blocks in the second number of prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows;

[0032] When there are prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows among the second number of prepared offset quantum blocks, the prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows are filled with preset values ​​to obtain prepared offset quantum blocks whose number of columns is equal to the preset number of columns and whose number of rows is equal to the preset number of rows, and the second number of prepared offset quantum blocks are used as the second number of offset quantum blocks; or when there are no prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows among the second number of prepared offset quantum blocks, the second number of the prepared offset quantum blocks are used as the second number of offset quantum blocks.

[0033] A second aspect of the present application provides an image feature extraction device, the device comprising:

[0034] An image sub-block acquisition module, used to acquire an image, convert the image into an image matrix, divide the image matrix to obtain image vectors, and divide the image vectors to obtain a first number of image sub-blocks;

[0035] A weight sub-block acquisition module, used to acquire a weight matrix, divide the weight matrix to obtain a second number of weight vectors, and divide each of the weight vectors to obtain a first number of weight sub-blocks;

[0036] An offset quantum block acquisition module, used to acquire an offset matrix, and divide the offset matrix to obtain a second number of offset quantum blocks;

[0037] An image feature calculation module is used to perform vector multiplication on a first number of image sub-blocks and a first number of weight sub-blocks of each weight vector to obtain a second number of feature sub-blocks, and add the second number of feature sub-blocks and the second number of offset sub-blocks to obtain image feature sub-blocks, wherein the image feature sub-blocks constitute image features.

[0038] A third aspect of the present application provides an electronic device, the electronic device comprising:

[0039] a memory storing at least one instruction; and

[0040] A processor executes the instructions stored in the memory to implement the image feature extraction method.

[0041] A fourth aspect of the present application provides a computer storage medium having a computer program stored thereon, wherein the computer program implements the image feature extraction method when executed by a processor.

[0042] In the present application, in this implementation, by dividing the image matrix, weight matrix and deviation matrix into sub-blocks for operation, the amount of convolution operation in the neural network during the image feature extraction process is reduced, thereby improving the efficiency of extracting features from the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Flow chart of an image feature extraction method in one embodiment of the present application.

[0044] Figure 2 It is a structural diagram of an image feature extraction device in one embodiment of the present application.

[0045] Figure 3 Schematic diagram of an electronic device in one embodiment of the present application.

[0046] Figure 4 Schematic diagram of an image feature extraction method in one embodiment of the present application.

[0047] Main component symbols

[0048] Image feature extraction device 30 Image sub-block acquisition module 301 Weight sub-block acquisition module 302 Offset quantum block acquisition module 303 Image feature calculation module 304 Electronic devices 6 Memory 61 processor 62 Computer Programs 63 DETAILED DESCRIPTION

[0049] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present application. The embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0052] Preferably, the image feature extraction method of the present application is applied in one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0053] The electronic device may be a computing device such as a desktop computer, a laptop computer, a tablet computer, a cloud server, etc. The electronic device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0054] Example 1

[0055] Figure 1 1 is a flow chart of an image feature extraction method in one embodiment of the present application. The image feature extraction method is applied to electronic devices. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0056] See also Figure 1 As shown, the image feature extraction method specifically includes the following steps:

[0057] Step S11, acquiring an image, converting the image into an image matrix, dividing the image matrix to obtain image vectors, and dividing the image vectors to obtain a first number of image sub-blocks.

[0058] In at least one embodiment of the present application, converting the image into an image matrix comprises:

[0059] reading the image;

[0060] Obtaining pixel points in the image;

[0061] The three primary color encoding values ​​of the pixel points are combined into pixel vectors, and the pixel vectors are combined into the image matrix.

[0062] In at least one embodiment of the present application, dividing the image matrix to obtain the image vector comprises:

[0063] Dividing the image matrix according to the preset number of rows to obtain a preliminary image vector set, wherein the preliminary image vector set includes at least one preliminary image vector;

[0064] Determining whether there are any preliminary image vectors in the preliminary image vector set whose number of rows is less than the preset number of rows;

[0065] When there exists an image vector whose number of rows is less than the preset number of rows in the prepared image vector, the preset value is used to fill the prepared image vector whose number of rows is less than the preset number of rows to obtain a prepared image vector whose number of rows is equal to the preset number of rows, and the prepared image vector is used as the image vector; or when there does not exist an image vector whose number of rows is less than the preset number of rows in the prepared image vector, the prepared image vector is used as the image vector.

[0066] In at least one embodiment of the present application, dividing the image vector to obtain a first number of image sub-blocks includes:

[0067] Dividing the image vector according to the preset number of columns to obtain a first number of preliminary image sub-blocks;

[0068] Determining whether there is a prepared image sub-block having a column number less than the preset column number in the prepared image sub-block;

[0069] When there are prepared image sub-blocks whose number of columns is less than the preset number of columns among the prepared image sub-blocks, preset values ​​are used to fill the prepared image sub-blocks whose number of columns is less than the preset number of columns to obtain prepared image sub-blocks whose number of columns is equal to the preset number of columns, and the first number of prepared image sub-blocks are used as the first number of image sub-blocks; or when there are no prepared image sub-blocks whose number of columns is less than the preset number of columns among the prepared image sub-blocks, the first number of prepared image sub-blocks are used as the first number of image sub-blocks.

[0070] For example, the preset number of rows may be 3 rows, the preset number of columns may be 3 columns, and the preset value may be 0. When the number of rows of the image matrix is ​​11 rows and the number of columns is 11 columns, the image matrix is ​​divided by 3 rows to obtain 4 preparatory image vectors, one of which is 2 rows and 11 columns, and the preparatory image vector is filled with 0 to obtain 4 image vectors with 3 rows and 11 columns; the image vector with 3 rows and 11 columns is divided by 3 columns to obtain 4 preparatory image sub-blocks, one of which is 3 rows and 2 columns, and the pre-set value 0 is used to fill the preparatory image sub-block to obtain 4 image sub-blocks with 3 rows and 3 columns.

[0071] For example, refer to Figure 4 , the image vector A is divided according to a preset number of columns to obtain m preliminary image sub-blocks. When the number of columns of the m-th preliminary image sub-block is less than the preset number of columns, the m-th preliminary image sub-block is filled with 0 to obtain m image sub-blocks.

[0072] Step S12: obtaining a weight matrix, dividing the weight matrix to obtain a second number of weight vectors, and dividing each of the weight vectors to obtain a first number of weight sub-blocks.

[0073] In at least one embodiment of the present application, dividing the weight matrix to obtain a second number of weight vectors includes:

[0074] Dividing the weight matrix according to the number of columns to obtain a second number of preliminary weight vectors;

[0075] Determining whether there is a preliminary weight vector having a column number less than the preset column number in the preliminary weight vectors;

[0076] When there are weight vectors with a number of columns less than the preset number of columns in the prepared weight vectors, the preset value is used to fill the prepared weight vectors with a number of columns less than the preset number of columns to obtain prepared weight vectors with a number of columns equal to the preset number of columns, and the second number of prepared weight vectors are used as the second number of weight vectors; or when there are no weight vectors with a number of columns less than the preset number of columns in the prepared weight vectors, the second number of prepared weight vectors are used as the second number of weight vectors.

[0077] In at least one embodiment of the present application, dividing each of the weight vectors to obtain a first number of weight sub-blocks includes:

[0078] Dividing each of the weight vectors according to the number of rows to obtain a fourth number of preliminary weight sub-blocks;

[0079] Determine whether the fourth number is equal to the first number, and when the fourth number is less than the first number, calculate a first difference between the fourth number and the first number, and increase the number of preliminary weight sub-blocks according to the first difference until the first number of preliminary weight sub-blocks are obtained; or when the fourth number is greater than the first number, select the first number of preliminary weight sub-blocks from the fourth number of preliminary weight sub-blocks;

[0080] Determining whether there is a preliminary weight sub-block having a column number less than the preset column number among the first number of preliminary weight sub-blocks;

[0081] When there are preparation weight sub-blocks with a number of columns less than the preset number of columns among the first number of preparation weight sub-blocks, preset values ​​are used to fill the preparation weight sub-blocks with a number of columns less than the preset number of columns to obtain preparation weight sub-blocks with a number of columns equal to the preset number of columns, and the first number of preparation weight sub-blocks are used as the first number of weight sub-blocks; or when there are no preparation weight sub-blocks with a number of columns less than the preset number of columns among the first number of preparation weight sub-blocks, the first number of preparation weight sub-blocks are used as the first number of weight sub-blocks.

[0082] In at least one embodiment of the present application, the weight matrix is ​​used to measure the importance of features in the image matrix.

[0083] In at least one embodiment of the present application, the weight matrix can be a matrix in the encoding layer of the autoencoder, or a matrix in the convolution layer of the convolutional neural network, and the weight matrix can be stored in an artificial intelligence chip for general matrix multiplication operations.

[0084] For example, when the weight matrix has 11 rows and 11 columns, the weight matrix is ​​divided by 3 rows to obtain 4 preliminary weight vectors, one of which has 2 rows and 11 columns, and the preliminary weight vector is filled with 0 to obtain 4 weight vectors with 3 rows and 11 columns; the weight vector with 3 rows and 11 columns is divided by 3 columns to obtain 4 preliminary weight sub-blocks, one of which has 3 rows and 2 columns, and the preliminary weight sub-block is filled with a preset value 0 to obtain 4 weight sub-blocks with 3 rows and 3 columns.

[0085] For example, refer to Figure 4 As shown, the weight matrix B is divided according to a preset number of columns to obtain n columns of preliminary weight vectors. When the number of columns of the n-th column of the preliminary weight vector is less than the preset number of columns, the n-th column of the preliminary weight vector is filled with 0 to obtain n columns of weight vectors; each column of the n-column weight vector is divided according to a preset number of rows to obtain m preliminary weight blocks. When the number of columns of the m-th preliminary weight sub-block is less than the preset number of columns, the m-th preliminary weight sub-block is filled with 0 to obtain m weight sub-blocks.

[0086] Step S13, obtaining an offset matrix, and dividing the offset matrix to obtain a second number of offset sub-blocks.

[0087] In at least one embodiment of the present application, dividing the offset matrix to obtain a second number of offset sub-blocks includes:

[0088] Dividing the offset matrix according to the preset number of rows to obtain prepared offset vectors, and selecting an offset vector from the prepared offset vectors;

[0089] Dividing the offset vector according to the preset number of columns to obtain a fifth number of preliminary offset sub-blocks;

[0090] Determine whether the fifth number is equal to the second number, and when the fifth number is less than the second number, calculate a second difference between the fifth number and the second number, and increase the number of the preliminary offset quantum blocks according to the second difference until the second number of preliminary offset quantum blocks is obtained; or when the fifth number is greater than the second number, select the second number of preliminary offset quantum blocks from the fifth number of preliminary offset quantum blocks;

[0091] Determining whether there are any prepared offset quantum blocks in the second number of prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows;

[0092] When there are prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows among the second number of prepared offset quantum blocks, the prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows are filled with preset values ​​to obtain prepared offset quantum blocks whose number of columns is equal to the preset number of columns and whose number of rows is equal to the preset number of rows, and the second number of prepared offset quantum blocks are used as the second number of offset quantum blocks; or when there are no prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows among the second number of prepared offset quantum blocks, the second number of the prepared offset quantum blocks are used as the second number of offset quantum blocks.

[0093] In at least one embodiment of the present application, the offset matrix is ​​used to measure the difficulty of the neural network to generate positive excitation or negative excitation.

[0094] In at least one embodiment of the present application, the offset matrix can be a matrix in the encoding layer of the autoencoder, or a matrix in the convolution layer of the convolutional neural network, and the offset matrix can be stored in an artificial intelligence chip for general matrix multiplication operations.

[0095] For example, when the weight matrix has 11 rows and 11 columns, the weight matrix is ​​divided by 3 rows to obtain 4 preliminary weight vectors, and the first preliminary weight vector is selected as the weight vector; the weight vector of 3 rows and 11 columns is divided by 3 columns to obtain 4 preliminary weight sub-blocks, one of which is 3 rows and 2 columns. The preliminary weight sub-block is filled with a preset value of 0 to obtain 4 weight sub-blocks of 3 rows and 3 columns.

[0096] For example, refer to Figure 4 As shown, the offset vector C is divided according to a preset number of columns to obtain n prepared offset quantum blocks. When the number of columns of the nth prepared offset quantum block is less than the preset number of columns, 0 is used to fill the nth prepared offset quantum block to obtain n offset quantum blocks.

[0097] Step S14, performing vector multiplication on the first number of image sub-blocks and the first number of weight sub-blocks of each of the weight vectors to obtain a second number of feature sub-blocks, adding the second number of feature sub-blocks and the second number of offset sub-blocks to obtain image feature sub-blocks, and the image feature sub-blocks constitute image features.

[0098] For example, the four weight sub-blocks of the weight vector are multiplied and summed with the four image sub-blocks of the image vector to obtain four feature sub-blocks, and the four feature sub-blocks are added to the four offset sub-blocks to obtain four image feature sub-blocks. The four image feature sub-blocks constitute the image feature.

[0099] For example, refer to Figure 4As shown, a vector multiplication operation is performed on the image vector A and each column of the weight vector in the weight matrix, that is, the m image sub-blocks in the image vector A and the m weight sub-blocks in each column of the weight vector are multiplied and summed in turn to obtain n feature sub-blocks, and the n feature sub-blocks and n offset sub-blocks are added in turn to obtain n image feature sub-blocks, and the n image feature sub-blocks constitute the image feature.

[0100] In at least one embodiment of the present application, in at least one embodiment of the present application, each of the image sub-block, the weight sub-block, and the offset sub-block has a preset number of rows and a preset number of columns, and the preset number of columns is the same as the preset number of rows.

[0101] In at least one embodiment of the present application, the image feature extraction method can be used for similar image judgment.

[0102] Specifically, using the image feature extraction method to perform similar image judgment includes:

[0103] Acquire a first image to be detected and a second image to be detected, and convert the first image to be detected and the second image to be detected into a first image to be detected vector and a second image to be detected vector;

[0104] Using the image feature extraction method, calculate a first image feature to be detected of the first image vector to be detected and a second image feature to be detected of the second image vector to be detected;

[0105] Calculating an error function value between the first image feature to be detected and the second image feature to be detected using a preset error function;

[0106] When the error function value is less than a preset threshold, it is determined that the first image to be detected and the second image to be detected are the same image; when the error function value is greater than the preset threshold, it is determined that the first image to be detected and the second image to be detected are different images.

[0107] In other embodiments of the present application, the image feature extraction method can also be used for an autoencoder.

[0108] Specifically, the image feature extraction method can be used in the encoding layer and / or decoding layer of an autoencoder, and reconstructing an image by the encoder includes:

[0109] Acquire an image to be reconstructed, and convert the image to be reconstructed into an image vector to be reconstructed;

[0110] Inputting the image vector to be reconstructed into the encoding layer of the autoencoder, the encoding layer of the autoencoder uses the image feature extraction method to obtain a reconstructed image latent vector;

[0111] The reconstructed image latent vector is input into the decoding layer of the autoencoder, and the decoding layer of the autoencoder uses the image feature extraction method to obtain a reconstructed image of the image to be reconstructed.

[0112] In this embodiment, by dividing the image matrix, weight matrix and deviation matrix into sub-blocks for operation, the amount of convolution operation in the neural network during the image feature extraction process is reduced, thereby improving the efficiency of extracting features from the image.

[0113] Example 2

[0114] Figure 2 1 is a structural diagram of an image feature extraction device 30 in one embodiment of the present application.

[0115] In some embodiments, the image feature extraction device 30 runs in an electronic device. The image feature extraction device 30 may include a plurality of functional modules composed of program code segments. The program code of each program segment in the image feature extraction device 30 may be stored in a memory and executed by at least one processor to perform the image feature extraction function.

[0116] In this embodiment, the image feature extraction device 30 can be divided into multiple functional modules according to the functions it performs. Figure 2 As shown, the image feature extraction device 30 may include an image sub-block acquisition module 301, a weight sub-block acquisition module 302, an offset sub-block acquisition module 303 and an image feature calculation module 304. The module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In some embodiments, the functions of each module will be described in detail in subsequent embodiments.

[0117] The image sub-block acquisition module 301 acquires an image, converts the image into an image matrix, divides the image matrix to obtain image vectors, and divides the image vectors to obtain a first number of image sub-blocks.

[0118] In at least one embodiment of the present application, the image sub-block acquisition module 301 converts the image into an image matrix including:

[0119] reading the image;

[0120] Obtaining pixel points in the image;

[0121] The three primary color encoding values ​​of the pixel points are combined into pixel vectors, and the pixel vectors are combined into the image matrix.

[0122] In at least one embodiment of the present application, the image sub-block acquisition module 301 divides the image matrix to obtain the image vector including:

[0123] Dividing the image matrix according to the preset number of rows to obtain a preliminary image vector set, wherein the preliminary image vector set includes at least one preliminary image vector;

[0124] Determining whether there are any preliminary image vectors in the preliminary image vector set whose number of rows is less than the preset number of rows;

[0125] When there exists an image vector whose number of rows is less than the preset number of rows in the prepared image vector, the preset value is used to fill the prepared image vector whose number of rows is less than the preset number of rows to obtain a prepared image vector whose number of rows is equal to the preset number of rows, and the prepared image vector is used as the image vector; or when there does not exist an image vector whose number of rows is less than the preset number of rows in the prepared image vector, the prepared image vector is used as the image vector.

[0126] In at least one embodiment of the present application, the image sub-block acquisition module 301 divides the image vector to obtain a first number of image sub-blocks, including:

[0127] Dividing the image vector according to the preset number of columns to obtain a first number of preliminary image sub-blocks;

[0128] Determining whether there is a prepared image sub-block having a column number less than the preset column number in the prepared image sub-block;

[0129] When there are prepared image sub-blocks whose number of columns is less than the preset number of columns among the prepared image sub-blocks, preset values ​​are used to fill the prepared image sub-blocks whose number of columns is less than the preset number of columns to obtain prepared image sub-blocks whose number of columns is equal to the preset number of columns, and the first number of prepared image sub-blocks are used as the first number of image sub-blocks; or when there are no prepared image sub-blocks whose number of columns is less than the preset number of columns among the prepared image sub-blocks, the first number of prepared image sub-blocks are used as the first number of image sub-blocks.

[0130] The weight sub-block acquisition module 302 acquires a weight matrix, divides the weight matrix to obtain a second number of weight vectors, and divides each of the weight vectors to obtain a first number of weight sub-blocks.

[0131] In at least one embodiment of the present application, the weight sub-block acquisition module 302 divides the weight matrix to obtain a second number of weight vectors including:

[0132] Dividing the weight matrix according to the number of columns to obtain a second number of preliminary weight vectors;

[0133] Determining whether there is a preliminary weight vector having a column number less than the preset column number in the preliminary weight vectors;

[0134] When there are weight vectors with a number of columns less than the preset number of columns in the prepared weight vectors, the preset value is used to fill the prepared weight vectors with a number of columns less than the preset number of columns to obtain prepared weight vectors with a number of columns equal to the preset number of columns, and the second number of prepared weight vectors are used as the second number of weight vectors; or when there are no weight vectors with a number of columns less than the preset number of columns in the prepared weight vectors, the second number of prepared weight vectors are used as the second number of weight vectors.

[0135] In at least one embodiment of the present application, the weight sub-block acquisition module 302 divides each of the weight vectors to obtain a first number of weight sub-blocks, including:

[0136] Dividing each of the weight vectors according to the number of rows to obtain a fourth number of preliminary weight sub-blocks;

[0137] Determine whether the fourth number is equal to the first number, and when the fourth number is less than the first number, calculate a first difference between the fourth number and the first number, and increase the number of preliminary weight sub-blocks according to the first difference until the first number of preliminary weight sub-blocks are obtained; or when the fourth number is greater than the first number, select the first number of preliminary weight sub-blocks from the fourth number of preliminary weight sub-blocks;

[0138] Determining whether there is a preliminary weight sub-block having a column number less than the preset column number among the first number of preliminary weight sub-blocks;

[0139] When there are preparation weight sub-blocks with a number of columns less than the preset number of columns among the first number of preparation weight sub-blocks, preset values ​​are used to fill the preparation weight sub-blocks with a number of columns less than the preset number of columns to obtain preparation weight sub-blocks with a number of columns equal to the preset number of columns, and the first number of preparation weight sub-blocks are used as the first number of weight sub-blocks; or when there are no preparation weight sub-blocks with a number of columns less than the preset number of columns among the first number of preparation weight sub-blocks, the first number of preparation weight sub-blocks are used as the first number of weight sub-blocks.

[0140] In at least one embodiment of the present application, the weight matrix is ​​used to measure the importance of features in the image matrix.

[0141] In at least one embodiment of the present application, the weight matrix can be a matrix in the encoding layer of the autoencoder, or a matrix in the convolution layer of the convolutional neural network, and the weight matrix can be stored in an artificial intelligence chip for general matrix multiplication operations.

[0142] The offset quantum block acquisition module 303 acquires an offset matrix, and divides the offset matrix to obtain a second number of offset quantum blocks.

[0143] In at least one embodiment of the present application, the offset quantum block acquisition module 303 divides the offset matrix to obtain a second number of offset quantum blocks, including:

[0144] Dividing the offset matrix according to the preset number of rows to obtain prepared offset vectors, and selecting an offset vector from the prepared offset vectors;

[0145] Dividing the offset vector according to the preset number of columns to obtain a fifth number of preliminary offset sub-blocks;

[0146] Determine whether the fifth number is equal to the second number, and when the fifth number is less than the second number, calculate a second difference between the fifth number and the second number, and increase the number of the preliminary offset quantum blocks according to the second difference until the second number of preliminary offset quantum blocks is obtained; or when the fifth number is greater than the second number, select the second number of preliminary offset quantum blocks from the fifth number of preliminary offset quantum blocks;

[0147] Determining whether there are any prepared offset quantum blocks in the second number of prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows;

[0148] When there are prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows among the second number of prepared offset quantum blocks, the prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows are filled with preset values ​​to obtain prepared offset quantum blocks whose number of columns is equal to the preset number of columns and whose number of rows is equal to the preset number of rows, and the second number of prepared offset quantum blocks are used as the second number of offset quantum blocks; or when there are no prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows among the second number of prepared offset quantum blocks, the second number of the prepared offset quantum blocks are used as the second number of offset quantum blocks.

[0149] In at least one embodiment of the present application, the offset matrix is ​​used to measure the difficulty of the neural network to generate positive excitation or negative excitation.

[0150] In at least one embodiment of the present application, the offset matrix can be a matrix in the encoding layer of the autoencoder, or a matrix in the convolution layer of the convolutional neural network, and the offset matrix can be stored in an artificial intelligence chip for general matrix multiplication operations.

[0151] The image feature calculation module 304 performs vector multiplication on the first number of image sub-blocks and the first number of weight sub-blocks of each weight vector to obtain a second number of feature sub-blocks, and adds the second number of feature sub-blocks to the second number of offset sub-blocks to obtain image feature sub-blocks, and the image feature sub-blocks constitute image features.

[0152] In at least one embodiment of the present application, in at least one embodiment of the present application, each of the image sub-block, the weight sub-block, and the offset sub-block has a preset number of rows and a preset number of columns, and the preset number of columns is the same as the preset number of rows.

[0153] In this embodiment, by dividing the image matrix, weight matrix and deviation matrix into sub-blocks for operation, the amount of convolution operation in the neural network during the image feature extraction process is reduced, thereby improving the efficiency of extracting features from the image.

[0154] Example 3

[0155] Figure 3 Schematic diagram of an electronic device 6 in one embodiment of the present application.

[0156] The electronic device 6 includes a memory 61, a processor 62, and a computer program 63 stored in the memory 61 and executable on the processor 62. When the processor 62 executes the computer program 63, the steps in the above-mentioned image feature extraction method embodiment are implemented, for example: Figure 1 Alternatively, when the processor 62 executes the computer program 63, the functions of each module / unit in the above-mentioned image feature extraction device embodiment are realized, for example Figure 2 Modules 301 to 304 in.

[0157] Exemplarily, the computer program 63 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 62 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 63 in the electronic device 6. For example, the computer program 63 may be divided into Figure 2 The image sub-block acquisition module 301, the weight sub-block acquisition module 302, the offset sub-block acquisition module 303 and the image feature calculation module 304, the specific functions of each module refer to Example 2.

[0158] In this embodiment, the electronic device 6 may be a computing device such as a desktop computer, a notebook, a PDA, a server, a cloud terminal device, etc. Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 6 and does not constitute a limitation on the electronic device 6. The electronic device 6 may include more or fewer components than shown in the diagram, or may combine certain components, or different components. For example, the electronic device 6 may also include input and output devices, network access devices, buses, etc.

[0159] The processor 62 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor 62 may be any conventional processor, etc. The processor 62 is the control center of the electronic device 6, and uses various interfaces and lines to connect various parts of the entire electronic device 6.

[0160] The memory 61 can be used to store the computer program 63 and / or modules / units. The processor 62 implements various functions of the electronic device 6 by running or executing the computer program and / or modules / units stored in the memory 61 and calling the data stored in the memory 61. The memory 61 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the electronic device 6 (such as audio data, a phone book, etc.), etc. In addition, the memory 61 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0161] If the module / unit integrated in the electronic device 6 is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0163] In addition, each functional module in each embodiment of the present application can be integrated into the same processing module, or each module can exist physically separately, or two or more modules can be integrated into the same module. The above integrated modules can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0164] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the attached claims rather than the above description, so it is intended to include all changes that fall within the meaning and scope of the equivalent elements of the claims in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other modules or steps, and the singular does not exclude the plural. Multiple modules or electronic devices stated in the electronic device claim can also be implemented by the same module or electronic device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A method for extracting image features, characterized in that: The method comprises: Acquire an image, convert the image into an image matrix, divide the image matrix to obtain image vectors, and divide the image vectors to obtain a first number of image sub-blocks; Acquire a weight matrix, divide the weight matrix to obtain a second number of weight vectors, and divide each of the weight vectors to obtain a first number of weight sub-blocks, wherein the weight matrix is ​​used to measure the importance of features in the image matrix; Obtaining an offset matrix, dividing the offset matrix to obtain a second number of offset sub-blocks, wherein the offset matrix is ​​used to measure the difficulty of the neural network to generate positive excitation or negative excitation; Vector multiplication is performed on a first number of image sub-blocks and a first number of weight sub-blocks of each of the weight vectors to obtain a second number of feature sub-blocks, and the second number of feature sub-blocks and the second number of offset sub-blocks are added to obtain image feature sub-blocks, wherein the image feature sub-blocks constitute image features, wherein each of the image sub-blocks, the weight sub-blocks, and the offset sub-blocks has a preset number of rows and a preset number of columns, and the preset number of columns is the same as the preset number of rows.

2. The image feature extraction method according to claim 1, characterized in that: The dividing the image matrix to obtain the image vector comprises: Dividing the image matrix according to the preset number of rows to obtain a preliminary image vector set, wherein the preliminary image vector set includes at least one preliminary image vector; Determining whether there are any preliminary image vectors in the preliminary image vector set whose number of rows is less than the preset number of rows; When there exists an image vector whose number of rows is less than the preset number of rows in the prepared image vector, the preset value is used to fill the prepared image vector whose number of rows is less than the preset number of rows to obtain a prepared image vector whose number of rows is equal to the preset number of rows, and the prepared image vector is used as the image vector; or when there does not exist an image vector whose number of rows is less than the preset number of rows in the prepared image vector, the prepared image vector is used as the image vector.

3. The image feature extraction method according to claim 1, characterized in that: Dividing the image vector to obtain a first number of image sub-blocks includes: Dividing the image vector according to the preset number of columns to obtain a first number of preliminary image sub-blocks; Determining whether there is a prepared image sub-block having a column number less than the preset column number in the prepared image sub-block; When there are prepared image sub-blocks whose number of columns is less than the preset number of columns among the prepared image sub-blocks, preset values ​​are used to fill the prepared image sub-blocks whose number of columns is less than the preset number of columns to obtain prepared image sub-blocks whose number of columns is equal to the preset number of columns, and the first number of prepared image sub-blocks are used as the first number of image sub-blocks; or when there are no prepared image sub-blocks whose number of columns is less than the preset number of columns among the prepared image sub-blocks, the first number of prepared image sub-blocks are used as the first number of image sub-blocks.

4. The image feature extraction method according to claim 1, characterized in that: Dividing the weight matrix to obtain a second number of weight vectors includes: Dividing the weight matrix according to the number of columns to obtain a second number of preliminary weight vectors; Determining whether there is a preliminary weight vector having a column number less than the preset column number in the preliminary weight vectors; When there are weight vectors with a number of columns less than the preset number of columns in the prepared weight vectors, the preset value is used to fill the prepared weight vectors with a number of columns less than the preset number of columns to obtain prepared weight vectors with a number of columns equal to the preset number of columns, and the second number of prepared weight vectors are used as the second number of weight vectors; or when there are no weight vectors with a number of columns less than the preset number of columns in the prepared weight vectors, the second number of prepared weight vectors are used as the second number of weight vectors.

5. The image feature extraction method according to claim 1, characterized in that: Dividing each of the weight vectors to obtain a first number of weight sub-blocks includes: Dividing each of the weight vectors according to the number of rows to obtain a fourth number of preliminary weight sub-blocks; Determine whether the fourth number is equal to the first number, and when the fourth number is less than the first number, calculate a first difference between the fourth number and the first number, and increase the number of preliminary weight sub-blocks according to the first difference until the first number of preliminary weight sub-blocks are obtained; or when the fourth number is greater than the first number, select the first number of preliminary weight sub-blocks from the fourth number of preliminary weight sub-blocks; Determining whether there is a preliminary weight sub-block having a column number less than the preset column number among the first number of preliminary weight sub-blocks; When there are preparation weight sub-blocks with a number of columns less than the preset number of columns among the first number of preparation weight sub-blocks, preset values ​​are used to fill the preparation weight sub-blocks with a number of columns less than the preset number of columns to obtain preparation weight sub-blocks with a number of columns equal to the preset number of columns, and the first number of preparation weight sub-blocks are used as the first number of weight sub-blocks; or when there are no preparation weight sub-blocks with a number of columns less than the preset number of columns among the first number of preparation weight sub-blocks, the first number of preparation weight sub-blocks are used as the first number of weight sub-blocks.

6. The image feature extraction method according to claim 1, characterized in that: The dividing the offset matrix to obtain a second number of offset sub-blocks comprises: Dividing the offset matrix according to the preset number of rows to obtain prepared offset vectors, and selecting an offset vector from the prepared offset vectors; Dividing the offset vector according to the preset number of columns to obtain a fifth number of preliminary offset sub-blocks; Determine whether the fifth number is equal to the second number, and when the fifth number is less than the second number, calculate a second difference between the fifth number and the second number, and increase the number of the preliminary offset quantum blocks according to the second difference until the second number of preliminary offset quantum blocks is obtained; or when the fifth number is greater than the second number, select the second number of preliminary offset quantum blocks from the fifth number of preliminary offset quantum blocks; Determining whether there are any prepared offset quantum blocks in the second number of prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows; When there are prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows among the second number of prepared offset quantum blocks, the prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows are filled with preset values ​​to obtain prepared offset quantum blocks whose number of columns is equal to the preset number of columns and whose number of rows is equal to the preset number of rows, and the second number of prepared offset quantum blocks are used as the second number of offset quantum blocks; or when there are no prepared offset quantum blocks whose number of columns is less than the preset number of columns or whose number of rows is less than the preset number of rows among the second number of prepared offset quantum blocks, the second number of the prepared offset quantum blocks are used as the second number of offset quantum blocks.

7. An image feature extraction device, characterized in that: The device comprises a module for implementing the image feature extraction method according to any one of claims 1 to 6, and the device comprises: An image sub-block acquisition module, used to acquire an image, convert the image into an image matrix, divide the image matrix to obtain image vectors, and divide the image vectors to obtain a first number of image sub-blocks; A weight sub-block acquisition module, used to acquire a weight matrix, divide the weight matrix to obtain a second number of weight vectors, and divide each of the weight vectors to obtain a first number of weight sub-blocks; An offset quantum block acquisition module, used to acquire an offset matrix, and divide the offset matrix to obtain a second number of offset quantum blocks; An image feature calculation module is used to perform vector multiplication on a first number of image sub-blocks and a first number of weight sub-blocks of each weight vector to obtain a second number of feature sub-blocks, and add the second number of feature sub-blocks and the second number of offset sub-blocks to obtain image feature sub-blocks, wherein the image feature sub-blocks constitute image features.

8. An electronic device, characterized in that: The electronic device comprises: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the image feature extraction method according to any one of claims 1 to 6.

9. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image feature extraction method according to any one of claims 1 to 6 is implemented.

Images