An image feature extraction method and device, an electronic device, and a storage medium
By combining tensor shrinkage and local feature extraction, the problem of inaccurate image feature extraction in existing technologies is solved, and accurate representation and recognition of target objects are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA CONSTRUCTION BANK
- Filing Date
- 2022-08-25
- Publication Date
- 2026-06-09
AI Technical Summary
Existing image feature extraction methods cannot accurately capture the original structural information in an image, resulting in insufficient accuracy in object recognition.
The first recognition model is used to perform tensor shrinking on the initial image tensor, and the higher-order tensor is extracted as the first feature, while preserving the original geometric structure of the image. At the same time, the second recognition model is used to extract local features to obtain the second feature with the required number of features. The target feature is determined by combining the two.
By preserving the original geometric structure of the image and including target features with a specific number of features, accurate representation of the target object is achieved, thus improving the accuracy of object recognition.
Smart Images
Figure CN115346053B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to an image feature extraction method, apparatus, electronic device, and storage medium. Background Technology
[0002] With the development of technology, identity recognition technology has been applied to many fields. Identity recognition technology includes characteristic object recognition, special knowledge recognition, and biometric recognition. Biometric recognition requires extracting features from a captured image containing the target object, and then identifying the target object based on the feature extraction results.
[0003] In related technologies, a local sensing approach is used, which combines a specified number of local features determined from the acquired image to obtain more comprehensive image features.
[0004] However, the above process extracts local features of the image. After combining local features, the resulting image features often cannot accurately contain the original structural information in the image. Therefore, the obtained image features are not accurate enough and affect the accuracy of object recognition. Summary of the Invention
[0005] This application provides an image feature extraction method, apparatus, electronic device, and storage medium for accurately extracting object features from images.
[0006] In a first aspect, embodiments of this application provide an image feature extraction method, the method comprising:
[0007] The image to be identified, which contains the target object, is transformed to obtain the initial image tensor corresponding to the image to be identified;
[0008] Based on a preset first recognition model, tensor simplification is performed on the initial image tensor to determine the first feature of the target object; and based on a preset second recognition model, local feature extraction is performed on the initial image tensor to determine the second feature of the target object.
[0009] Based on the first feature and the second feature, the target features of the target object are determined.
[0010] In some optional implementations, determining the target features of the target object based on the first feature and the second feature includes:
[0011] The first feature and the second feature are combined to obtain the target feature of the target object; wherein the number of features of the target feature is the sum of the number of features of the first feature and the number of features of the second feature.
[0012] In some optional implementations, the first recognition model includes multiple shrinking filters; based on the preset first recognition model, tensor shrinking is performed on the initial image tensor to determine the first feature of the target object, including:
[0013] For any shrink filter, the initial image tensor is shrunk with the shrink filter to obtain the mixing matrix corresponding to the shrink filter;
[0014] Based on a preset feature order, all obtained mixing matrices are aggregated to determine the first feature of the target object.
[0015] In some optional implementations, the initial image tensor is shrunk with the shrinking filter to obtain a mixing matrix corresponding to the shrinking filter, including:
[0016] The initial image tensor is summed with the preset data index corresponding to the shrinking filter to obtain the mixing matrix corresponding to the shrinking filter.
[0017] In some optional implementations, the second recognition model includes multiple convolutional kernels; based on the preset second recognition model, local feature extraction is performed on the initial image tensor to determine the second feature of the target object, including:
[0018] For any convolution kernel, local feature extraction and pooling operations are performed on the corresponding pixels in the initial image tensor based on the convolution kernel to determine the local features corresponding to the convolution kernel;
[0019] A fully connected operation is performed on the local features corresponding to all convolutional kernels to determine the second feature of the target object.
[0020] Some optional implementations also include:
[0021] The target feature is matched with each preset feature information, and the identity information of the target object is determined based on the matching result.
[0022] Secondly, embodiments of this application also provide an image feature extraction apparatus, the apparatus comprising:
[0023] The tensor determination module is used to transform the image to be identified, which contains the target object, to obtain the initial image tensor corresponding to the image to be identified.
[0024] The feature determination module is used to perform tensor simplification on the initial image tensor based on a preset first recognition model to determine the first feature of the target object; and to perform local feature extraction on the initial image tensor based on a preset second recognition model to determine the second feature of the target object.
[0025] The feature combination module is used to determine the target features of the target object based on the first feature and the second feature.
[0026] Thirdly, embodiments of this application provide an electronic device, including at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor performs the image feature extraction method described in any of the first aspects above.
[0027] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer, which, when run on the computer, causes the computer to perform the image feature extraction method described in any of the first aspects above.
[0028] Fifthly, embodiments of this application provide a computer program product comprising computer-executable instructions for causing a computer to perform the image feature extraction method as described in any of the first aspects.
[0029] This application provides an image feature extraction method, apparatus, electronic device, and storage medium. The method involves performing tensor simplification on an initial image tensor using a first recognition model, extracting a higher-order tensor as a first feature, which preserves the original geometric structure of the image to be recognized. A second recognition model is then used to extract local features from the initial image tensor to obtain a second feature with the required number of features. Based on the first and second features, a target feature is obtained that preserves the original geometric structure of the image to be recognized and contains a specific number of features. This target feature can accurately characterize the target object in the image to be recognized. Attached Figure Description
[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1 This is a schematic diagram illustrating an application scenario provided in the embodiments of this application;
[0032] Figure 2A flowchart illustrating the first image feature extraction method provided in this application embodiment;
[0033] Figure 3 This is a schematic diagram of the feature combination process provided in an embodiment of this application;
[0034] Figure 4 A flowchart illustrating the first feature extraction method provided in an embodiment of this application;
[0035] Figure 5 A schematic diagram of a tensor shrinking operation provided for an embodiment of this application;
[0036] Figure 6 A schematic diagram illustrating the initial image tensor shrinking process provided in this application embodiment;
[0037] Figure 7 A schematic flowchart of the second feature extraction method provided in the embodiments of this application;
[0038] Figure 8 A flowchart illustrating the second image feature extraction method provided in this application embodiment;
[0039] Figure 9 This is a schematic diagram of the image feature extraction device provided in the embodiments of this application;
[0040] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0042] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the term "connection" should be interpreted broadly. For example, it can refer to a direct connection, an indirect connection through an intermediate medium, or a connection within two devices. Those skilled in the art can understand the specific meaning of the above term in this application based on the specific circumstances.
[0043] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating relative importance or implying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.
[0044] Identity recognition technologies include feature-based object recognition, special knowledge recognition, and biometric recognition. Biometric recognition requires extracting features from an image containing the target object, and then identifying the target object based on the feature extraction results.
[0045] In related technologies, a local sensing approach is used, which combines a specified number of local features determined from the acquired image to obtain more comprehensive image features.
[0046] However, the above process extracts local features of the image. After combining local features, the resulting image features often cannot accurately contain the original structural information in the image. Therefore, the obtained image features are not accurate enough and affect the accuracy of object recognition.
[0047] In view of this, embodiments of this application propose an image feature extraction method, apparatus, electronic device, and storage medium. The method includes: converting an image to be identified containing a target object to obtain an initial image tensor corresponding to the image to be identified; performing tensor simplification on the initial image tensor based on a preset first recognition model to determine a first feature of the target object; and performing local feature extraction on the initial image tensor based on a preset second recognition model to determine a second feature of the target object; and determining a target feature of the target object based on the first feature and the second feature.
[0048] The above scheme performs tensor condensation on the initial image tensor using a first recognition model, and uses the extracted higher-order tensor as the first feature, which preserves the original geometric structure of the image to be recognized. It also performs local feature extraction on the initial image tensor using a second recognition model to obtain the second feature with the required number of features. Based on the first and second features, a target feature that preserves the original geometric structure of the image to be recognized and contains a specific number of features is obtained. This target feature can accurately represent the target object in the image to be recognized.
[0049] For ease of understanding, the following explanations are provided for some of the nouns or terms that appear in the description of the embodiments of this application:
[0050] The term "tensor" refers to a multilinear mapping defined on the Cartesian product of a vector space and its dual space. It can be characterized as a high-dimensional array of order d. For example, there exists a tensor A, denoted as .
[0051] The term "tensor contraction" refers to an algebraic operation performed on the same data dimension (denoted as the axis of the tensor) of two different tensors. For example, there exist tensors... tensor Among them, those with I n =J m Then, after performing the contraction operation on the same axis as above, we can obtain a tensor C of order (N+M-2), denoted as As shown in the following formula:
[0052]
[0053] It is important to note that the above tensor shrinking will only take effect if the two tensors have the same data dimension, and that data dimension is a specific data dimension.
[0054] Furthermore, when tensors A and B are completely identical... Tensor shrinkage performed on all axes except the k-th axis can be expressed by the following equation:
[0055]
[0056] At this point, the calculated result The representation is the obtained second-order array (i.e., represented as a matrix), and the following relationship exists:
[0057]
[0058] Among them, A (k) B (k) Let A and B be represented by their modulo n expansions, respectively, i.e., the element matrices formed by rearranging the elements of tensors A and B, respectively.
[0059] Furthermore, when there exists a tensor B that is a second-order tensor (i.e., a matrix), and its size is restricted to J×I... n On a specific axis I n When tensors A and B are condensed, the following results are obtained:
[0060]
[0061] It can be observed that the tensor B used in the tensor shrinking process is a second-order tensor, and the size of tensor B is J×I. nIn this case, the tensor contraction between tensors A and B can also be expressed as the product of their modulo n (A × n). n B)i 1，……， i n-1， i n， i n+1，……， i M This refers to a special form of tensor shrinking, which is the product of tensors modulo n. It is important to understand that the product of tensors modulo n is mainly used to reduce the dimensionality of the data, while tensor shrinking is mainly used to reduce the order of the data.
[0062] Multidimensional Tensor Contraction Operation Layer (MTCOL): This refers to a neural network layer based on tensor contraction. The main computational method of this operation layer is tensor contraction, as mentioned above. The basic operation definition of MTCOL is as follows:
[0063]
[0064] in, If there exist data with the same dimension, and that data dimension is a higher-order tensor of a specific data dimension, then the gradients of the basic operations of MTCOL are as follows:
[0065]
[0066] Based on the above operational definitions, it can be seen that through the tensor shrinking operation in the multidimensional tensor shrinking operation layer, the corresponding initial image tensor A can be transformed. i Transform it into the corresponding mixed matrix.
[0067] The data (such as the image to be identified) obtained in the embodiments of this application all comply with legal regulations.
[0068] Based on the above explanations of terms and nouns, the image recognition method provided in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.
[0069] See Figure 1 The illustration shows a possible application scenario provided by an embodiment of this application. The application scenario includes an image acquisition device 100 and an electronic device 200. Data transmission can be performed between the image acquisition device 100 and the electronic device 200. This embodiment does not limit the specific communication method between the image acquisition device 100 and the electronic device 200.
[0070] Image acquisition device 100 is used to acquire an image to be identified containing a target object and send the image to be identified to electronic device 200;
[0071] The electronic device 200 is used to convert an image to be identified containing a target object to obtain an initial image tensor corresponding to the image to be identified; based on a preset first recognition model, to perform tensor simplification on the initial image tensor to determine a first feature of the target object; and based on a preset second recognition model, to perform local feature extraction on the initial image tensor to determine a second feature of the target object; and based on the first feature and the second feature, to determine a target feature of the target object.
[0072] In some alternative implementations, the image acquisition device 100 described above is a device that can provide voice and / or data connectivity to the user, including handheld electronic devices with wireless connectivity, vehicle-mounted electronic devices, etc.
[0073] Electronic device 200 can be a mobile phone, computer, wireless electronic device, or other device equipped with the aforementioned first recognition model and second recognition model.
[0074] In practice, the aforementioned electronic devices can be connected to one or more image acquisition devices.
[0075] The above application scenarios are merely illustrative examples. This application does not limit specific application scenarios. For example, in other scenarios, if the electronic device is a device with image acquisition function, then the application scenario may not include an additional image acquisition device, and the electronic device may perform the steps of image acquisition and image feature extraction.
[0076] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with reference to the accompanying drawings and specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0077] This application provides a first image feature extraction method, applied to the aforementioned electronic device, as shown in Figure 2, including the following steps:
[0078] Step S201: Transform the image to be identified containing the target object to obtain the initial image tensor corresponding to the image to be identified.
[0079] For example, for an image to be identified acquired by an image acquisition device or electronic device, after determining that the image contains a target object, it can be converted into a corresponding initial image tensor based on specified image parameters. For example, for an image to be identified with RGB 3 channels, it can be converted into a corresponding initial image tensor [h,w,c1] based on the image height h (the image contains h rows of pixels), image width w (the image contains w columns of pixels), and color depth information (RGB values).
[0080] Step S202: Based on a preset first recognition model, perform tensor simplification on the initial image tensor to determine the first feature of the target object; and based on a preset second recognition model, perform local feature extraction on the initial image tensor to determine the second feature of the target object.
[0081] For example, the first recognition model has a global receptive field. Through the first recognition model, the initial image tensor is tensor condensed to obtain a first feature that retains the original geometric structure of the image to be recognized. However, the number of features of the first feature is the same as that of the initial image tensor. If the number of features of the initial image tensor is small, the number of features of the first feature is also small, which affects the subsequent identification of the target object.
[0082] The second recognition model has a local receptive field. Through the second recognition model, local features are extracted from the initial image tensor to obtain a second feature with a different geometric structure from the image to be recognized (the tensor is converted into a vector). The number of features of the second feature can be adjusted by setting the second recognition model.
[0083] Step S203: Based on the first feature and the second feature, determine the target feature of the target object.
[0084] As mentioned above, the first feature extracted by the first recognition model retains the structural and multilinear information of the initial image tensor, but the number of features is limited; the second feature extracted by the second recognition model flattens the initial image tensor to obtain a data vector, but the number of features of the second feature is more flexible.
[0085] Based on this, this embodiment can obtain target features that retain the original geometric structure of the image to be identified and contain a specific number of features based on the first feature and the second feature, thus making up for the problems of the small number of features in the first feature and the large amount of missing information in the second feature.
[0086] The above scheme performs tensor condensation on the initial image tensor using a first recognition model, and uses the extracted higher-order tensor as the first feature, which preserves the original geometric structure of the image to be recognized. It also performs local feature extraction on the initial image tensor using a second recognition model to obtain the second feature with the required number of features. Based on the first and second features, a target feature that preserves the original geometric structure of the image to be recognized and contains a specific number of features is obtained. This target feature can accurately represent the target object in the image to be recognized.
[0087] In some optional implementations, step S203 above can be implemented in, but is not limited to, the following ways:
[0088] The first feature and the second feature are combined to obtain the target feature of the target object; wherein the number of features of the target feature is the sum of the number of features of the first feature and the number of features of the second feature.
[0089] For example, when the number of features of the first feature is small, the number of features of the second feature can be determined based on a specific number of features (that is, the number of features of the target feature mentioned above) and the number of features of the first feature. That is, the number of features of the second feature is c2 = c3 - c1, where c3 is the number of features of the target feature and c1 is the number of features of the first feature.
[0090] See Figure 3 As shown, the initial image tensor A [h,w,c1] Input the first recognition model and the second recognition model respectively; the first recognition model for A [h,w,c1] Tensor shrinking is performed to obtain the first feature B. [h,w,c1] The second recognition model for A [h,w,c1] Local feature extraction is performed to obtain the second feature C. [h,w,c2] B [h,w,c1] With C [h,w,c2] By combining the features, we obtain the target feature D. [h,w,c3] c3 = c1 + c2.
[0091] The above scheme combines the first feature and the second feature. By adjusting the number of features in the second feature, a specific number of target features can be obtained, thus making up for the problem of insufficient feature number in the first feature.
[0092] In some optional implementations, the first recognition model includes multiple abbreviated filters. Correspondingly, embodiments of this application provide a first feature extraction method, such as... Figure 4 As shown, it includes the following steps:
[0093] Step S401: For any shrinking filter, shrink the initial image tensor with the shrinking filter to obtain the mixing matrix corresponding to the shrinking filter.
[0094] Specifically, by using the training tensor in the first recognition model, the initial image tensor is subjected to a corresponding shrinking operation, thereby reducing the data order of the initial image tensor and eliminating redundant parameters while preserving the structural information of the initial image tensor, thus reducing the training parameters required by the first recognition model.
[0095] In this embodiment of the application, for ease of description, each training tensor that performs a shrinking operation with the initial image tensor is regarded as a shrinking filter in the corresponding multidimensional tensor shrinking operation layer. As can be seen from the basic operation definition of MTCOL above, in order to achieve parametric shrinking of the initial image tensor under a specific axis, each shrinking filter should be at the same data order as the initial image tensor, and each shrinking filter should contain at least one data index that is the same as the initial image tensor. Based on the shrinking operation performed by each set shrinking filter and the initial image tensor, the initial image tensor is shrunk into the corresponding mixing matrix in each of its specified data axes.
[0096] Understandably, see Figure 5 As shown, since tensor shrinking is represented as the sum of all data of two different tensors on the same data index, each shrinking filter in this embodiment can calculate the representation of each position by the weighted sum of all positional elements. Unlike convolutional filters using local receptive fields, the above method enables the shrinking filter to have a global receptive field, so that each shrinking filter can extract all identifiable information contained in the image to be identified at the same time during the shrinking operation of the initial image tensor, ensuring the accuracy of the extracted first feature.
[0097] Specifically, the shrinking operation of the initial image tensor by the above-mentioned shrinking filter can be based on the Einstein summation convention, which is characterized by summing the initial image tensor with the same data index of a shrinking filter. The above summation convention is as follows:
[0098]
[0099] Furthermore, in an optional embodiment, in order to make the extracted target features richer, the initial image tensor can be shrunk based on M preset shrinking filters, so as to extract more distinctive multi-angle first features of the target object under multiple different angles and different views, where M is an integer greater than or equal to one.
[0100] Step S402: Based on the preset feature order, aggregate all the obtained mixing matrices to determine the first feature of the target object.
[0101] Furthermore, in order to preserve the original structural information of the image to be identified, in this embodiment, the higher-order tensor obtained by aggregating the above M mixing matrices according to the specified data order is used as the first feature of the target recognition model for the target object; the above method preserves the structural information and multilinear information of the initial image tensor, ensuring that the first feature with integrated distinguishing information is extracted.
[0102] See Figure 6 The diagram illustrates the merging process of the initial image tensor and the set M merging filters. As shown in the diagram, based on the merging operation performed on each merging filter and the initial image tensor, the mixing matrix corresponding to each specified data axis of the initial image tensor is obtained. It can be understood that since the data between each pair of mixing matrices is usually similar, in this embodiment, the obtained multiple mixing matrices are aggregated according to the specified data order to further obtain the first distinctive feature of the target object under multiple angles and multiple views.
[0103] In some optional implementations, the second recognition model includes multiple convolutional kernels. Correspondingly, embodiments of this application provide a second feature extraction method, such as... Figure 7 As shown, it includes the following steps:
[0104] Step S701: For any convolution kernel, perform local feature extraction and pooling operations on the corresponding pixels in the initial image tensor based on the convolution kernel to determine the local features corresponding to the convolution kernel;
[0105] Step S702: Perform a fully connected operation on the local features corresponding to all convolution kernels to determine the second feature of the target object.
[0106] The second recognition model includes multiple convolutional kernels with different local receptive fields. After the initial image tensor is input into the second recognition model, each convolutional kernel (also called a filter) extracts local features from the initial image tensor according to its set stride to determine the features corresponding to each convolutional kernel.
[0107] Pooling operations are performed on the features corresponding to each convolution kernel (such as calculating the maximum or average value of the features corresponding to the convolution kernel) to obtain the local features corresponding to each convolution kernel.
[0108] Perform a fully connected operation on the local features corresponding to all convolution kernels to convert the tensors into vectors and obtain the second feature of the target object.
[0109] This application provides a second image feature extraction method, such as... Figure 8 As shown, it includes the following steps:
[0110] Step S801: Transform the image to be identified containing the target object to obtain the initial image tensor corresponding to the image to be identified;
[0111] Step S802: Based on a preset first recognition model, tensor shrunk the initial image tensor to determine the first feature of the target object; and based on a preset second recognition model, local feature extraction is performed on the initial image tensor to determine the second feature of the target object.
[0112] Step S803: Based on the first feature and the second feature, determine the target feature of the target object.
[0113] The specific implementation of steps S801 to S803 can be found in the above embodiments, and will not be repeated here.
[0114] Step S804: Match the target feature with each preset feature information, and determine the identity information of the target object based on the matching result.
[0115] For example, this embodiment sets up a feature information database, which contains the correspondence between preset feature information of multiple preset objects and the identity information of preset objects; the target features are matched with each preset feature information, and the identity information of the target object is determined based on the matching results.
[0116] In some optional implementations, the similarity between the target feature and each preset feature information is determined, and the identity information corresponding to the preset feature information with the highest similarity is determined as the identity information of the target object.
[0117] In practice, there may not be any preset feature information that matches the target feature information well. Based on this, after selecting the preset feature information with the highest similarity from the preset feature information, it is first determined that the similarity between the target feature and the preset feature information reaches the similarity threshold. Then, the identity information corresponding to the preset feature information is determined as the identity information of the target object. Otherwise, it is determined that the identity recognition of the target object has failed.
[0118] The method for determining the preset feature information is similar to the process for determining the target features described above, and will not be repeated here.
[0119] The above scheme, after determining the target features that accurately represent the target object, accurately and efficiently determines the identity information of the target object by matching the target features with preset feature information.
[0120] Based on the same inventive concept, this application provides an image feature extraction device. Referring to FIG9, the image feature extraction device 900 includes:
[0121] Tensor determination module 901 is used to transform the image to be identified containing the target object to obtain the initial image tensor corresponding to the image to be identified;
[0122] The feature determination module 902 is used to perform tensor simplification on the initial image tensor based on a preset first recognition model to determine the first feature of the target object; and to perform local feature extraction on the initial image tensor based on a preset second recognition model to determine the second feature of the target object.
[0123] The feature combination module 903 is used to determine the target features of the target object based on the first feature and the second feature.
[0124] In some optional implementations, the feature combination module 903 is specifically used for:
[0125] The first feature and the second feature are combined to obtain the target feature of the target object; wherein the number of features of the target feature is the sum of the number of features of the first feature and the number of features of the second feature.
[0126] In some optional implementations, the first recognition model includes multiple abbreviated filters; the feature determination module 902 is specifically used for:
[0127] For any shrink filter, the initial image tensor is shrunk with the shrink filter to obtain the mixing matrix corresponding to the shrink filter;
[0128] Based on a preset feature order, all obtained mixing matrices are aggregated to determine the first feature of the target object.
[0129] In some optional implementations, the feature determination module 902 is specifically used for:
[0130] The initial image tensor is summed with the preset data index corresponding to the shrinking filter to obtain the mixing matrix corresponding to the shrinking filter.
[0131] In some optional implementations, the second recognition model includes multiple convolutional kernels; the feature determination module 902 is specifically used for:
[0132] For any convolution kernel, local feature extraction and pooling operations are performed on the corresponding pixels in the initial image tensor based on the convolution kernel to determine the local features corresponding to the convolution kernel;
[0133] A fully connected operation is performed on the local features corresponding to all convolutional kernels to determine the second feature of the target object.
[0134] In some optional embodiments, the image feature extraction device 900 further includes an identity recognition module 904, used for:
[0135] The target feature is matched with each preset feature information, and the identity information of the target object is determined based on the matching result.
[0136] Since this device is the same as the device in the method of this application embodiment, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described again.
[0137] Based on the same technical concept, this application also provides an electronic device 1000, as shown in FIG10, including at least one processor 1001 and a memory 1002 connected to the at least one processor. This application does not limit the specific connection medium between the processor 1001 and the memory 1002. Figure 10 Taking the connection between processor 1001 and memory 1002 via bus 1003 as an example. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0138] The processor 1001 is the control center of the electronic device, capable of connecting various parts of the electronic device via various interfaces and lines. It performs data processing by running or executing instructions stored in the memory 1002 and accessing data stored in the memory 1002. Optionally, the processor 1001 may include one or more processing units. The processor 1001 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles issuing instructions. It is understood that the modem processor may not be integrated into the processor 1001. In some embodiments, the processor 1001 and the memory 1002 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.
[0139] The processor 1001 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the image feature extraction method can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0140] Memory 1002, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1002 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 1002 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 1002 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0141] In this embodiment, the memory 1002 stores a computer program, which, when executed by the processor 1001, causes the processor 1001 to perform the following:
[0142] The image to be identified, which contains the target object, is transformed to obtain the initial image tensor corresponding to the image to be identified;
[0143] Based on a preset first recognition model, tensor simplification is performed on the initial image tensor to determine the first feature of the target object; and based on a preset second recognition model, local feature extraction is performed on the initial image tensor to determine the second feature of the target object.
[0144] Based on the first feature and the second feature, the target features of the target object are determined.
[0145] In some optional implementations, processor 1001 specifically performs:
[0146] The first feature and the second feature are combined to obtain the target feature of the target object; wherein the number of features of the target feature is the sum of the number of features of the first feature and the number of features of the second feature.
[0147] In some optional implementations, the first identification model includes multiple abbreviated filters; the processor 1001 specifically executes:
[0148] For any shrink filter, the initial image tensor is shrunk with the shrink filter to obtain the mixing matrix corresponding to the shrink filter;
[0149] Based on a preset feature order, all obtained mixing matrices are aggregated to determine the first feature of the target object.
[0150] In some optional implementations, processor 1001 specifically performs:
[0151] The initial image tensor is summed with the preset data index corresponding to the shrinking filter to obtain the mixing matrix corresponding to the shrinking filter.
[0152] In some optional implementations, the second recognition model includes multiple convolutional kernels; processor 1001 specifically executes:
[0153] For any convolution kernel, local feature extraction and pooling operations are performed on the corresponding pixels in the initial image tensor based on the convolution kernel to determine the local features corresponding to the convolution kernel;
[0154] A fully connected operation is performed on the local features corresponding to all convolutional kernels to determine the second feature of the target object.
[0155] In some optional implementations, processor 1001 also performs:
[0156] The target feature is matched with each preset feature information, and the identity information of the target object is determined based on the matching result.
[0157] Since the electronic device is the same as the electronic device in the method of this application embodiment, and the principle of the electronic device in solving the problem is similar to that of the method, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be described again.
[0158] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing a computer program executable by a computer, which, when run on the computer, causes the computer to perform the steps of the above-described image feature extraction method.
[0159] In some alternative implementations, various aspects of the image feature extraction method provided in this application can also be implemented as a program product containing computer-executable instructions. When the program product is run on a computer device, the computer-executable instructions are used to cause the computer device to perform the steps of the image feature extraction method according to the various exemplary embodiments of this application described above.
[0160] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0161] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0162] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0163] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0164] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0165] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. An image feature extraction method, characterized in that, include: The image to be identified, which contains the target object, is transformed to obtain the initial image tensor corresponding to the image to be identified; Based on a preset first recognition model, tensor simplification is performed on the initial image tensor to determine the first feature of the target object; and based on a preset second recognition model, local feature extraction is performed on the initial image tensor to determine the second feature of the target object. Based on the first feature and the second feature, the target features of the target object are determined; The first recognition model includes multiple abbreviated filters; Based on a preset first recognition model, tensor simplification is performed on the initial image tensor to determine the first feature of the target object, including: For any shrink filter, the initial image tensor is shrunk with the shrink filter to obtain the mixing matrix corresponding to the shrink filter; Based on a preset feature order, all obtained mixing matrices are aggregated to determine the first feature of the target object.
2. The method as described in claim 1, characterized in that, Based on the first feature and the second feature, the target features of the target object are determined, including: The first feature and the second feature are combined to obtain the target feature of the target object; wherein the number of features of the target feature is the sum of the number of features of the first feature and the number of features of the second feature.
3. The method as described in claim 1, characterized in that, The initial image tensor is shrunk with the shrinking filter to obtain the blending matrix corresponding to the shrinking filter, including: The initial image tensor is summed with the preset data index corresponding to the shrinking filter to obtain the mixing matrix corresponding to the shrinking filter.
4. The method as described in claim 1, characterized in that, The second recognition model includes multiple convolutional kernels; based on the preset second recognition model, local feature extraction is performed on the initial image tensor to determine the second feature of the target object, including: For any convolution kernel, local feature extraction and pooling operations are performed on the corresponding pixels in the initial image tensor based on the convolution kernel to determine the local features corresponding to the convolution kernel; A fully connected operation is performed on the local features corresponding to all convolutional kernels to determine the second feature of the target object.
5. The method according to any one of claims 1-4, characterized in that, Also includes: The target feature is matched with each preset feature information, and the identity information of the target object is determined based on the matching result.
6. An image feature extraction device, characterized in that, The device includes: The tensor determination module is used to transform the image to be identified, which contains the target object, to obtain the initial image tensor corresponding to the image to be identified. The feature determination module is used to perform tensor simplification on the initial image tensor based on a preset first recognition model to determine the first feature of the target object; and to perform local feature extraction on the initial image tensor based on a preset second recognition model to determine the second feature of the target object. A feature combination module is used to determine the target features of the target object based on the first feature and the second feature; The first recognition model includes multiple abbreviated filters; the feature determination module is specifically used for: For any shrink filter, the initial image tensor is shrunk with the shrink filter to obtain the mixing matrix corresponding to the shrink filter; Based on a preset feature order, all obtained mixing matrices are aggregated to determine the first feature of the target object.
7. An electronic device, characterized in that, It includes at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer, which, when run on the computer, causes the computer to perform the method as described in any one of claims 1 to 5.
9. A computer program product, characterized in that, It includes computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 5.