Image feature extraction method, device, equipment and storage medium

CN112215243BInactive Publication Date: 2024-11-29BAIDU (CHINA) CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202011196627.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-10-30
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing technologies suffer from insufficient feature accuracy in image feature extraction, which affects the results of subsequent image processing.

Method used

By segmenting the image to be processed using the associated segmentation features, target extraction features and features to be transferred are obtained. The features to be transferred are then used to update subsequent segmentation features to help determine the target extraction features of subsequent segmentation features, thereby realizing information transfer between channels and feature reuse.

Benefits of technology

It improves the accuracy of image feature extraction, enriches the receptive field of subsequent segmentation features, increases the information content of feature extraction results, and reduces the amount of data computation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112215243B_ABST
    Figure CN112215243B_ABST
Patent Text Reader

Abstract

The present application discloses an image feature extraction method, device, equipment and storage medium, which relate to artificial intelligence fields such as computer vision and deep learning. The specific implementation scheme is: segment the current segmentation feature of at least two segmentation features associated with the image to be processed, and obtain the target extraction feature and the feature to be transferred of the current segmentation feature; wherein the feature to be transferred of the current segmentation feature is used to update the subsequent segmentation feature after the current segmentation feature, and determine the target extraction feature of the subsequent segmentation feature; according to the target extraction feature of at least two segmentation features, the feature extraction result of the image to be processed is obtained. The embodiment of the present application improves the accuracy of image feature extraction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, in particular to artificial intelligence fields such as computer vision and deep learning, and specifically to an image feature extraction method, device, equipment and storage medium. Background Art

[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] When performing image processing based on artificial intelligence technology, in order to improve the accuracy of image processing results, it is usually necessary to first perform feature extraction on the image to be processed. When performing image feature extraction in the prior art, there is a problem of insufficient accuracy of the extracted features, which has a certain impact on the subsequent image processing results. Summary of the invention

[0004] The present application provides an image feature extraction method, device, equipment and storage medium with better image feature extraction accuracy.

[0005] According to one aspect of the present application, there is provided an image feature extraction method, comprising:

[0006] Perform segmentation processing on a current segmentation feature of at least two segmentation features associated with the processed image to obtain a target extraction feature and a feature to be transferred of the current segmentation feature; wherein the feature to be transferred of the current segmentation feature is used to update a subsequent segmentation feature located after the current segmentation feature, and to determine a target extraction feature of the subsequent segmentation feature;

[0007] A feature extraction result of the image to be processed is obtained by extracting features according to the target of the at least two segmentation features.

[0008] According to another aspect of the present application, there is also provided an image feature extraction device, comprising:

[0009] A segmentation processing module, used for performing segmentation processing on a current segmentation feature of at least two segmentation features associated with the processed image, and obtaining a target extraction feature and a feature to be transferred of the current segmentation feature; wherein the feature to be transferred of the current segmentation feature is used to update a subsequent segmentation feature located after the current segmentation feature, and to determine a target extraction feature of the subsequent segmentation feature;

[0010] The feature extraction result obtaining module is used to extract features according to the target of the at least two segmentation features to obtain the feature extraction result of the image to be processed.

[0011] According to another aspect of the present application, an electronic device is also provided, including:

[0012] at least one processor; and

[0013] a memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the image feature extraction methods provided in the embodiments of the present application.

[0015] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to enable the computer to execute any one of the image feature extraction methods provided in the embodiments of the present application.

[0016] According to the technical solution of the present application, the accuracy of image feature extraction is improved.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0019] Figure 1 is a flow chart of an image feature extraction method provided in an embodiment of the present application;

[0020] Figure 2 is a flow chart of another image feature extraction method provided in an embodiment of the present application;

[0021] Figure 3A is a flow chart of another image feature extraction method provided in an embodiment of the present application;

[0022] Figure 3B It is a structural schematic diagram of a plug-and-play neural network model provided in an embodiment of the present application;

[0023] Figure 4 is a structural diagram of an image feature extraction device provided in an embodiment of the present application;

[0024] Figure 5It is a block diagram of an electronic device used to implement the image feature extraction method of the embodiment of the present application. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0026] The image feature extraction methods and image feature extraction devices provided in the embodiments of the present application are applicable to the case where image feature extraction is performed during image classification, target detection, target tracking or target segmentation. The image feature extraction methods provided in the embodiments of the present application can be performed by an image feature extraction device, which is implemented by software and / or hardware and is specifically configured in an electronic device.

[0027] See also Figure 1 An image feature extraction method shown includes:

[0028] S101, performing segmentation processing on a current segmentation feature among at least two segmentation features associated with an image to be processed, and obtaining a target extraction feature and a feature to be transferred of the current segmentation feature; wherein the feature to be transferred of the current segmentation feature is used to update a subsequent segmentation feature located after the current segmentation feature, and to determine a target extraction feature of the subsequent segmentation feature.

[0029] The image to be processed may be image data acquired by an image acquisition device, or may be a picture frame obtained by performing frame extraction processing on video data.

[0030] Optionally, the image to be processed can be pre-stored locally in the electronic device, in other storage devices associated with the electronic device, or in the cloud. When it is necessary to extract features from the image to be processed, the image to be processed is searched and acquired based on its identification information, and segmentation features are determined based on the image to be processed.

[0031] In order to reduce the amount of data calculation in the process of image feature extraction for the image to be processed, optionally, at least two segmentation features associated with the image to be processed can be pre-stored locally in the electronic device, in other storage devices associated with the electronic device, or in the cloud, and when feature extraction is required for the image to be processed, the segmentation features can be searched and acquired based on the identification information of the image to be processed.

[0032] Exemplarily, the segmentation features associated with the image to be processed may be determined in the following manner: segmenting the image to be processed according to the number of channels of the image to be processed, and extracting features from at least two segmentation results to obtain at least two corresponding segmentation features.

[0033] Exemplarily, the segmentation features associated with the image to be processed may also be determined in the following manner: extracting features from the image to be processed, and segmenting the feature extraction results according to the number of channels of the image to be processed to obtain at least two segmentation features.

[0034] The current segmentation feature may be understood as a segmentation feature that performs a segmentation processing operation among at least two segmentation features.

[0035] In an optional embodiment, a current segmentation feature among at least two segmentation features associated with the image to be processed is segmented to obtain a target extraction feature and a feature to be transferred of the current segmentation feature. This may be: a current segmentation feature among at least two segmentation features associated with the image to be processed is randomly segmented to obtain a target extraction feature and a feature to be transferred of the current segmentation feature.

[0036] In order to realize the module integration of the image feature extraction process and the flexible use of the integrated modules, in another optional embodiment, the current segmentation feature of at least two segmentation features associated with the image to be processed is segmented to obtain the target extraction feature and the feature to be transferred of the current segmentation feature. It can also be: according to the segmentation width and / or segmentation scale, combined with the number of channels of the image to be processed, the feature extraction result is segmented to obtain at least two segmentation features. Among them, the segmentation width can be understood as the number of channels corresponding to each segmentation feature obtained; the segmentation scale can be understood as the number of segmentation features finally obtained. Among them, the segmentation width and / or segmentation scale can be flexibly set by the technician according to the needs.

[0037] In order to achieve module integration of the image feature extraction process and weaken the traces of artificial construction in the integrated feature extraction module, while facilitating the subsequent processing of the target extraction features and the features to be transferred, in another optional embodiment, the current segmentation feature of the at least two segmentation features associated with the processed image is segmented to obtain the target extraction features and the features to be transferred of the current segmentation feature. It can also be: the current segmentation feature of the at least two segmentation features associated with the processed image is evenly segmented to obtain the target extraction features and the features to be transferred of the current segmentation feature.

[0038] Optionally, after segmenting the current segmentation feature, part of the segmentation results are used as target extraction features, and at least part of the segmentation results other than the target extraction features are used as features to be transferred. The number of segmentation results is at least two. In one embodiment, the target extraction features and the features to be transferred can be randomly selected from each segmentation result, or can be selected according to a set rule. The set rule can be to select the target extraction features and / or the features to be transferred in order according to the channel arrangement order of the segmentation results in the current segmentation feature.

[0039] The number of target extraction features of the current segmentation feature is at least one. It is understandable that, in order to facilitate the subsequent calculation of feature extraction results of the image to be processed, usually, the number of target extraction features of the current segmentation feature is 1.

[0040] Among them, the number of features to be transferred of the current segmentation feature is at least one. It is understandable that the number of features to be transferred can be determined by technicians according to needs or experience, or repeatedly determined through a large number of experiments. For example, the number of features to be transferred of the current segmentation feature can be set to 1.

[0041] In one embodiment, the feature to be transferred of the current segmentation feature is used to update the subsequent segmentation feature located after the current segmentation feature, and to determine the target extraction feature of the subsequent segmentation feature.

[0042] The subsequent segmentation feature can be understood as a segmentation feature that is located after the current segmentation feature in the channel sequence. The number of subsequent segmentation features can be at least one, that is, the feature to be transferred of the current segmentation feature can update at least one subsequent segmentation feature that is located after the current segmentation feature.

[0043] In order to facilitate the update operation of at least one segmentation feature in the image to be processed, a segmentation feature that is located after the current segmentation feature and adjacent to the current segmentation feature is typically used as a subsequent segmentation feature.

[0044] Specifically, the current segmentation feature is segmented to obtain the target extraction feature and the feature to be transferred of the current segmentation feature; the subsequent segmentation feature after the current segmentation feature is updated according to the feature to be transferred of the current segmentation feature, and the subsequent segmentation feature is used as the new current segmentation feature, and the segmentation processing operation is returned to be performed, and the cycle is repeated to obtain the target extraction feature of each segmentation feature.

[0045] S102, extracting features according to the targets of at least two segmentation features to obtain feature extraction results of the image to be processed.

[0046] Optionally, obtaining feature extraction results of the image to be processed based on target extraction features of at least two segmentation features may be: obtaining feature extraction results of the image to be processed based on target extraction features of at least two segmentation features and / or features to be transferred.

[0047] In order to take into account both the accuracy of the feature extraction results and the amount of data calculation, optionally, the feature extraction results of the image to be processed are obtained based on the target extraction results of at least two segmentation features. It can also be: based on the target extraction results of at least two segmentation features and the features to be transferred at the tail segmentation features, the feature extraction results of the image to be processed are obtained.

[0048] The embodiment of the present application realizes information transmission between channels by introducing the feature to be transferred of the current segmentation feature, updating the subsequent segmentation feature after the current segmentation feature, and assisting in determining the target extraction feature of the subsequent segmentation feature, thereby enriching the receptive field of the subsequent segmentation feature, increasing the amount of information of the target extraction feature in the subsequent segmentation feature, and thus improving the feature extraction accuracy of the feature extraction result of the image to be processed. At the same time, by introducing the target extraction feature of the current segmentation feature, feature reuse is realized, thereby improving the feature extraction accuracy of the feature extraction result of the image to be processed.

[0049] On the basis of the above technical solutions, in order to reduce the amount of data calculation when determining the target extraction features of each segmentation feature, data dimension reduction processing can also be performed in the process of determining the segmentation features associated with the image to be processed.

[0050] Exemplarily, before the current segmentation feature of at least two segmentation features associated with the image to be processed is segmented, the image to be processed may be subjected to dimensionality reduction processing to obtain the feature to be processed; the feature to be processed is segmented according to the number of channels of the feature to be processed to obtain at least two segmentation features. The segmentation width and the segmentation scale in the segmentation process may be determined by the technician according to the needs or experience, or may be determined repeatedly through a large number of experiments. Exemplarily, the segmentation process may be a uniform segmentation process, that is, the segmentation widths of the segmentation features are the same.

[0051] Since the segmentation features of the image to be processed are different from the dimension of the image to be processed, that is, the number of channels, in order to ensure that the final feature extraction result of the image to be processed is the same as the latitude of the image to be processed, data dimensionality upgrading is also required when determining the feature extraction result of the image to be processed.

[0052] Exemplarily, obtaining the feature extraction result of the image to be processed according to the target extraction features of the at least two segmentation features may be: performing dimensionality up-scaling processing on the target extraction features of the at least two segmentation features to obtain the feature extraction result of the image to be processed.

[0053] In order to solve the problem of gradient divergence in the process of image feature extraction, in an optional embodiment, after extracting features based on the target of at least two segmentation features and obtaining the feature extraction result of the image to be processed, the feature extraction result can also be corrected, that is: based on the image to be processed, the feature extraction result of the image to be processed is updated.

[0054] Exemplarily, a shortcut structure may be used to process the image to be processed to obtain features to be fused; the features to be fused are fused with feature extraction results of the image to be processed to update the feature extraction results of the image to be processed.

[0055] Optionally, a direct connection structure is used to process the image to be processed to obtain the features to be fused, and the image to be processed can be directly used as the feature to be fused; or optionally, a set convolution kernel is used to extract features of the image to be processed to obtain the features to be fused; or optionally, a set pooling function is used to downsample the image to be processed; the downsampled result is upsampled by setting the convolution kernel to obtain the features to be fused. Among them, the setting of the convolution kernel and / or the setting of the pooling function can be determined by the technician according to the needs or experience, or repeatedly determined through a large number of experiments. For example, the setting of the convolution kernel can use 1*1 convolution, and the setting of the pooling function can use the average pooling function.

[0056] It should be noted that the specific direct connection structure to be used to determine the features to be fused can be repeatedly determined through a large number of experiments during the training phase of the integrated feature extraction model (such as a neural network model).

[0057] On the basis of the above-mentioned technical solutions, before the operation of "segmenting the current segmentation feature among at least two segmentation features associated with the image to be processed", it is also possible to add "update the current segmentation feature according to the feature to be transferred of the previous segmentation feature before the current segmentation feature" to realize the update operation of the current segmentation feature, thereby realizing the transfer of the previous segmentation feature to the current segmentation feature.

[0058] See also Figure 2 An image feature extraction method shown includes:

[0059] S201. For a current segmentation feature among at least two segmentation features associated with an image to be processed, update the current segmentation feature according to a feature to be transferred of a previous segmentation feature that precedes the current segmentation feature.

[0060] The preceding segmentation feature may be at least one segmentation feature located before the current segmentation feature. Accordingly, the current segmentation feature is updated through the features to be transferred of the preceding segmentation features.

[0061] In order to simplify the amount of data calculation in the image feature extraction process of the image to be processed, the previous segmentation feature is typically set to the segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature. That is, the feature to be transferred of the previous segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature is extracted to obtain the target transfer feature; according to the target transfer feature, the current segmentation feature is updated.

[0062] The number of features to be transferred of the previous segmentation feature is at least one, so that feature data of different channels can be introduced into the current segmentation feature as needed.

[0063] When the number of features to be transferred of the previous segmentation feature is at least two, in order to enrich the receptive field size in the target transfer feature and thereby increase the receptive field size in the current segmentation feature, in an optional embodiment, feature extraction is performed on the features to be transferred of the previous segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature to obtain the target transfer feature, which can be: convolution processing is performed on each feature to be transferred of the previous segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature to obtain the target transfer feature; wherein the sizes of the convolution kernels used in the convolution processing are different.

[0064] It should be noted that the number of features to be transferred corresponding to different segmentation features in the image to be processed may be the same or different, and the sizes of the convolution kernels used may be the same or different.

[0065] In an optional embodiment, updating the current segmentation feature according to the target transfer feature may be: integrating the target transfer feature with the current segmentation feature to update the current segmentation feature.

[0066] It can be understood that by integrating the target transfer features of the previous segmentation features and the current segmentation features, an updated current segmentation feature is obtained, so that the updated current segmentation feature contains information from different channels at different scales, thereby strengthening the information exchange between channels, thereby increasing the nonlinear factors in the subsequently determined target extraction features, laying the foundation for the comprehensiveness of the feature extraction results of the final image to be processed, and thus improving the accuracy of the feature extraction results.

[0067] For example, a set convolution kernel may be used to integrate the target transfer feature and the current segmentation feature, wherein the size of the set convolution kernel may be determined by a technician according to needs or experience, or may be determined repeatedly through a large number of experiments. For example, the set convolution kernel may be a 1×1 convolution.

[0068] S202, performing segmentation processing on the current segmentation feature to obtain the target extraction feature and the feature to be transferred of the current segmentation feature; wherein the feature to be transferred of the current segmentation feature is used to update the subsequent segmentation feature after the current segmentation feature, and to determine the target extraction feature of the subsequent segmentation feature.

[0069] Specifically, the current segmentation feature is segmented to obtain the target extraction feature and the feature to be transferred of the current segmentation feature; the subsequent segmentation feature after the current segmentation feature is updated according to the feature to be transferred of the current segmentation feature, and the subsequent segmentation feature is used as the new current segmentation feature, and the segmentation processing operation is returned to be performed, and the cycle is repeated to obtain the target extraction feature of each segmentation feature.

[0070] Since there is no subsequent segmentation feature at the tail segmentation feature, it is impossible to update the subsequent segmentation feature and determine the target extraction feature of the subsequent segmentation feature after the feature to be transferred at the tail segmentation feature. Subsequently, the feature extraction result of the image to be processed can be determined based on the target extraction features of each segmentation feature that associates the feature to be transferred at the tail segmentation feature with the image to be processed.

[0071] However, since the to-be-transferred feature of the segmentation feature at the end has lost its own feature transfer function, processing the segmentation feature at the end in the same manner as other segmentation features except the segmentation feature at the end will result in a waste of computing resources. In order to avoid the above situation, when the segmentation feature at the end is the current segmentation feature, that is, when the current segmentation feature does not have a subsequent segmentation feature, there is no need to perform segmentation processing, and the target extraction feature of the current segmentation feature is directly determined.

[0072] Exemplarily, if the current segmentation feature does not have a subsequent segmentation feature, then the current segmentation feature among the at least two segmentation features associated with the processed image is segmented to obtain the target extraction feature of the current segmentation feature and the feature to be transferred, which can be: feature extraction is performed on the current segmentation feature to obtain the target extraction feature of the current segmentation feature.

[0073] Optionally, the set convolution kernel can be directly used to extract the current segmentation feature (the segmentation feature at the end) to obtain the target extraction feature of the current segmentation feature. The set convolution kernel can be determined by the technician according to the needs or experience, or repeatedly determined through a large number of experiments. It should be noted that the set convolution kernel involved here can be the same or different from the set convolution kernel category involved in the previous text, and the convolution kernel size can be the same or different.

[0074] S203: Extract features according to the targets of at least two segmentation features to obtain feature extraction results of the image to be processed.

[0075] Exemplarily, the target extracted features of at least two segmented features are integrated, and the integrated features are used as feature extraction results of the image to be processed.

[0076] The embodiment of the present application updates the current segmentation feature by adding a feature to be transferred based on a previous segmentation feature located before the current segmentation feature before segmenting the current segmentation feature of at least two segmentation features associated with the image to be processed, so as to improve the update operation of the current segmentation feature, thereby introducing the feature to be transferred of the previous segmentation feature into the current segmentation feature to assist in determining the target extraction feature of the current segmentation feature, realizing the exchange of information between channels, thereby enriching the receptive field of subsequent segmentation features, increasing the amount of information of the target extraction feature in the subsequent segmentation features, and further improving the feature extraction accuracy of the feature extraction result of the image to be processed.

[0077] On the basis of the above-mentioned technical solutions, in an optional embodiment of the present application, a preferred implementation method for image feature extraction is provided.

[0078] See also Figure 3A An image feature extraction method shown includes:

[0079] S301, performing dimensionality reduction processing on the image to be processed to obtain features to be processed, and performing segmentation processing on the features to be processed to obtain at least two segmentation features.

[0080] See also Figure 3B A structural schematic diagram of a plug-and-play neural network model is shown and described in detail.

[0081] The dimension reduction process can be implemented by 1*1 convolution to obtain the features to be processed. The segmentation process can be performed by technicians to set the segmentation width and segmentation scale as needed. Figure 3B The segmentation scale is set to 4 for exemplary description, and segmentation features X1, X2, X3 and X4 are obtained, which should not be understood as limiting the segmentation scale of the present application. In addition, Figure 3B The slicing process is performed by uniform slicing, and other slicing methods can be used as needed. The embodiments of the present application do not impose any limitation on this.

[0082] S302: sequentially obtain one of the segmentation features as the current segmentation feature.

[0083] S303, determine whether the current segmentation feature is the first segmentation feature; if not, execute S304; otherwise, execute S307.

[0084] S304: Obtain a target transfer feature of a previous segmentation feature that is immediately adjacent to the current segmentation feature.

[0085] S305 , extracting the to-be-transferred feature of the previous segmentation feature that is immediately adjacent to the current segmentation feature to obtain a target transfer feature.

[0086] S306: Integrate the target transfer feature with the current segmentation feature to update the current segmentation feature.

[0087] S307, determine whether the current segmentation feature is the segmentation feature at the end; if not, continue to execute S308; otherwise, execute S310;

[0088] S308 , split the current segmented feature for a second time to obtain the target extracted feature and the feature to be transferred of the current segmented feature.

[0089] S309, extracting the features to be transferred of the current segmentation features to obtain target transfer features of the current segmentation features; and returning to execute S302.

[0090] S310, extracting features from the current segmentation features to obtain target extracted features of the current segmentation features. Then proceed to S311.

[0091] Continue to see Figure 3B The feature extraction process is described in detail.

[0092] If the current segmentation feature is X1, then segment X1 to obtain the segmentation result X 11 and X 12 ; For the segmentation result X 11 Without any processing, directly extract feature y1 as the target of X1; 12 As the feature to be transferred; 12 For feature extraction (for example, 3×3 convolution can be used to extract X 12 Perform feature extraction) and convert X 12 The feature extraction result of is integrated with X2 (for example, 1×1 convolution can be used for feature integration), so as to update X2.

[0093] It is determined that X2 is not the segmentation feature at the end, so the updated X2 is segmented twice to obtain the segmentation result X 21 and X 22 ; For the segmentation result X 21 Without any processing, directly extract feature y2 as the target of X2; 22 As the feature to be transferred; 22 For feature extraction (for example, 3×3 convolution can be used to extract X 22 Perform feature extraction) and convert X 22 The feature extraction results of are integrated with X3 (for example, 1×1 convolution can be used for feature integration) to update X3.

[0094] It is determined that X3 is not the segmentation feature at the end, so the updated X3 is segmented twice to obtain the segmentation result X 31 and X 32 ; For the segmentation result X 31 Without any processing, directly extract feature y3 as the target of X3; 32 As the feature to be transferred; 32 For feature extraction (for example, 3×3 convolution can be used to extract X 32 Perform feature extraction) and convert X 22 The feature extraction results of X3 are integrated with features (for example, 1×1 convolution can be used for feature integration) to update X4.

[0095] Determine X4 as the segmentation feature at the end. At this time, the updated X4 is no longer segmented twice, and feature extraction is performed directly on the updated X4 to obtain the target extraction feature y4 of X4.

[0096] It should be noted that for each segmentation feature X i The feature X to be transferred in (i=1,2,3) i2 When performing feature extraction, the convolution kernel sizes used can be the same or different. The specific sizes can be determined by technicians based on needs or experience, or can be determined repeatedly through a large number of experiments.

[0097] Figure 3B Split feature X by pair i (i=1,2,3) is used as an example to perform a secondary segmentation operation with a segmentation scale of 2. It is understandable that the segmentation scale can also be set to be greater than 2, such as 3 or 4, as needed. It should be noted that when the segmentation scale is greater than 2, the segmentation feature X i The feature X to be transferred i2 The number of is at least 2. Accordingly, for each feature X to be transferred, i2 Perform feature extraction and transform each X i2 The feature extraction results and subsequent segmentation features (X i+1 ) to perform feature integration. i2 The convolution kernel scales used for feature extraction can be the same or different.

[0098] S311, performing feature dimension upgrading processing on the target extracted features of each segmentation feature to obtain a feature extraction result of the image to be processed.

[0099] Since S302 performs dimensionality reduction processing on the image to be processed, in order to make the feature extraction result of the image to be processed consistent with the number of channels of the image to be processed, it is also necessary to perform feature dimensionality increase processing on the target extracted features of each segmentation feature. Exemplarily, 1×1 convolution can be used for feature dimensionality increase processing.

[0100] Continue to see Figure 3B , for the segmentation feature X i (i=1,2,3,4) target extraction feature y i (i=1,2,3,4) perform feature dimension upgrading and integration to obtain feature extraction results.

[0101] S312, processing the image to be processed through the direct connection structure to obtain the features to be fused.

[0102] Among them, the direct connection structure can directly use the image to be processed as the feature to be fused, or can perform feature extraction on the image to be processed (for example, 1*1 convolution can be used for feature extraction) to obtain the feature to be fused; or it can also downsample the image to be processed by setting a pooling function (for example, an average pooling function can be used for downsampling), and upsample the downsampled result (for example, 1*1 convolution can be used for upsampling) to obtain the feature to be fused.

[0103] S313: Fusing the features to be fused with the feature extraction results of the image to be processed to update the feature extraction results.

[0104] S314: performing image classification, target detection or target segmentation on the image to be processed according to the feature extraction result of the image to be processed.

[0105] The image feature extraction process of the present application can be integrated into a plug-and-play neural network model - image feature extraction model, to facilitate migration or secondary development. Compared with the existing ResNet (Deep residual network) model, the image feature extraction module function does not simply perform convolution processing on the input features. Without increasing the amount of calculation, the input image to be processed is converted into segmentation features of multiple scales. The segmentation width and segmentation scale can be flexibly set, which is more versatile and improves the feature extraction accuracy.

[0106] Compared with the existing ResNeXt model, it does not use group convolution operations, reduces latency, and enhances information exchange between channels by fusing features at different scales, thereby improving feature extraction accuracy.

[0107] Compared with the existing Res2Net, the target extraction features and the features to be transferred are obtained by secondary segmentation of the segmentation features of different scales. The target extraction features realize feature reuse; the features to be transferred realize the transfer update of the subsequent segmentation features, which can enable the subsequent segmentation features to retain different receptive field sizes and increase the size of the receptive field, thereby enriching the information content of the feature extraction results and improving the feature extraction accuracy.

[0108] Compared with the existing PyConvResNet (Pyramidal Convolution Deep residual network), the use of large convolution kernels is reduced, thereby reducing the amount of data calculation. In addition, the segmentation features at different scales introduce the features transmitted by the previous segmentation features during subsequent updates, enriching the sizes of different receptive fields and increasing the receptive field size, thereby enriching the information content of the feature extraction results and improving the feature extraction accuracy.

[0109] For each of the above-mentioned image feature extraction methods, the present application also provides an embodiment of a virtual device for implementing the image feature extraction method. Figure 4 The image feature extraction device 400 shown in the figure includes: a segmentation processing module 401 and a feature extraction result obtaining module 402.

[0110] The segmentation processing module 401 is used to perform segmentation processing on the current segmentation feature of at least two segmentation features associated with the processed image to obtain the target extraction feature and the feature to be transferred of the current segmentation feature; wherein the feature to be transferred of the current segmentation feature is used to update the subsequent segmentation feature located after the current segmentation feature, and determine the target extraction feature of the subsequent segmentation feature;

[0111] The feature extraction result obtaining module 402 is used to extract features according to the objectives of at least two segmentation features to obtain feature extraction results of the image to be processed.

[0112] The embodiment of the present application realizes information transmission between channels by introducing the feature to be transferred of the current segmentation feature, updating the subsequent segmentation feature after the current segmentation feature, and assisting in determining the target extraction feature of the subsequent segmentation feature, thereby enriching the receptive field of the subsequent segmentation feature, increasing the amount of information of the target extraction feature in the subsequent segmentation feature, and thus improving the feature extraction accuracy of the feature extraction result of the image to be processed. At the same time, by introducing the target extraction feature of the current segmentation feature, feature reuse is realized, thereby improving the feature extraction accuracy of the feature extraction result of the image to be processed.

[0113] Furthermore, the device also includes:

[0114] The current segmentation feature updating module is used to update the current segmentation feature according to the to-be-transferred feature of the previous segmentation feature located before the current segmentation feature is segmented before the current segmentation feature is segmented.

[0115] Furthermore, the current segmentation feature update module includes:

[0116] The target transfer feature obtaining unit is used to extract the to-be-transferred feature of the previous segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature, so as to obtain the target transfer feature;

[0117] The current segmentation feature updating unit is used to update the current segmentation feature according to the target transfer feature.

[0118] Furthermore, if the current segmentation feature does not have a subsequent segmentation feature, the segmentation processing module 401 further includes:

[0119] The feature extraction unit is used to extract features from the current segmentation features to obtain target extraction features of the current segmentation features.

[0120] Furthermore, if the number of features to be transferred of the preceding segmentation feature is at least two, the target transfer feature obtaining unit includes:

[0121] The convolution processing subunit is used to perform convolution processing on each to-be-transferred feature of the previous segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature to obtain the target transfer feature; wherein the sizes of the convolution kernels used in the convolution processing are different.

[0122] Furthermore, the current segmentation feature updating unit includes:

[0123] The feature integration subunit is used to integrate the target transfer feature with the current segmentation feature to update the current segmentation feature.

[0124] Furthermore, the segmentation processing module 401 includes:

[0125] The uniform segmentation processing unit is used to perform uniform segmentation processing on a current segmentation feature among at least two segmentation features associated with the image to be processed.

[0126] Furthermore, the device also includes:

[0127] A dimension reduction processing module is used to perform dimension reduction processing on the image to be processed before performing segmentation processing on the current segmentation feature of at least two segmentation features associated with the image to be processed to obtain a feature to be processed;

[0128] A segmentation feature obtaining module is used to segment the feature to be processed according to the number of channels of the feature to be processed to obtain at least two segmentation features;

[0129] The feature extraction result module includes:

[0130] The dimension-increasing processing unit is used to perform dimension-increasing processing on the target extraction features of at least two segmentation features to obtain feature extraction results of the image to be processed.

[0131] Furthermore, the device also includes:

[0132] The feature extraction result updating module is used to extract features according to at least two segmentation features to obtain the feature extraction result of the image to be processed, and then update the feature extraction result of the image to be processed according to the image to be processed.

[0133] Furthermore, the feature extraction result updating module includes:

[0134] A direct connection processing unit, used for processing the image to be processed by using a direct connection structure to obtain features to be fused;

[0135] The feature fusion unit is used to fuse the features to be fused with the feature extraction results of the image to be processed, so as to update the feature extraction results of the image to be processed.

[0136] The above-mentioned image feature extraction device can execute the image feature extraction method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the image feature extraction method.

[0137] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0138] like Figure 5 , is a block diagram of an electronic device for implementing the image feature extraction method of an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0139] like Figure 5As shown, the electronic device includes: one or more processors 501, a memory 502, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 501 is taken as an example.

[0140] The memory 502 is a non-transitory computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to perform the image feature extraction method provided in the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the image feature extraction method provided in the present application.

[0141] The memory 502 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the image feature extraction method in the embodiment of the present application (for example, the attached Figure 4 The processor 501 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 502, that is, the image feature extraction method in the above method embodiment is implemented.

[0142] The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created by the use of an electronic device that implements the image feature extraction method, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 502 may optionally include a memory remotely arranged relative to the processor 501, and these remote memories may be connected to the electronic device that implements the image feature extraction method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0143] The electronic device implementing the image feature extraction method may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0144] The input device 503 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device implementing the image feature extraction method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 504 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0145] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0147] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0148] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0149] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0150] According to the technical solution of the embodiment of the present application, by introducing the to-be-transferred feature of the current segmentation feature, updating the subsequent segmentation feature after the current segmentation feature, and assisting in determining the target extraction feature of the subsequent segmentation feature, information transmission between channels is realized, thereby enriching the receptive field of the subsequent segmentation feature, increasing the amount of information of the target extraction feature in the subsequent segmentation feature, and thus improving the feature extraction accuracy of the feature extraction result of the image to be processed. At the same time, by introducing the target extraction feature of the current segmentation feature, feature reuse is realized, thereby improving the feature extraction accuracy of the feature extraction result of the image to be processed.

[0151] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0152] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for extracting image features, include: Perform segmentation processing on a current segmentation feature of at least two segmentation features associated with the processed image to obtain a target extraction feature and a feature to be transferred of the current segmentation feature; wherein the feature to be transferred of the current segmentation feature is used to update a subsequent segmentation feature located after the current segmentation feature, and to determine a target extraction feature of the subsequent segmentation feature; Extracting features according to the at least two segmentation features to obtain a feature extraction result of the image to be processed; Wherein, before performing segmentation processing on the current segmentation feature among the at least two segmentation features associated with the image to be processed, the method further comprises: The current segmentation feature is updated according to the to-be-transferred feature of the previous segmentation feature that precedes the current segmentation feature.

2. The method according to claim 1, in, The updating of the current segmentation feature according to the to-be-transferred feature of the previous segmentation feature located before the current segmentation feature comprises: Extracting the to-be-transferred feature of the preceding segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature to obtain a target transfer feature; The current segmentation feature is updated according to the target transfer feature.

3. The method according to claim 2, if the current segmentation feature does not have a subsequent segmentation feature, then the current segmentation feature of the at least two segmentation features associated with the image to be processed is segmented to obtain the target extraction feature and the feature to be transferred of the current segmentation feature, include: Feature extraction is performed on the current segmentation feature to obtain a target extraction feature of the current segmentation feature.

4. The method according to claim 2, in, If the number of the features to be transferred of the preceding segmentation feature is at least two, extracting the features to be transferred of the preceding segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature to obtain the target transfer feature includes: Convolution processing is performed on each of the to-be-transferred features of the preceding segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature to obtain the target transfer feature; wherein the sizes of the convolution kernels used in the convolution processing are different.

5. The method according to claim 2, in, The updating of the current segmentation feature according to the target transfer feature includes: The target transfer feature is integrated with the current segmentation feature to update the current segmentation feature.

6. The method according to any one of claims 1 to 5, in, The performing segmentation processing on the current segmentation feature of at least two segmentation features associated with the image to be processed includes: A uniform segmentation process is performed on a current segmentation feature of at least two segmentation features associated with the image to be processed.

7. The method according to any one of claims 1 to 5, in, Before performing segmentation processing on a current segmentation feature among at least two segmentation features associated with the image to be processed, the method further includes: Performing dimensionality reduction processing on the image to be processed to obtain features to be processed; According to the number of channels of the feature to be processed, the feature to be processed is segmented to obtain the at least two segmented features; The step of extracting features according to the at least two segmentation features to obtain a feature extraction result of the image to be processed includes: The target extracted features of the at least two segmentation features are subjected to dimensionality increase processing to obtain a feature extraction result of the image to be processed.

8. The method according to any one of claims 1 to 5, in, After extracting features according to the target of the at least two segmentation features to obtain a feature extraction result of the image to be processed, the method further includes: According to the image to be processed, a feature extraction result of the image to be processed is updated.

9. The method according to claim 8, in, The updating of the feature extraction result of the image to be processed according to the image to be processed includes: The image to be processed is processed using a direct connection structure to obtain features to be fused; The feature to be fused is fused with the feature extraction result of the image to be processed to update the feature extraction result of the image to be processed.

10. An image feature extraction device, include: A segmentation processing module, used for performing segmentation processing on a current segmentation feature of at least two segmentation features associated with the processed image, and obtaining a target extraction feature and a feature to be transferred of the current segmentation feature; wherein the feature to be transferred of the current segmentation feature is used to update a subsequent segmentation feature located after the current segmentation feature, and to determine a target extraction feature of the subsequent segmentation feature; A feature extraction result obtaining module, used for extracting features according to the target of the at least two segmentation features to obtain a feature extraction result of the image to be processed; The current segmentation feature updating module is used to update the current segmentation feature according to the to-be-transferred feature of the previous segmentation feature located before the current segmentation feature is segmented before the current segmentation feature is segmented.

11. The device according to claim 10, in, The current segmentation feature updating module includes: A target transfer feature obtaining unit is used to extract the to-be-transferred feature of the preceding segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature, so as to obtain the target transfer feature; The current segmentation feature updating unit is used to update the current segmentation feature according to the target transfer feature.

12. The device according to claim 10, if the current segmentation feature does not have a subsequent segmentation feature, the segmentation processing module further include: The feature extraction unit is used to extract features from the current segmentation features to obtain target extraction features of the current segmentation features.

13. The device according to claim 11, in, If the number of the features to be transferred of the preceding segmentation feature is at least two, the target transfer feature obtaining unit includes: The convolution processing subunit is used to perform convolution processing on each of the to-be-transferred features of the preceding segmentation feature that is located before the current segmentation feature and adjacent to the current segmentation feature, so as to obtain the target transfer feature; wherein the sizes of the convolution kernels used in the convolution processing are different.

14. The device according to claim 11, in, The current segmentation feature updating unit includes: The feature integration subunit is used to integrate the target transfer feature with the current segmentation feature to update the current segmentation feature.

15. The device according to any one of claims 11 to 14, in, The segmentation processing module includes: The uniform segmentation processing unit is used to perform uniform segmentation processing on a current segmentation feature among at least two segmentation features associated with the image to be processed.

16. The device according to any one of claims 11 to 14, further comprising: include: A dimension reduction processing module, used for performing dimension reduction processing on the image to be processed to obtain a feature to be processed before performing segmentation processing on a current segmentation feature among at least two segmentation features associated with the image to be processed; A segmentation feature obtaining module, used for performing segmentation processing on the feature to be processed according to the number of channels of the feature to be processed, to obtain the at least two segmentation features; The feature extraction result obtaining module includes: The dimension-increasing processing unit is used to perform dimension-increasing processing on the target extraction features of the at least two segmentation features to obtain the feature extraction result of the image to be processed.

17. The device according to any one of claims 11 to 14, further comprising: include: The feature extraction result updating module is used to update the feature extraction result of the image to be processed according to the image to be processed after extracting features according to the target of the at least two segmentation features to obtain the feature extraction result of the image to be processed.

18. The device according to claim 17, in, The feature extraction result updating module comprises: A direct connection processing unit, used to process the image to be processed by using a direct connection structure to obtain features to be fused; The feature fusion unit is used to perform feature fusion on the feature to be fused and the feature extraction result of the image to be processed, so as to update the feature extraction result of the image to be processed.

19. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute an image feature extraction method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to enable a computer to execute an image feature extraction method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Methods for Image Segmentation, Computer Devices, and Storage Mediums

    US20210374478A1