A feature extraction method, device, electronic equipment, storage medium and program product

Abstract

The application discloses a feature extraction method and device, electronic equipment, a storage medium and a program product. The specific implementation scheme comprises the following steps: determining an input image and at least one to-be-fused feature map corresponding to the input image; for each to-be-fused feature map, determining a target feature map corresponding to the to-be-fused feature map based on a target resolution; splicing the target feature maps to obtain a spliced feature map; and performing convolution fusion operation on the spliced feature map to obtain a fused feature map corresponding to the input image. By splicing the target feature maps, the depth fusion of feature information is realized, the spliced feature map is obtained, and the convolution fusion operation is performed on the spliced feature map, so that the accuracy of the fused feature map is improved without significantly increasing the calculation amount, the alignment and fusion of the to-be-fused feature maps of various scales are realized, and the accuracy and robustness of the image processing process are improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a feature extraction method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] Binocular stereo matching is one of the core problems in the field of computer vision, aiming to recover the depth information of a scene by calculating the pixel parallax between the left and right camera images.

[0003] While existing stereo matching algorithms perform well in textured regions, they still face significant challenges when dealing with repetitive or weakly textured areas (such as tiled floors, fences, and regularly arranged patterns). For example, they suffer from: local feature ambiguity: on a single-scale high-resolution feature map of an image, the local features of repetitive textured regions are highly similar, leading to multiple peaks in the matching cost calculation and a high likelihood of false matches. Lack of contextual information: relying solely on feature extraction at a specific scale often fails to capture the complete features of the image and does not adequately fuse features across multiple scales, resulting in insufficient robustness of stereo matching algorithms when handling complex textures. Summary of the Invention

[0004] This invention provides a feature extraction method, apparatus, electronic device, storage medium, and program product to achieve alignment and fusion of feature maps at various scales, thereby improving the accuracy and robustness of the image processing process.

[0005] According to one aspect of the present invention, a feature extraction method is provided, comprising: Determine an input image and at least one feature map to be fused corresponding to the input image, wherein the feature map to be fused includes an image obtained after feature extraction of the input image; For each feature map to be fused, a target feature map corresponding to the feature map to be fused is determined based on the target resolution, wherein the target resolution is related to the resolution of the input image; The target feature maps are spliced ​​together to obtain a spliced ​​feature map, which contains feature information of each of the feature maps to be fused. A convolutional fusion operation is performed on the stitched feature map to obtain the fused feature map corresponding to the input image.

[0006] According to another aspect of the present invention, a feature extraction apparatus is provided, comprising: The first determining module is used to determine an input image and at least one feature map to be fused corresponding to the input image, wherein the feature map to be fused includes an image obtained after feature extraction of the input image; The second determining module is used to determine the target feature map corresponding to each feature map to be fused based on the target resolution, wherein the target resolution is related to the resolution of the input image; The splicing module is used to splice the target feature maps to obtain a spliced ​​feature map, wherein the spliced ​​feature map contains feature information of each of the feature maps to be fused. The fusion module is used to perform a convolutional fusion operation on the stitched feature map to obtain the fused feature map corresponding to the input image.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the feature extraction method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the feature extraction method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the feature extraction method described in any embodiment of the present invention.

[0010] The technical solution of this invention involves determining an input image and at least one feature map to be fused corresponding to the input image; for each feature map to be fused, determining a target feature map corresponding to the feature map to be fused based on the target resolution; concatenating the target feature maps to obtain a concatenated feature map; and performing a convolutional fusion operation on the concatenated feature map to obtain a fused feature map corresponding to the input image. By concatenating the target feature maps, deep fusion of feature information is achieved, resulting in a concatenated feature map. Performing a convolutional fusion operation on the concatenated feature map ensures that the accuracy of the fused feature map is improved without significantly increasing the computational load. This achieves alignment and fusion of feature maps to be fused at various scales, improving the accuracy and robustness of the image processing process.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a feature extraction method provided according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of a target feature map determination method according to Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of a feature extraction device according to Embodiment 3 of the present invention; Figure 4 This is a block diagram of an electronic device provided according to Embodiment 4 of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Example 1 Figure 1This is a flowchart of a feature extraction method according to Embodiment 1 of the present invention. This embodiment is applicable to the extraction of image features. The method can be executed by a feature extraction device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes: S110. Determine the input image and at least one feature map to be fused corresponding to the input image.

[0017] The feature map to be fused includes the image obtained after feature extraction from the input image.

[0018] In this embodiment, the input image can be understood as the image to be feature extracted, and the input image can be the image input into the feature extractor. The feature map to be fused can be the image obtained after feature extraction from the input image, and multiple feature maps can be extracted.

[0019] Specifically, an input image for feature extraction can be identified and fed into a feature extractor. The feature extractor stage then obtains feature maps of different resolutions to be fused.

[0020] For example, the input image is fed into a feature extractor, which then obtains four feature maps of different resolutions to be fused. These features typically have different spatial resolutions (e.g., 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image's resolution).

[0021] S120. For each feature map to be fused, determine the target feature map corresponding to the feature map to be fused based on the target resolution.

[0022] The target resolution is related to the resolution of the input image.

[0023] In this embodiment, the target resolution can be understood as a numerical value related to the resolution of the input image, and the target resolution can be a multiple of the resolution of the input image. The target feature map can be understood as the image obtained by aligning the feature maps to be fused using the target resolution.

[0024] Specifically, all feature maps of different resolutions to be fused are spatially aligned to the scale indicated by the target resolution. This spatial alignment can be achieved by upsampling the feature maps.

[0025] For example, the target resolution can be one-quarter of the input image's resolution. For the other three feature maps to be fused that are lower than this resolution, an upsampling operation can be used to uniformly align all the feature maps of different levels in the spatial dimension.

[0026] S130. The target feature maps are spliced ​​together to obtain the spliced ​​feature map.

[0027] The stitched feature map contains feature information of each of the feature maps to be fused.

[0028] In this embodiment, the stitched feature map can be understood as an image obtained by stitching together the target feature maps. The stitched feature map contains the feature information corresponding to each feature map to be fused, and the stitched feature map is a feature map with multi-level information.

[0029] Specifically, the target feature maps, which have been aligned spatially in the above steps, are concatenated along the channel dimension to obtain a concatenated feature map. This concatenated feature map retains both the geometric details of the low-level features and the global semantic information of the high-level features in the feature maps to be fused. The low-level and high-level features are feature information from feature maps to be fused at different resolutions.

[0030] For example, low-level features correspond to higher-resolution feature maps (e.g., 1 / 4) in the feature map to be fused. Low-level features mainly contain geometric details of the input image, such as edges, texture direction, and sharp corners. High-level features correspond to lower-resolution feature maps (images that need upsampling for alignment) but with a very large receptive field (e.g., 1 / 32) in the feature map to be fused. High-level features contain global semantic information, i.e., a macroscopic understanding of the input image.

[0031] S140. Perform a convolutional fusion operation on the stitched feature map to obtain the fused feature map corresponding to the input image.

[0032] In this embodiment, the fused feature map can be understood as the feature map obtained by fusing the feature information contained in the stitched feature map. The fused feature map can be used to replace the input image containing single-scale features or the feature map to be fused for subsequent cost volume calculation.

[0033] Specifically, convolutional fusion operations can be performed on the stitched feature maps to fuse feature information from different sources and restore the fused features to the number of channels required by the network. The number of channels required by the network can be the number of channels indicated by the target resolution.

[0034] For example, assuming the target feature map has dimensions (B, C, H, W), the concatenated feature map has dimensions (B, 4C, H, W). Here, B is the batch dimension, C is the channel dimension, and H and W are the height and width of the target feature map. After performing a convolutional fusion operation on the concatenated feature map, the resulting image undergoes channel transformation and dimensionality reduction, yielding a fused feature map with dimensions (B, C, H, W).

[0035] The technical solution of this invention involves determining an input image and at least one feature map to be fused corresponding to the input image; for each feature map to be fused, determining a target feature map corresponding to the feature map to be fused based on the target resolution; concatenating the target feature maps to obtain a concatenated feature map; and performing a convolutional fusion operation on the concatenated feature map to obtain a fused feature map corresponding to the input image. By concatenating the target feature maps, deep fusion of feature information is achieved, resulting in a concatenated feature map. Performing a convolutional fusion operation on the concatenated feature map ensures that the accuracy of the fused feature map is improved without significantly increasing the computational load. This achieves alignment and fusion of feature maps to be fused at various scales, improving the accuracy and robustness of the image processing process.

[0036] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0037] In one embodiment, the step of concatenating the target feature maps to obtain the concatenated feature map includes: For each target feature map, the target channel dimension corresponding to the target feature map is determined, and the target channel dimension includes the channel dimension of the target feature map; The sum of the target channel dimensions corresponding to each of the target feature maps is used to determine the stitching dimension; According to the splicing dimension, the target feature maps are spliced ​​together to obtain the spliced ​​feature map.

[0038] In this embodiment, the target channel dimension can be understood as the channel dimension of the target feature map, which can be obtained from the feature information of the target feature map. The stitching dimension can be understood as the channel dimension of the stitched feature map.

[0039] For example, suppose the four aligned target feature maps all have dimensions (B, C, H, W), where B is the batch size, C is the target channel dimension, and H and W are the height and width. Then, using 4C as the stitching dimension, the target feature maps are stacked along dim=1 (channel dimension). The size of the stitched feature map becomes (B, 4C, H, W). Intuitively, imagine the target feature maps as four transparent colored films; stitching is like stacking these four transparent colored films together. Each layer of film represents feature information at one scale. Although the resolution (H, W) of the stitched feature maps becomes the same, the thickness of the content they carry (i.e., the channel dimension) increases. If the four aligned target feature maps have different sizes, and their channel dimensions are C1, C2, C3, and C4 respectively, then the stitching dimension is set to C1+C2+C3+C4.

[0040] In one embodiment, performing a convolutional fusion operation on the concatenated feature map to obtain the fused feature map corresponding to the input image includes: The spliced ​​feature map is subjected to a convolutional fusion operation using spatial neighborhood information to obtain an initial feature map. The spatial neighborhood information includes information about the elements contained in the spliced ​​feature map. The pixel values ​​contained in the initial feature map are standardized to obtain a standardized feature map; The standardized feature map is subjected to a nonlinear transformation to obtain the transformed feature map; The channel dimension of the transformed feature map is transformed to a preset channel dimension to obtain a fused feature map, wherein the preset channel dimension includes the channel dimension of the target feature map.

[0041] In this embodiment, the initial feature map can be understood as the image obtained by performing convolutional enrichment operations on the stitched feature map, which can be done by convolving the feature information of each channel of the stitched image. The standardized feature map can be understood as the image obtained by normalizing the pixels in the initial feature map, solving the problem of inconsistent numerical magnitudes at different levels in the initial feature map. The transformed feature map can be understood as the image obtained by performing a nonlinear transformation on the elements of the standardized feature map. The preset channel dimension can be understood as the channel dimension of the target feature map.

[0042] Specifically, the concatenated feature map is a simple physical stacking of the target feature maps, and there is no information exchange between the feature information contained in each channel dimension. Therefore, spatial neighborhood information can be used to perform convolutional fusion on the concatenated feature map to achieve preliminary feature fusion and obtain an initial feature map. Next, each pixel value in the initial feature map is standardized to prevent the signal of one feature information from being too strong and overshadowing other information, thus obtaining a standardized feature map. Then, a nonlinear transformation is introduced to perform a nonlinear transformation on the elements contained in the standardized feature map to obtain a transformed feature map. Finally, channel transformation and dimensionality reduction are performed on the transformed feature map to transform its channel dimensions to the preset channel dimensions, resulting in the fused feature map.

[0043] For example, a 3x3 sliding window is used to perform convolutional fusion, allowing cross-scale information interaction between feature information at different scales in the concatenated feature map, resulting in an initial feature map. Next, each pixel value in the initial feature map is standardized, making the mean close to 0 and the standard deviation close to 1, resulting in a standardized feature map. Then, a non-linear transformation is performed on the elements contained in the standardized feature map, retaining the negative axis information. The non-linear transformation operates on elements with values ​​less than 0 in the standardized feature map. Finally, channel transformation and dimensionality reduction are performed on the transformed feature map, changing its channel dimensions to a preset channel dimension, i.e., compressing the thickened transformed feature map (4C) back to the standard thickness (C), eliminating redundancy, and obtaining the fused feature map.

[0044] Optionally, performing a nonlinear transformation on the standardized feature map to obtain the transformed feature map includes: Determine at least one target element contained in the normalized feature map, wherein the target element includes a tensor of the normalized feature map; For each target element, if the value of the target element is less than the value of a set element, then the target element is subjected to a nonlinear transformation to obtain a first transformed element; otherwise, the target element is determined as a second transformed element. Determine the transformed feature map composed of the first transformed element and the second transformed element.

[0045] In this embodiment, the standardized feature map is essentially a multi-dimensional tensor, storing the target elements. The first transformed elements include those obtained by performing a nonlinear transformation on the target elements. The second transformed elements include the target elements whose values ​​are greater than a set element.

[0046] For example, the ordinary ReLU activation function directly converts all negative numbers to 0, causing some neurons to stop transmitting information. The LeakyReLU activation function preserves a small slope when x < 0. The negative sign of an element's value sometimes represents directional or relative deviation information; preserving this negative axis information allows for the learning of more subtle disparity features. For each target element contained in the normalized feature map, if the target element's value is less than the set element's value of 0, the LeakyReLU activation function is used to perform a non-linear transformation on the target element, resulting in the first transformed element. If the target element's value is greater than the set element's value of 0, the target element is directly determined as the second transformed element. Finally, the image composed of the first and second transformed elements is determined as the transformed feature map.

[0047] In one embodiment, determining the input image and at least one feature map to be fused corresponding to the input image includes: Determine the input image; At least one resolution to be extracted is preset, and the resolution to be extracted is related to the resolution of the input image; For each resolution to be extracted, feature extraction is performed on the input image according to the resolution to be extracted to obtain the feature map to be fused corresponding to the resolution to be extracted.

[0048] In this embodiment, the resolution to be extracted can be understood as the resolution set for feature extraction of the input image, and the resolution to be extracted can be set as a multiple of the resolution of the input image.

[0049] For example, the resolution to be extracted can be 1 / 4, 1 / 8, 1 / 16, or 1 / 32 of the resolution of the input image. Then, according to each resolution to be extracted, the feature extractor performs feature extraction on the input image to obtain the feature map to be fused corresponding to that resolution. The feature extractor can perform feature extraction on the input image in stages, sequentially in the order of 1 / 4, 1 / 8, 1 / 16, and 1 / 32.

[0050] Example 2 Figure 2 This is a flowchart of a target feature map determination method according to Embodiment 2 of the present invention. This embodiment focuses on the target feature map determination steps in the above embodiment. Figure 2 As shown, the method includes: S210. Determine the input image and at least one feature map to be fused corresponding to the input image.

[0051] S220. For each feature map to be fused, execute S221-S223.

[0052] S221. Determine the feature map resolution of the feature map to be fused.

[0053] The feature map resolution includes the resolution of the feature map to be fused.

[0054] In this embodiment, the feature map resolution can be understood as the resolution of the feature map to be fused, and the feature map resolution is related to the extraction resolution set when extracting the feature map to be fused.

[0055] Specifically, the feature map resolution of the feature maps to be fused is determined from the feature information of the feature maps to be fused.

[0056] For example, the feature map resolution of the feature maps to be fused can be 1 / 4, 1 / 8, 1 / 16, or 1 / 32 of the resolution of the input image.

[0057] S222. Compare the feature map resolution and the target resolution to obtain the comparison result.

[0058] The comparison result indicates whether the feature map resolution and the target resolution are the same.

[0059] In this embodiment, the comparison result can be understood as the comparison result between the feature map resolution and the target resolution.

[0060] Specifically, the numerical values ​​of the feature map resolution and the target resolution are compared to obtain a comparison result indicating whether the numerical values ​​of the feature map resolution and the target resolution are the same.

[0061] S223. Based on the comparison results, determine the target feature map corresponding to the feature map to be fused.

[0062] Specifically, the target feature map is determined based on whether the feature map resolution and the target resolution indicated by the comparison results are the same. If the comparison results indicate that the feature map resolution and the target resolution are the same, the feature map to be fused can be directly used as the target feature map. If the comparison results indicate that the feature map resolution and the target resolution are different, the feature maps to be fused need to be aligned in spatial dimensions to obtain the target feature map.

[0063] Optionally, determining the target feature map corresponding to the feature map to be fused based on the comparison result includes: If the comparison result indicates that the feature map resolution and the target resolution are the same, the feature map to be fused is determined as the target feature map; If the comparison result indicates that the feature map resolution and the target resolution are different, an upsampling operation is performed on the feature map to be fused to obtain the target feature map corresponding to the feature map to be fused.

[0064] For example, the target resolution can be 1 / 4 of the input image resolution. If the comparison result indicates that the feature map resolution and the target resolution are the same, it means that the feature map to be fused (1 / 4) meets the requirement of the target resolution (1 / 4), and the feature map to be fused is directly determined as the target feature map. If the comparison result indicates that the feature map resolution and the target resolution are different, it means that the feature map to be fused (1 / 8, 1 / 16, 1 / 32) does not meet the requirement of the target resolution (1 / 4), and a bilinear interpolation operator needs to be used to upsample the feature map to be fused to obtain the target feature map corresponding to the feature map to be fused.

[0065] S230. The target feature maps are spliced ​​together to obtain the spliced ​​feature map.

[0066] S240. Perform a convolutional fusion operation on the stitched feature map to obtain the fused feature map corresponding to the input image.

[0067] The technical solution of this invention involves determining the feature map resolution of the feature map to be fused; comparing the feature map resolution with the target resolution to obtain a comparison result; and determining the target feature map corresponding to the feature map to be fused based on the comparison result. By comparing the feature map resolution with the target resolution, the feature maps to be fused at different resolutions are unified to obtain the target feature map. This ensures that high-level semantic information is not damaged, and high-level semantic information can be used to constrain low-level detail matching, significantly reducing disparity jumps and mismatches, and improving the accuracy and robustness of the image processing process.

[0068] Example 3 Figure 3 This is a schematic diagram of a feature extraction device according to Embodiment 3 of the present invention. Figure 3 As shown, the device includes: The first determining module 310 is used to determine an input image and at least one feature map to be fused corresponding to the input image, wherein the feature map to be fused includes an image obtained after feature extraction of the input image; The second determining module 320 is used to determine the target feature map corresponding to each feature map to be fused based on the target resolution, wherein the target resolution is related to the resolution of the input image; The splicing module 330 is used to splice the target feature maps to obtain a spliced ​​feature map, wherein the spliced ​​feature map contains feature information of each of the feature maps to be fused. The fusion module 340 is used to perform a convolutional fusion operation on the stitched feature map to obtain the fused feature map corresponding to the input image.

[0069] The feature extraction device provided in this embodiment of the invention determines an input image and at least one feature map to be fused corresponding to the input image through a first determining module; determines a target feature map corresponding to each feature map to be fused based on a target resolution through a second determining module; concatenates the target feature maps through a stitching module to obtain a stitched feature map; and performs a convolutional fusion operation on the stitched feature map through a fusion module to obtain a fused feature map corresponding to the input image. Through the cooperation between the modules, the target feature maps are stitched together, achieving deep fusion of feature information to obtain a stitched feature map. The convolutional fusion operation on the stitched feature map ensures that the accuracy of the fused feature map is improved without significantly increasing the computational load, achieving alignment and fusion of feature maps to be fused at various scales, and improving the accuracy and robustness of the image processing process.

[0070] In one embodiment, the second determining module 320 includes: The first determining unit is used to determine the feature map resolution of the feature map to be fused, wherein the feature map resolution includes the resolution of the feature map to be fused; A comparison unit is used to compare the feature map resolution and the target resolution to obtain a comparison result, wherein the comparison result indicates whether the feature map resolution and the target resolution are the same; The second determining unit is used to determine the target feature map corresponding to the feature map to be fused based on the comparison result.

[0071] In one embodiment, the second determining unit is specifically used for: If the comparison result indicates that the feature map resolution and the target resolution are the same, the feature map to be fused is determined as the target feature map; If the comparison result indicates that the feature map resolution and the target resolution are different, an upsampling operation is performed on the feature map to be fused to obtain the target feature map corresponding to the feature map to be fused.

[0072] In one embodiment, the splicing module 330 is specifically used for: For each target feature map, the target channel dimension corresponding to the target feature map is determined, and the target channel dimension includes the channel dimension of the target feature map; The sum of the target channel dimensions corresponding to each of the target feature maps is used to determine the stitching dimension; According to the splicing dimension, the target feature maps are spliced ​​together to obtain the spliced ​​feature map.

[0073] In one embodiment, the fusion module 340 includes: The fusion unit is used to perform a convolutional fusion operation on the stitched feature map using spatial neighborhood information to obtain an initial feature map. The spatial neighborhood information includes information about the elements contained in the stitched feature map. The processing unit is used to standardize the pixel values ​​contained in the initial feature map to obtain a standardized feature map. A transformation unit is used to perform a nonlinear transformation on the standardized feature map to obtain a transformed feature map. A preset unit is used to transform the channel dimension of the transformed feature map to a preset channel dimension to obtain a fused feature map, wherein the preset channel dimension includes the channel dimension of the target feature map.

[0074] In one embodiment, the transformation unit is specifically used for: Determine at least one target element contained in the normalized feature map, wherein the target element includes a tensor of the normalized feature map; For each target element, if the value of the target element is less than the value of a set element, then the target element is subjected to a nonlinear transformation to obtain a first transformed element; otherwise, the target element is determined as a second transformed element. Determine the transformed feature map composed of the first transformed element and the second transformed element.

[0075] In one embodiment, the first determining module 310 is specifically used for: Determine the input image; At least one resolution to be extracted is preset, and the resolution to be extracted is related to the resolution of the input image; For each resolution to be extracted, feature extraction is performed on the input image according to the resolution to be extracted to obtain the feature map to be fused corresponding to the resolution to be extracted.

[0076] The feature extraction device provided in this embodiment of the invention can execute the feature extraction method provided in any embodiment of the invention. Through the cooperation and collaborative work between the modules, the feature extraction is completed, and it has the corresponding functional modules and beneficial effects of the method.

[0077] Example 4 According to embodiments of the present invention, the present invention also provides an electronic device, a computer-readable storage medium, and a computer program product.

[0078] Figure 4This is a block diagram of an electronic device according to Embodiment 4 of the present invention, which implements the feature extraction method described in the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0079] like Figure 4 As shown, the electronic device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0080] Multiple components in the electronic device are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, optical disk, etc.; and a communication unit 419, such as a network card, modem, wireless transceiver, etc. The communication unit 419 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0081] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as feature extraction methods.

[0082] In some embodiments, the feature extraction method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the feature extraction method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured to perform the feature extraction method by any other suitable means (e.g., by means of firmware).

[0083] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0084] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0085] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0086] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0087] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0088] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0089] In some embodiments, the computer program product includes a computer program that, when executed by a processor, implements the feature extraction method provided in the embodiments of the present invention.

[0090] The technical solution of this invention provides a feature extraction method, apparatus, electronic device, storage medium, and program product. It involves determining an input image and at least one feature map to be fused corresponding to the input image; for each feature map to be fused, determining a target feature map based on a target resolution; concatenating the target feature maps to obtain a concatenated feature map; and performing a convolutional fusion operation on the concatenated feature map to obtain a fused feature map corresponding to the input image. By concatenating the target feature maps, deep fusion of feature information is achieved, resulting in a concatenated feature map. Performing a convolutional fusion operation on the concatenated feature map ensures that the accuracy of the fused feature map is improved without significantly increasing the computational load. This achieves alignment and fusion of feature maps at various scales, improving the accuracy and robustness of the image processing process.

[0091] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0092] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A feature extraction method, characterized in that, include: Determine an input image and at least one feature map to be fused corresponding to the input image, wherein the feature map to be fused includes an image obtained after feature extraction of the input image; For each feature map to be fused, a target feature map corresponding to the feature map to be fused is determined based on the target resolution, wherein the target resolution is related to the resolution of the input image; The target feature maps are spliced ​​together to obtain a spliced ​​feature map, which contains feature information of each of the feature maps to be fused. A convolutional fusion operation is performed on the stitched feature map to obtain the fused feature map corresponding to the input image.

2. The method according to claim 1, characterized in that, The step of determining the target feature map corresponding to the feature map to be fused based on the target resolution includes: Determine the feature map resolution of the feature map to be fused, wherein the feature map resolution includes the resolution of the feature map to be fused; The feature map resolution and the target resolution are compared to obtain a comparison result, which indicates whether the feature map resolution and the target resolution are the same. Based on the comparison results, the target feature map corresponding to the feature map to be fused is determined.

3. The method according to claim 2, characterized in that, The step of determining the target feature map corresponding to the feature map to be fused based on the comparison result includes: If the comparison result indicates that the feature map resolution and the target resolution are the same, the feature map to be fused is determined as the target feature map; If the comparison result indicates that the feature map resolution and the target resolution are different, an upsampling operation is performed on the feature map to be fused to obtain the target feature map corresponding to the feature map to be fused.

4. The method according to claim 1, characterized in that, The process of concatenating the target feature maps to obtain a concatenated feature map includes: For each target feature map, the target channel dimension corresponding to the target feature map is determined, and the target channel dimension includes the channel dimension of the target feature map; The sum of the target channel dimensions corresponding to each of the target feature maps is used to determine the stitching dimension; According to the splicing dimension, the target feature maps are spliced ​​together to obtain the spliced ​​feature map.

5. The method according to claim 1, characterized in that, The step of performing a convolutional fusion operation on the concatenated feature map to obtain the fused feature map corresponding to the input image includes: The spliced ​​feature map is subjected to a convolutional fusion operation using spatial neighborhood information to obtain an initial feature map. The spatial neighborhood information includes information about the elements contained in the spliced ​​feature map. The pixel values ​​contained in the initial feature map are standardized to obtain a standardized feature map; The standardized feature map is subjected to a nonlinear transformation to obtain the transformed feature map; The channel dimension of the transformed feature map is transformed to a preset channel dimension to obtain a fused feature map, wherein the preset channel dimension includes the channel dimension of the target feature map.

6. The method according to claim 5, characterized in that, The step of performing a nonlinear transformation on the standardized feature map to obtain the transformed feature map includes: Determine at least one target element contained in the normalized feature map, wherein the target element includes a tensor of the normalized feature map; For each target element, if the value of the target element is less than the value of a set element, then the target element is subjected to a nonlinear transformation to obtain a first transformed element; otherwise, the target element is determined as a second transformed element. Determine the transformed feature map composed of the first transformed element and the second transformed element.

7. The method according to claim 1, characterized in that, The determination of the input image and at least one feature map to be fused corresponding to the input image includes: Determine the input image; At least one resolution to be extracted is preset, and the resolution to be extracted is related to the resolution of the input image; For each resolution to be extracted, feature extraction is performed on the input image according to the resolution to be extracted to obtain the feature map to be fused corresponding to the resolution to be extracted.

8. A feature extraction device, characterized in that, include: The first determining module is used to determine an input image and at least one feature map to be fused corresponding to the input image, wherein the feature map to be fused includes an image obtained after feature extraction of the input image; The second determining module is used to determine the target feature map corresponding to each feature map to be fused based on the target resolution, wherein the target resolution is related to the resolution of the input image; The splicing module is used to splice the target feature maps to obtain a spliced ​​feature map, wherein the spliced ​​feature map contains feature information of each of the feature maps to be fused. The fusion module is used to perform a convolutional fusion operation on the stitched feature map to obtain the fused feature map corresponding to the input image.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the feature extraction method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the feature extraction method according to any one of claims 1-7.

11. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the feature extraction method according to any one of claims 1-7.

Images