Image recognition method, device, equipment and storage medium
By performing convolution and clustering on image feature representations, spatial grouping feature representations are generated, which solves the problem of high computational complexity in high-resolution image recognition and improves recognition efficiency and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TENCENT TECHNOLOGY (SHENZHEN) CO LTD
- Filing Date
- 2023-02-27
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies have high computational complexity in high-resolution image recognition, resulting in low recognition efficiency.
By performing convolution and clustering on the initial feature representation, multiple spatial groups are generated, and feature representations within and between groups are obtained, reducing the computational cost of global feature representation and improving generation efficiency.
It improves the efficiency and accuracy of image recognition, reduces the computational cost of generating global feature representations, and ensures the learning of the overall characteristics of the image.
Smart Images
Figure CN116958563B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to an image recognition method, apparatus, device, and storage medium. Background Technology
[0002] With the development of artificial intelligence technology, neural network models are widely used in computer vision tasks, such as image classification, object detection, and semantic segmentation. Among these, the model structure of neural networks plays a crucial role in computer vision tasks.
[0003] In related technologies, an image recognition model that can be used to generate feature representations of images is constructed by combining the self-attention mechanism in Transformer. However, for each position in the image, the self-attention weights need to be calculated with the global position of the image. Therefore, the overall computational complexity of related technologies increases exponentially with the image resolution. For high-resolution images, the image recognition efficiency of related technologies is low. Summary of the Invention
[0004] This application provides an image recognition method, apparatus, device, and storage medium, which can improve the efficiency of feature representation generation, thereby improving image recognition efficiency. The technical solution is as follows:
[0005] According to one aspect of the embodiments of this application, an image recognition method is provided, the method comprising:
[0006] Obtain an initial feature representation of the input image at a first pixel scale, wherein the initial feature representation includes multiple sub-features with location information;
[0007] Convolution is performed on the initial feature representation to obtain the transition feature representation of the input image at the second pixel scale, where the first pixel scale is larger than the second pixel scale;
[0008] Clustering is performed on the transition sub-features in the transition feature representation to obtain multiple spatial groups, wherein the spatial groups include multiple transition sub-features clustered according to location information in the spatial dimension;
[0009] Obtain the intra-group feature representation and inter-group feature representation corresponding to the transition sub-feature in the multiple spatial groupings. The intra-group feature representation is used to characterize the feature representation of the transition sub-feature under its corresponding spatial grouping constraints, and the inter-group feature representation is used to characterize the feature representation of the transition sub-feature under the constraints of the multiple spatial groupings.
[0010] Based on the intra-group and inter-group feature representations corresponding to the transition sub-features, a global feature representation of the input image at the first pixel scale is obtained, and the global feature representation is used to characterize the overall characteristics of the input image.
[0011] The recognition result of the input image is obtained based on the global feature representation of the input image at the first pixel scale.
[0012] According to one aspect of the embodiments of this application, an image recognition apparatus is provided, the apparatus comprising:
[0013] An initial feature acquisition module is used to acquire an initial feature representation of the input image at a first pixel scale, wherein the initial feature representation includes multiple sub-features with location information;
[0014] The transition feature acquisition module is used to convolve the initial feature representation to obtain the transition feature representation of the input image at a second pixel scale, wherein the first pixel scale is larger than the second pixel scale.
[0015] The spatial grouping acquisition module is used to cluster the transition sub-features in the transition feature representation to obtain multiple spatial groups, wherein the spatial groups include multiple transition sub-features clustered according to location information in the spatial dimension.
[0016] The group feature acquisition module is used to acquire the intra-group feature representation and inter-group feature representation corresponding to the transition sub-feature in the multiple spatial groups. The intra-group feature representation is used to characterize the feature representation of the transition sub-feature under its corresponding spatial group constraint, and the inter-group feature representation is used to characterize the feature representation of the transition sub-feature under the multiple spatial group constraints.
[0017] The global feature acquisition module is used to obtain the global feature representation of the input image at the first pixel scale based on the intra-group feature representation and inter-group feature representation corresponding to the transition sub-feature. The global feature representation is used to characterize the overall characteristics of the input image.
[0018] The recognition result acquisition module is used to acquire the recognition result of the input image based on the global feature representation of the input image at the first pixel scale.
[0019] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described image recognition method.
[0020] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, the computer program being loaded and executed by a processor to implement the above-described image recognition method.
[0021] According to one aspect of the embodiments of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the image recognition method described above.
[0022] The technical solutions provided in this application embodiment may have the following beneficial effects:
[0023] By clustering transitional sub-features in the transitional feature representation into multiple spatial groups, and then obtaining the within-group and between-group feature representations corresponding to the transitional sub-features on a per-spatial-group basis, without needing to calculate weight parameters between the transitional sub-features and the global transitional sub-features, the computational cost of generating the global feature representation is reduced, the generation efficiency of the global feature representation is improved, and thus the image processing efficiency is enhanced. Furthermore, by obtaining the global feature representation based on the within-group and between-group feature representations corresponding to the transitional sub-features, the global feature representation learns the overall characteristics of the input image, thereby improving the accuracy of the generated global feature representation. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a schematic diagram of the implementation environment of a solution provided in one embodiment of this application;
[0026] Figure 2 This is a schematic diagram of a feature extraction network provided in one embodiment of this application;
[0027] Figure 3 This is a schematic diagram of a dual-path feature extraction module provided in one embodiment of this application;
[0028] Figure 4 This is a schematic diagram of a first feature extraction subnetwork provided in one embodiment of this application;
[0029] Figure 5 This is a schematic diagram of a second feature extraction subnetwork provided in one embodiment of this application;
[0030] Figure 6 This is a flowchart of an image recognition method provided in one embodiment of this application;
[0031] Figure 7 This is a block diagram of an image recognition device provided in one embodiment of this application;
[0032] Figure 8 This is a block diagram of an image recognition device provided in another embodiment of this application;
[0033] Figure 9 This is a block diagram of a computer device provided in one embodiment of this application. Detailed Implementation
[0034] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0035] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.
[0036] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.
[0037] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, video processing, OCR (Optical Character Recognition), video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, and map building, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.
[0038] The technical solution provided in this application relates to computer vision technology in artificial intelligence. It uses computer vision technology to recognize input images, obtains feature representations of the input images, and then recognizes the input images based on the feature representations to obtain recognition results corresponding to the input images, such as type, target, segmentation, semantics, etc.
[0039] The method provided in this application embodiment can be executed by a computer device, which refers to an electronic device with data computing, processing, and storage capabilities. This computer device can be a terminal device such as a PC (Personal Computer), tablet computer, smartphone, wearable device, intelligent robot, or vehicle-mounted device; or it can be a server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.
[0040] The technical solutions provided in this application are applicable to any scenario requiring image processing, such as image recognition, autonomous driving, intelligent transportation, and pathological analysis. The technical solutions provided in this application can improve image recognition efficiency while ensuring image recognition accuracy.
[0041] In one example, please refer to Figure 1 The diagram illustrates an implementation environment for a solution provided in one embodiment of this application. This implementation environment may include: a terminal device 10 and a server 20.
[0042] Terminal device 10 can be a mobile phone, tablet computer, game console, multimedia playback device, PC, intelligent robot, vehicle-mounted device, or other similar device. A client application for a target application can be installed on terminal device 10. This target application can be an image recognition application, an autonomous driving application, a traffic monitoring application, or similar applications.
[0043] Server 20 provides background services to the client of the target application (such as an image recognition application) in terminal device 10. For example, server 20 can be a background server for the aforementioned target application (such as an image recognition application). Server 20 can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center.
[0044] Terminal device 10 and server 20 can communicate with each other via network 30. This network 30 can be a wired network or a wireless network.
[0045] Optionally, an image recognition model is deployed in the terminal device 10 or the server 20. This image recognition model is used to recognize the input image to obtain recognition results, such as object detection results, image segmentation results, semantic recognition results, and object following results. The following description uses the deployment of an image recognition model in the server 20 as an example to illustrate the technical solution provided in the embodiments of this application.
[0046] refer to Figure 1 After acquiring the input image, the client in terminal device 10 sends the input image to server 20. Server 20 recognizes the input image using image recognition model 30 to obtain the recognition result corresponding to the input image. For example, it obtains the feature representation corresponding to the input image through feature extraction network 31 in image recognition model 30, and then obtains the recognition result corresponding to the input image based on the feature representation in image recognition model 30 through recognition network 32. Server 20 sends the recognition result to the client so that the client can display the recognition result corresponding to the input image to the user.
[0047] Optionally, the image recognition model 30 may include multiple feature extraction networks 31, each of which is used to obtain the feature representation of the input image at different pixel scales. That is, the image recognition model 30 can obtain the feature representation of the input image at different pixel scales.
[0048] For example, refer to Figure 2Taking the image recognition model 30, which includes four feature extraction networks 31, as an example, feature extraction network 31a is used to obtain the feature representation of the input image at a 1 / 4 pixel scale (relative to the pixel scale of the input image), feature extraction network 31b is used to obtain the feature representation of the input image at a 1 / 8 pixel scale, feature extraction network 31c is used to obtain the feature representation of the input image at a 1 / 16 pixel scale, and feature extraction network 31d is used to obtain the feature representation of the input image at a 1 / 32 pixel scale.
[0049] For the first feature extraction network 31a among the multiple feature extraction networks 31, it generates a feature representation of the input image at a 1 / 4 pixel scale based on the input image. For the target feature extraction network (excluding the first feature extraction network) among the multiple feature extraction networks 31, the target feature extraction network is used to obtain a feature representation of the input image at a target pixel scale based on the output of the previous feature extraction network adjacent to the target feature extraction network. For example, feature extraction network 31b obtains a feature representation of the input image at a 1 / 8 pixel scale based on the output of feature extraction network 31a.
[0050] Different feature extraction networks 31 may include different numbers of dual-path feature extraction modules. The dual-path feature extraction modules are used to obtain the local feature representation and global feature representation of the input image, and obtain the feature representation corresponding to the input image based on the local feature representation and global feature representation of the input image.
[0051] For example, the target feature extraction network includes at least one dual-path feature extraction module. The target dual-path feature extraction module (excluding the first dual-path feature extraction module) is used to obtain a feature representation of the input image at a target pixel scale based on the output of the preceding dual-path feature extraction module adjacent to it. Here, the target pixel scale is the pixel scale corresponding to the target feature extraction network. For the first dual-path feature extraction module, it obtains the feature representation of the input image at the target pixel scale based on the input of the target feature extraction network.
[0052] Optionally, the dual-path feature extraction module may include a first feature extraction sub-network and a second feature extraction sub-network. For example, refer to... Figure 3The target feature extraction network includes at least one dual-path feature extraction module 301. The dual-path feature extraction module 301 includes a first feature extraction sub-network 3011 and a second feature extraction sub-network 3012. The first feature extraction sub-network 3011 extracts local feature representations of the input image at the target pixel scale, and the second feature extraction sub-network 3012 extracts global feature representations of the input image at the target pixel scale. Global feature representations characterize the overall characteristics of the input image, while local feature representations characterize the local characteristics of the input image. Thus, through the dual-path feature extraction module, feature representations of the input image at different receptive fields can be obtained, thereby ensuring the accuracy of feature representation extraction and improving image recognition accuracy.
[0053] The first feature extraction sub-network 3011 can be constructed based on 1*1 convolutional layers and 3*3 depthwise separable convolutional layers (i.e., Depthwise Conv). For example, refer to... Figure 4 The first feature extraction subnetwork 3011 is constructed from two 1*1 convolutional layers and one 3*3 depthwise separable convolutional layer.
[0054] refer to Figure 5 The second feature extraction subnetwork 3012 may include a grouping subnetwork 3012a, an intra-group feature extraction subnetwork 3012b, and an inter-group feature extraction subnetwork 3012c. The grouping subnetwork 3012a is used to obtain spatial groupings corresponding to the input image, the intra-group feature extraction subnetwork 3012b is used to obtain intra-group feature representations corresponding to the input image, and the inter-group feature extraction subnetwork 3012c is used to obtain inter-group feature representations corresponding to the input image. Optionally, the second feature extraction subnetwork 3012 is constructed based on a clustering-based self-attention mechanism. Compared to the self-attention mechanism in related technologies, this embodiment can reduce the computational cost of generating global feature representations, thereby improving the efficiency of global feature representation generation.
[0055] It should be noted that the model structures of the first feature extraction subnetwork and the second feature extraction subnetwork described above are merely illustrative, and any model structure applicable to local feature representation generation or global feature representation generation is within the protection scope of the embodiments of this application.
[0056] The technical solution of this application will be described below through method embodiments.
[0057] Please refer to Figure 6 The diagram illustrates a flowchart of an image recognition method provided in one embodiment of this application. The execution entity for each step of the method may be... Figure 1 The terminal device 10 or server 20 in the implementation environment of the scheme shown may include the following steps (601-606).
[0058] Step 601: Obtain the initial feature representation of the input image at the first pixel scale. The initial feature representation includes multiple sub-features with location information.
[0059] The input image can refer to the image used for recognition processing, such as the input to the image recognition model described above. For example, the input image can refer to a video frame, a photograph, a screenshot, etc., but this application embodiment does not limit this.
[0060] Pixel scale is used to characterize the resolution of the input image. For example, the height and width of the input image can be used to characterize the pixel scale, denoted as H*W*3, where 3 represents the three color channels of the input image, i.e., the RGB (Red-Green-Blue) three color channels. The first pixel scale can refer to any pixel scale, which can be set and adjusted according to actual usage requirements. For example, refer to... Figure 2 The first pixel scale can be the pixel scale corresponding to feature extraction network 31a or the pixel scale corresponding to feature extraction network 31b.
[0061] In this application's embodiments, the initial feature representation may refer to the feature representation obtained by preprocessing the input image. In one example, the process of obtaining the initial feature representation can be as follows:
[0062] 1. Divide the input image into multiple image blocks.
[0063] Optionally, refer to Figure 2 By uniformly dividing the input image, multiple image blocks of the same size can be obtained. For example, if the input image size is 224*224, it can be uniformly divided into 196 image blocks of size 16*16.
[0064] 2. Project the multiple image patches separately to obtain the sub-features corresponding to each image patch.
[0065] Optionally, for each image patch, a tensor (or word embedding) is used to represent it, i.e., a sub-feature. For the input image, since each image patch is ordered, each sub-feature also needs to include a location identifier to represent its positional information.
[0066] 3. Combine the sub-features corresponding to multiple image patches to obtain the initial feature representation of the input image.
[0067] Optionally, the sequence formed by multiple sub-features is determined as the initial feature representation corresponding to the input image.
[0068] 4. Downsample the initial feature representation corresponding to the input image to obtain the initial feature representation of the input image at the first pixel scale.
[0069] For example, refer to Figure 2 If the original pixel scale of the input image is H*W, then the initial feature representation of the input image at a 1 / 4 pixel scale can be obtained by downsampling, denoted as H / 4*W / 4*D1, where D1 is the dimension of the initial feature representation, which can be set and adjusted according to actual usage requirements.
[0070] Optionally, for two adjacent feature extraction networks, the output of the preceding feature extraction network can be regarded as the initial feature representation corresponding to the following feature extraction network. For example, refer to... Figure 2 Alternatively, the output of feature extraction network 31a can be determined as the initial feature representation corresponding to feature extraction network 31b, the output of feature extraction network 31b can be determined as the initial feature representation corresponding to feature extraction network 31c, and the output of feature extraction network 31c can be determined as the initial feature representation corresponding to feature extraction network 31d. This application embodiment does not limit this.
[0071] Step 602: Convolve the initial feature representation to obtain the transition feature representation of the input image at the second pixel scale, where the first pixel scale is larger than the second pixel scale.
[0072] Optionally, the initial feature representation can be convolved by the second feature extraction subnetwork in the image recognition model to obtain the transitional feature representation of the input image at the second pixel scale.
[0073] For example, refer to Figure 5 After obtaining the initial feature representation, the second feature extraction subnetwork 3012 convolves the initial feature representation through the DWConv layer to obtain the transition feature representation of the input image at the second pixel scale, denoted as H / 4k*W / 4k*D1, where k is the stride, which can be set and adjusted according to actual usage requirements, such as 2, 3, 4, etc.
[0074] The DW Conv layer is a network consisting of two convolutional parts. The first part performs channel-wise convolution on the initial feature representation using DepthwiseConv to obtain the first convolution result. The second part, PointwiseConv, combines the first convolution result to obtain the transition feature representation of the input image at the second pixel scale.
[0075] In this embodiment, the resolution of the initial feature representation is reduced by using the DW Conv layer, which can reduce the number of computational parameters of the second feature extraction sub-network, thereby improving the computational efficiency of the second feature extraction sub-network and further improving the generation efficiency of the global feature representation.
[0076] Step 603: Cluster the transition sub-features in the transition feature representation to obtain multiple spatial groups, which include multiple transition sub-features clustered according to location information in the spatial dimension.
[0077] Optionally, multiple spatial groups can be obtained by clustering the transition sub-features through the grouping sub-network in the second feature extraction sub-network.
[0078] In one example, the spatial grouping process can be as follows: For the target transition sub-feature in the transition feature representation, obtain the distance between the target transition sub-feature and each cluster center. Obtain the minimum distance among the distances between the target transition sub-feature and each cluster center; determine the spatial grouping corresponding to the cluster center with the minimum distance as the spatial grouping corresponding to the target transition sub-feature.
[0079] The target transition feature can refer to any transition feature in the transition feature representation, and the cluster center and the number of cluster centers can be set and adjusted based on empirical values.
[0080] For example, the above process can be represented by the following formula:
[0081]
[0082] Where C is the transition sub-feature set (each transition sub-feature includes location information), K is the number of cluster centers (i.e., the number of spatial groups), and μ k For the k-th cluster center, C k For the k-th spatial group, x i Let L be the i-th transition feature in C, and let L() be the distance function, such as Euclidean distance, Manhattan distance, etc.
[0083] Step 604: Obtain the intra-group feature representation and inter-group feature representation corresponding to the transition sub-feature in multiple spatial groupings. The intra-group feature representation is used to characterize the feature representation of the transition sub-feature under its corresponding spatial grouping constraints, and the inter-group feature representation is used to characterize the feature representation of the transition sub-feature under multiple spatial grouping constraints.
[0084] Optionally, the intra-group feature representation corresponding to the transition sub-feature is obtained through the intra-group feature extraction sub-network in the second feature extraction sub-network, and the inter-group feature representation corresponding to the transition sub-feature is obtained through the inter-group feature extraction sub-network in the second feature extraction sub-network. This application obtains the intra-group and inter-group feature representations corresponding to the transition sub-feature by spatial grouping, without needing to calculate weight parameters between the transition sub-feature and the global transition sub-feature, thereby reducing the computational load of generating the global feature representation and improving the generation efficiency of the global feature representation.
[0085] In one example, step 604 may also include the following:
[0086] 1. Perform linear transformations on the transition sub-features in multiple spatial groups to obtain the transformed sub-features corresponding to each spatial group.
[0087] For example, refer to Figure 5 The transition sub-features are obtained by linearly transforming the transition sub-features through the Linear layer in the second feature extraction sub-network 3012. The Linear layer is used to perform linear transformations on the features to obtain features of the desired style. Optionally, the transition sub-features can also be linearly transformed before obtaining spatial groupings; this embodiment of the application does not limit this.
[0088] 2. For the target spatial group among multiple spatial groups, obtain the intra-group weight parameters of the transition sub-features in the target spatial group based on the transition sub-features in the target spatial group.
[0089] The target spatial group can refer to any spatial group among multiple spatial groups. The intra-group weight parameter can refer to the weight of a certain transition sub-feature in a spatial group within that spatial group.
[0090] In one example, the ratio between a certain transitional sub-feature in the target space group and the sum of all transitional sub-features in the target space group can be determined as the in-group weight parameter corresponding to that transitional sub-feature.
[0091] For example, the within-group weight parameter can be represented as follows:
[0092]
[0093] Among them, w i For I k x in i The corresponding within-group weight parameter, I k For the k-th spatial group, x j For I k The j-th transition feature in x i For I ｋ The i-th transition feature in.
[0094] 3. Based on the intra-group weight parameters of the target transition sub-features in the target space grouping, and the transformation sub-features corresponding to the target transition sub-features, obtain the intra-group feature representations corresponding to the target transition sub-features.
[0095] The intra-group feature representation corresponding to the target transition sub-feature refers to the feature representation of the target transition sub-feature under the grouping constraints of the target space.
[0096] In one example, the process of obtaining the intra-group feature representation can be as follows: sum the intra-group weight parameters corresponding to each transition sub-feature in the target space group to obtain the first sum; divide the intra-group weight parameter of the target transition sub-feature by the first sum to obtain the first divisor; multiply the first divisor by the transformation sub-feature corresponding to the target transition sub-feature to obtain the intra-group feature representation corresponding to the target transition sub-feature.
[0097] For example, the within-group feature representation can be represented as follows:
[0098]
[0099] in, For x ｉ The corresponding transformation sub-features, For x ｉ The corresponding within-group feature representation, The first divisor, This is the first sum.
[0100] 4. Based on the transformation sub-features corresponding to the target space group and the length of the target space group, obtain the feature representation of the target space group.
[0101] Optionally, the length of the target space group can be determined by the number of transition sub-features in the target space group, or the length of the tensor composed of each transition sub-feature in the target space group can be determined by the length of the target space group. This application does not limit this.
[0102] In one example, the process of obtaining the feature representation of the target space group can be as follows: sum the transformation sub-features corresponding to the target space group to obtain the second sum value; divide the second sum value by the length of the target space group to obtain the feature representation of the target space group.
[0103] For example, the between-group feature representation can be expressed as follows:
[0104]
[0105] Among them, l ｉ For I k Length, Feature representation for grouping the target space. This is the second sum mentioned above.
[0106] 5. Based on the feature representation of the target space group and the learning weight parameters corresponding to the target transition sub-features, obtain the inter-group feature representation corresponding to the target transition sub-features.
[0107] Among them, the learning weight parameter is used to characterize the information interaction relationship between different spatial groups. That is, the inter-group feature representation corresponding to the target transition sub-feature can refer to the feature that lands at the spatial location corresponding to the target transition sub-feature after different spatial groups have interacted with each other.
[0108] For example, the between-group feature representation can be expressed as follows:
[0109]
[0110] Where, m i For x ｉ The corresponding learning weight parameters, For x i The corresponding between-group features are represented.
[0111] Step 605: Based on the intra-group feature representation and inter-group feature representation corresponding to the transition sub-features, the global feature representation of the input image at the first pixel scale is obtained. The global feature representation is used to characterize the overall properties of the input image.
[0112] The global feature representation of the input image can reflect the global information of the input image. Optionally, the global feature representation of the input image at the first pixel scale can be obtained by the second feature extraction sub-network based on the intra-group and inter-group feature representations corresponding to the transition sub-features.
[0113] In one example, the process of obtaining the global feature representation can be as follows:
[0114] 1. Based on the intra-group feature representations corresponding to the transition sub-features in multiple spatial groupings, obtain the intra-group feature representation of the input image; and based on the inter-group feature representations corresponding to the transition sub-features in multiple spatial groupings, obtain the inter-group feature representation of the input image.
[0115] Optionally, the sequence of intra-group feature representations corresponding to transition sub-features in multiple spatial groups is determined as the intra-group feature representation of the input image, and the sequence of inter-group feature representations corresponding to transition sub-features in multiple spatial groups is determined as the inter-group feature representation of the input image.
[0116] 2. The intra-group and inter-group feature representations of the input image are concatenated to obtain the preliminary global feature representation of the input image at the second pixel scale.
[0117] Optionally, by concatenating the intra-group and inter-group feature representations of the input image in the channel dimension through the second feature extraction sub-network, a preliminary global feature representation of the input image at the second pixel scale can be obtained.
[0118] 3. Transform the preliminary global feature representation of the input image at the second pixel scale to obtain the transitional global feature representation of the input image at the first pixel scale.
[0119] Optionally, since the second feature extraction subnetwork reduces the pixel scale of the initial feature representation after obtaining it, the second feature extraction subnetwork also needs to restore the pixel scale to the first pixel scale.
[0120] In one example, unlike upsampling, this application embodiment uses a Channel to Spatial approach to scale up pixels. This process can be as follows: obtain the attention weights corresponding to each channel in the preliminary global feature representation; obtain the feature map by channel weighting based on the attention weights and the preliminary global feature representation; obtain the spatial attention weights based on the feature map; adjust the preliminary global feature representation based on the spatial attention weights and channel attention weights to obtain the transitional global feature representation of the input image at the first pixel scale.
[0121] 4. The transitional global feature representation of the input image at the first pixel scale is fused with the initial feature representation of the input image at the first pixel scale to obtain the global feature representation of the input image at the first pixel scale.
[0122] Optionally, the transitional global feature representation of the input image at the first pixel scale can be obtained by adding (or multiplying) the initial feature representation of the input image at the first pixel scale to the transitional global feature representation of the input image at the first pixel scale through the second feature extraction sub-network.
[0123] Step 606: Obtain the recognition result of the input image based on the global feature representation of the input image at the first pixel scale.
[0124] Optionally, the recognition result of the input image can be obtained by using the recognition network in the image recognition model based on the global feature representation of the input image at the first pixel scale.
[0125] Considering that the lack of spatial information in the global feature representation leads to inaccurate recognition results, the image recognition model in this application embodiment also supports incorporating the local feature representation corresponding to the input image into the global feature representation to obtain feature representations with multiple receptive fields, thereby improving the accuracy of the recognition results.
[0126] In one example, step 606 may also include the following:
[0127] 1. Extract features from the initial feature representation of the input image at the first pixel scale to obtain the local feature representation of the input image at the first pixel scale. The local feature representation is used to characterize the local properties of the input image.
[0128] Optionally, features can be extracted from multiple regions in the input image through the first feature extraction sub-network described above to obtain multiple extraction results. The multiple extraction results can then be fused to obtain the local feature representation corresponding to the input image.
[0129] 2. The local and global feature representations of the input image at the first pixel scale are fused to obtain the feature representation of the input image at the first pixel scale.
[0130] Optionally, by fusing the local and global feature representations of the input image at the first pixel scale using the dual-path feature extraction module described above, the feature representation of the input image at the first pixel scale can be obtained.
[0131] In an exemplary embodiment, for a target feature extraction network in an image recognition model, the initial feature representation of the input image at a first pixel scale is processed sequentially by at least one dual-path feature extraction module corresponding to the target feature extraction network to obtain the final feature representation of the input image at the first pixel scale. Processing the initial feature representation through at least one dual-path feature extraction module improves the accuracy of the feature representation. For example, suppose the target feature extraction network includes a dual-path feature extraction module 1 and a dual-path feature extraction module 2. The dual-path feature extraction module 1 processes the initial feature representation of the input image at the first pixel scale to obtain a first feature representation of the input image at the first pixel scale. Then, the dual-path feature extraction module 2 processes the first feature representation of the input image at the first pixel scale to obtain the final feature representation of the input image at the first pixel scale.
[0132] Optionally, feature representations of the input image at different pixel scales can be obtained through an image recognition model. For example, refer to... Figure 2 The initial feature representation of the input image at a 1 / 4 pixel scale is processed by feature extraction network 31a to obtain the feature representation of the input image at a 1 / 4 pixel scale. This feature representation at the 1 / 4 pixel scale is then downsampled to obtain the initial feature representation of the input image at a 1 / 8 pixel scale. This initial feature representation at a 1 / 8 pixel scale is then processed by feature extraction network 31b to obtain the final feature representation of the input image at a 1 / 8 pixel scale. The method for obtaining the feature representation of the input image at the target pixel scale is the same as the method for obtaining the feature representation of the input image at the first pixel scale, and will not be repeated here.
[0133] Optionally, after obtaining the feature representation of the input image at different pixel scales, the recognition result of the input image can be obtained based on the feature representation of the input image at different pixel scales.
[0134] For example, in object detection tasks, the recognition network in an image recognition model determines the target in the input image, such as an object, person, or animal, based on the feature representation of the input image at different pixel scales. In image classification tasks, the classification network in an image recognition model determines the type of the input image based on the feature representation of the input image at different pixel scales; for example, it obtains the scene corresponding to the input image through the classification network and then classifies the input image according to the scene. In semantic recognition tasks, the recognition network in an image recognition model determines the semantic result corresponding to the input image, such as attributes, state, or content, based on the feature representation of the input image at different pixel scales.
[0135] In summary, the technical solution provided in this application clusters the transition sub-features in the transition feature representation into multiple spatial groups, and then obtains the intra-group and inter-group feature representations corresponding to the transition sub-features on a spatial group basis. This eliminates the need to calculate weight parameters between the transition sub-features and the global transition sub-features, thereby reducing the computational load of generating the global feature representation, improving its generation efficiency, and ultimately enhancing image processing efficiency. Furthermore, by obtaining the global feature representation based on the intra-group and inter-group feature representations corresponding to the transition sub-features, the global feature representation learns the overall characteristics of the input image, thus improving the accuracy of the generated global feature representation.
[0136] In addition, by fusing the local and global feature representations of the input image through a dual-path feature extraction module, the feature representation of the input image is obtained. This allows the feature representation of the input image to learn features under different receptive fields, thereby improving the accuracy of feature representation generation and thus improving the accuracy of image recognition.
[0137] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.
[0138] refer to Figure 7 This diagram illustrates a block diagram of an image recognition apparatus according to an embodiment of this application. The apparatus has the functionality to implement the method example described above; this functionality can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the computer device described above, or it can be located within a computer device. Figure 7As shown, the device 700 includes: an initial feature acquisition module 701, a transition feature acquisition module 702, a spatial grouping acquisition module 703, a grouping feature acquisition module 704, a global feature acquisition module 705, and a recognition result acquisition module 706.
[0139] The initial feature acquisition module 701 is used to acquire the initial feature representation of the input image at the first pixel scale, wherein the initial feature representation includes multiple sub-features with positional information.
[0140] The transition feature acquisition module 702 is used to convolve the initial feature representation to obtain the transition feature representation of the input image at a second pixel scale, wherein the first pixel scale is larger than the second pixel scale.
[0141] The spatial grouping acquisition module 703 is used to cluster the transition sub-features in the transition feature representation to obtain multiple spatial groups, wherein the spatial groups include multiple transition sub-features clustered according to location information in the spatial dimension.
[0142] The group feature acquisition module 704 is used to acquire the intra-group feature representation and inter-group feature representation corresponding to the transition sub-feature in the multiple spatial groups. The intra-group feature representation is used to characterize the feature representation of the transition sub-feature under its corresponding spatial group constraint, and the inter-group feature representation is used to characterize the feature representation of the transition sub-feature under the multiple spatial group constraints.
[0143] The global feature acquisition module 705 is used to obtain the global feature representation of the input image at the first pixel scale based on the intra-group feature representation and inter-group feature representation corresponding to the transition sub-feature, wherein the global feature representation is used to characterize the overall characteristics of the input image.
[0144] The recognition result acquisition module 706 is used to acquire the recognition result of the input image based on the global feature representation of the input image at the first pixel scale.
[0145] In some embodiments, such as Figure 8 As shown, the group feature acquisition module 704 includes: a transformation feature acquisition submodule 704a, an intra-group weight acquisition submodule 704b, an intra-group feature acquisition submodule 704c, a group representation generation submodule 704d, and an inter-group feature acquisition submodule 704e.
[0146] The transformation feature acquisition submodule 704a is used to perform linear transformation on the transition sub-features in the multiple spatial groups respectively to obtain the transformation sub-features corresponding to the multiple spatial groups respectively.
[0147] The intra-group weight acquisition submodule 704b is used to acquire the intra-group weight parameters of the transition sub-features in the target spatial group based on the transition sub-features in the target spatial group among the multiple spatial groups.
[0148] The intra-group feature acquisition submodule 704c is used to acquire the intra-group feature representation corresponding to the target transition sub-feature based on the intra-group weight parameters of the target transition sub-feature in the target space grouping and the transformation sub-feature corresponding to the target transition sub-feature.
[0149] The grouping representation generation submodule 704d is used to obtain the feature representation of the target space group based on the transformation sub-features corresponding to the target space group and the length of the target space group.
[0150] The inter-group feature acquisition submodule 704e is used to acquire the inter-group feature representation corresponding to the target transition sub-feature based on the feature representation of the target space grouping and the learning weight parameters corresponding to the target transition sub-feature.
[0151] In some embodiments, the intra-group feature acquisition submodule 704c is used for:
[0152] The weight parameters within each transition sub-feature in the target space grouping are summed to obtain the first sum value;
[0153] Divide the intra-group weight parameter of the target transition sub-feature by the first sum value to obtain the first divisor value;
[0154] Multiply the first divisor and the transformation sub-feature corresponding to the target transition sub-feature by multiplying them to obtain the in-group feature representation corresponding to the target transition sub-feature.
[0155] In some embodiments, the grouping represents the generation submodule 704d, used for:
[0156] The transformation sub-features corresponding to the target space grouping are summed to obtain a second sum value;
[0157] Divide the second sum by the length of the target space group to obtain the feature representation of the target space group.
[0158] In some embodiments, the global feature acquisition module 705 is configured to:
[0159] Based on the intra-group feature representations corresponding to the transition sub-features in the multiple spatial groupings, the intra-group feature representation of the input image is obtained; and based on the inter-group feature representations corresponding to the transition sub-features in the multiple spatial groupings, the inter-group feature representation of the input image is obtained.
[0160] The intra-group feature representation and inter-group feature representation corresponding to the input image are concatenated to obtain the preliminary global feature representation of the input image at the second pixel scale;
[0161] The preliminary global feature representation of the input image at the second pixel scale is transformed to obtain the transitional global feature representation of the input image at the first pixel scale.
[0162] The transitional global feature representation of the input image at the first pixel scale is fused with the initial feature representation of the input image at the first pixel scale to obtain the global feature representation of the input image at the first pixel scale.
[0163] In some embodiments, the spatial grouping acquisition module 703 is configured to:
[0164] For the target transition sub-feature in the transition feature representation, obtain the distance between the target transition sub-feature and each cluster center;
[0165] Obtain the minimum distance among the distances between the target transition sub-feature and each cluster center;
[0166] The spatial grouping corresponding to the cluster center of the minimum distance is determined as the spatial grouping corresponding to the target transition sub-feature.
[0167] In some embodiments, the initial feature acquisition module 701 is configured to:
[0168] The input image is divided into multiple image blocks;
[0169] Projecting each of the multiple image blocks separately yields the sub-features corresponding to each of the multiple image blocks;
[0170] The sub-features corresponding to the multiple image patches are combined to obtain the initial feature representation of the input image;
[0171] The initial feature representation corresponding to the input image is downsampled to obtain the initial feature representation of the input image at the first pixel scale.
[0172] In some embodiments, such as Figure 8 As shown, the device 700 further includes a local feature acquisition module 707 and a feature representation generation module 708.
[0173] The local feature acquisition module 707 is used to extract features from the initial feature representation of the input image at the first pixel scale to obtain the local feature representation of the input image at the first pixel scale. The local feature representation is used to characterize the local features of the input image.
[0174] The feature representation generation module 708 is used to fuse the local feature representation and global feature representation of the input image at the first pixel scale to obtain the feature representation of the input image at the first pixel scale.
[0175] In some embodiments, the feature representation generation module 708 is used to obtain feature representations of the input image at different pixel scales through an image recognition model; wherein, the image recognition model includes multiple feature extraction networks, and a target feature extraction network among the multiple feature extraction networks is used to obtain the feature representation of the input image at a target pixel scale based on the output of the previous feature extraction network adjacent to the target feature extraction network; the target feature extraction network includes at least one dual-path feature extraction module, and a target dual-path feature extraction module among the at least one dual-path feature extraction module is used to obtain the feature representation of the input image at the target pixel scale based on the output of the previous dual-path feature extraction module adjacent to the target dual-path feature extraction module; the dual-path feature extraction module includes a first feature extraction sub-network and a second feature extraction sub-network, the first feature extraction sub-network being used to extract the local feature representation of the input image at the target pixel scale, and the second feature extraction sub-network being used to extract the global feature representation of the input image at the target pixel scale.
[0176] In summary, the technical solution provided in this application clusters the transition sub-features in the transition feature representation into multiple spatial groups, and then obtains the intra-group and inter-group feature representations corresponding to the transition sub-features on a spatial group basis. This eliminates the need to calculate weight parameters between the transition sub-features and the global transition sub-features, thereby reducing the computational load of generating the global feature representation, improving its generation efficiency, and ultimately enhancing image processing efficiency. Furthermore, by obtaining the global feature representation based on the intra-group and inter-group feature representations corresponding to the transition sub-features, the global feature representation learns the overall characteristics of the input image, thus improving the accuracy of the generated global feature representation.
[0177] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0178] Please refer to Figure 9 This diagram illustrates a structural block diagram of a computer device according to an embodiment of this application. The computer device 900 can be used to implement the image recognition method provided in the above embodiments. The computer device 900 can be any electronic device with data computing, processing, and storage functions; specifically:
[0179] The computer device 900 includes a central processing unit (such as a CPU, GPU, or FPGA) 901, a system memory 904 including RAM (Random-Access Memory) 902 and ROM (Read-Only Memory) 903, and a system bus 905 connecting the system memory 904 and the central processing unit 901. The computer device 900 also includes a basic input / output system 906 that facilitates information transfer between various devices within the server, and a large-capacity storage device 907 for storing the operating system 913, application programs 914, and other program modules 915.
[0180] The basic input / output system 906 includes a display 908 for displaying information and an input device 909 for user input, such as a mouse or keyboard. Both the display 908 and the input device 909 are connected to the central processing unit 901 via an input / output controller 910 connected to the system bus 905. The basic input / output system 906 may also include the input / output controller 910 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 910 also provides output to a display screen, printer, or other types of output devices.
[0181] The mass storage device 907 is connected to the central processing unit 901 via a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable media provide non-volatile storage for the computer device 900. That is, the mass storage device 907 may include computer-readable media (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.
[0182] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 904 and mass storage device 907 described above can be collectively referred to as memory.
[0183] According to an embodiment of this application, the computer device 900 can also be connected to a remote computer on a network, such as the Internet, for operation. That is, the computer device 900 can be connected to a network 912 via a network interface unit 911 connected to the system bus 905, or the network interface unit 911 can be used to connect to other types of networks or remote computer systems (not shown).
[0184] The memory also includes a computer program stored in the memory and configured to be executed by one or more processors to implement the image recognition method described above.
[0185] In some embodiments, a computer-readable storage medium is also provided, wherein a computer program is stored therein, which, when executed by a processor, implements the image recognition method described above.
[0186] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).
[0187] In some embodiments, a computer program product is also provided, the computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the computer device to perform the image recognition method described above.
[0188] It should be noted that all information (including but not limited to object device information, object personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the object or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the input images and model structures involved in this application were obtained with full authorization.
[0189] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.
[0190] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. An image recognition method, characterized in that, The method includes: Obtain an initial feature representation of the input image at a first pixel scale, wherein the initial feature representation includes multiple sub-features with location information; Convolution is performed on the initial feature representation to obtain the transition feature representation of the input image at the second pixel scale, where the first pixel scale is larger than the second pixel scale; Clustering is performed on the transition sub-features in the transition feature representation to obtain multiple spatial groups, wherein the spatial groups include multiple transition sub-features clustered according to location information in the spatial dimension; The transition sub-features in the plurality of spatial groups are linearly transformed to obtain the transformed sub-features corresponding to the plurality of spatial groups respectively. Based on the intra-group weight parameters of the target transition sub-feature in the target spatial group and the transformation sub-feature corresponding to the target transition sub-feature, the intra-group feature representation corresponding to the target transition sub-feature is obtained. The intra-group feature representation is used to characterize the feature representation of the transition sub-feature under its corresponding spatial group constraint. The target spatial group is any spatial group among the multiple spatial groups. The intra-group weight parameters refer to the weight of the transition sub-feature in the spatial group to which the transition sub-feature belongs. Based on the transformation sub-features corresponding to the target space group and the length of the target space group, the feature representation of the target space group is obtained; Based on the feature representation of the target spatial group and the learning weight parameters corresponding to the target transition sub-feature, the inter-group feature representation corresponding to the target transition sub-feature is obtained. The inter-group feature representation is used to characterize the feature representation of the transition sub-feature under the constraints of the multiple spatial groups, and the learning weight parameters are used to characterize the information interaction relationship between different spatial groups. Based on the intra-group and inter-group feature representations corresponding to the transition sub-features, a global feature representation of the input image at the first pixel scale is obtained, and the global feature representation is used to characterize the overall characteristics of the input image. The recognition result of the input image is obtained based on the global feature representation of the input image at the first pixel scale.
2. The method according to claim 1, characterized in that, The step of obtaining the intra-group feature representation corresponding to the target transition sub-feature based on the intra-group weight parameters of the target transition sub-feature in the target space grouping and the transformation sub-feature corresponding to the target transition sub-feature includes: The weight parameters within each transition sub-feature in the target space grouping are summed to obtain the first sum value; Divide the intra-group weight parameter of the target transition sub-feature by the first sum value to obtain the first divisor value; Multiply the first divisor and the transformation sub-feature corresponding to the target transition sub-feature by multiplying them to obtain the in-group feature representation corresponding to the target transition sub-feature.
3. The method according to claim 1, characterized in that, The step of obtaining the feature representation of the target spatial group based on the transformation sub-feature corresponding to the target spatial group and the length of the target spatial group includes: The transformation sub-features corresponding to the target space grouping are summed to obtain a second sum value; Divide the second sum by the length of the target space group to obtain the feature representation of the target space group.
4. The method according to claim 1, characterized in that, The step of obtaining the global feature representation of the input image at the first pixel scale based on the intra-group feature representation and inter-group feature representation corresponding to the transition sub-feature includes: Based on the intra-group feature representations corresponding to the transition sub-features in the multiple spatial groupings, the intra-group feature representation of the input image is obtained; and based on the inter-group feature representations corresponding to the transition sub-features in the multiple spatial groupings, the inter-group feature representation of the input image is obtained. The intra-group feature representation and inter-group feature representation corresponding to the input image are concatenated to obtain the preliminary global feature representation of the input image at the second pixel scale; The preliminary global feature representation of the input image at the second pixel scale is transformed to obtain the transitional global feature representation of the input image at the first pixel scale. The transitional global feature representation of the input image at the first pixel scale is fused with the initial feature representation of the input image at the first pixel scale to obtain the global feature representation of the input image at the first pixel scale.
5. The method according to claim 1, characterized in that, The clustering of transition sub-features in the transition feature representation yields multiple spatial groups, including: For the target transition sub-feature in the transition feature representation, obtain the distance between the target transition sub-feature and each cluster center; Obtain the minimum distance among the distances between the target transition sub-feature and each cluster center; The spatial grouping corresponding to the cluster center of the minimum distance is determined as the spatial grouping corresponding to the target transition sub-feature.
6. The method according to claim 1, characterized in that, The process of obtaining the initial feature representation of the input image at the first pixel scale includes: The input image is divided into multiple image blocks; Projecting each of the multiple image blocks separately yields the sub-features corresponding to each of the multiple image blocks; The sub-features corresponding to the multiple image patches are combined to obtain the initial feature representation of the input image; The initial feature representation corresponding to the input image is downsampled to obtain the initial feature representation of the input image at the first pixel scale.
7. The method according to claim 1, characterized in that, After obtaining the global feature representation of the input image at the first pixel scale based on the intra-group feature representation and inter-group feature representation corresponding to the transition sub-feature, the method further includes: Feature extraction is performed on the initial feature representation of the input image at the first pixel scale to obtain the local feature representation of the input image at the first pixel scale. The local feature representation is used to characterize the local properties of the input image. The local feature representation and global feature representation of the input image at the first pixel scale are fused to obtain the feature representation of the input image at the first pixel scale.
8. The method according to claim 1, characterized in that, The method further includes: The feature representations of the input image at different pixel scales are obtained through an image recognition model; The image recognition model includes multiple feature extraction networks, and the target feature extraction network among the multiple feature extraction networks is used to obtain the feature representation of the input image at the target pixel scale based on the output of the previous feature extraction network adjacent to the target feature extraction network. The target feature extraction network includes at least one dual-path feature extraction module. The target dual-path feature extraction module in the at least one dual-path feature extraction module is used to obtain the feature representation of the input image at the target pixel scale based on the output of the previous dual-path feature extraction module adjacent to the target dual-path feature extraction module. The dual-path feature extraction module includes a first feature extraction sub-network and a second feature extraction sub-network. The first feature extraction sub-network is used to extract the local feature representation of the input image at the target pixel scale, and the second feature extraction sub-network is used to extract the global feature representation of the input image at the target pixel scale.
9. An image recognition device, characterized in that, The device includes: An initial feature acquisition module is used to acquire an initial feature representation of the input image at a first pixel scale, wherein the initial feature representation includes multiple sub-features with location information; The transition feature acquisition module is used to convolve the initial feature representation to obtain the transition feature representation of the input image at a second pixel scale, wherein the first pixel scale is larger than the second pixel scale. The spatial grouping acquisition module is used to cluster the transition sub-features in the transition feature representation to obtain multiple spatial groups, wherein the spatial groups include multiple transition sub-features clustered according to location information in the spatial dimension. The group feature acquisition module is used to perform linear transformations on the transition sub-features in the plurality of spatial groups respectively, to obtain the transformed sub-features corresponding to the plurality of spatial groups respectively; based on the intra-group weight parameters of the target transition sub-feature in the target spatial group and the transformed sub-feature corresponding to the target transition sub-feature, the module obtains the intra-group feature representation corresponding to the target transition sub-feature, which is used to characterize the feature representation of the transition sub-feature under the constraints of its corresponding spatial group, where the target spatial group is any one of the plurality of spatial groups, and the intra-group weight parameters refer to the weight of the transition sub-feature in the spatial group to which the transition sub-feature belongs; based on the transformed sub-feature corresponding to the target spatial group and the length of the target spatial group, the module obtains the feature representation of the target spatial group; based on the feature representation of the target spatial group and the learning weight parameters corresponding to the target transition sub-feature, the module obtains the inter-group feature representation corresponding to the target transition sub-feature, which is used to characterize the feature representation of the transition sub-feature under the constraints of the plurality of spatial groups, and the learning weight parameters are used to characterize the information interaction relationship between different spatial groups; The global feature acquisition module is used to obtain the global feature representation of the input image at the first pixel scale based on the intra-group feature representation and inter-group feature representation corresponding to the transition sub-feature. The global feature representation is used to characterize the overall characteristics of the input image. The recognition result acquisition module is used to acquire the recognition result of the input image based on the global feature representation of the input image at the first pixel scale.
10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program, which is loaded and executed by the processor to implement the image recognition method as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the image recognition method as described in any one of claims 1 to 8.
12. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, and a processor reads from and executes the computer program to implement the image recognition method as described in any one of claims 1 to 8.