Image recognition method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an image recognition method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a to-be-detected image and an image recognition model; the image recognition model comprises an extraction unit, a gating unit, a plurality of expert units and a feature fusion unit; performing feature extraction on the to-be-detected image based on the extraction unit to obtain an optimized feature map, and determining a scheduling coefficient based on the gating unit and the optimized feature map; performing linear transformation on the optimized feature map according to the plurality of expert units to obtain a mapping feature representation corresponding to each expert unit; determining prompt features corresponding to the optimized feature map according to the scheduling coefficient and the mapping feature representation, and performing feature fusion on the prompt features and the optimized feature map based on a feature fusion unit to obtain feature lexical elements; and determining an identification result of the to-be-detected image based on the feature lexical elements. By adopting the method, the accuracy of image recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an object recognition method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] In specific scenarios, the dimensionality requirements of objects that need to be identified in image recognition are getting higher and higher, and it is necessary to identify multiple attributes of the target objects in the image, such as the gender of pedestrians, clothing color, hats, umbrellas, masks, life jackets, etc.

[0003] In traditional technology, a shared feature extraction network is used to extract features from the image to be identified. After obtaining the feature graph, the feature graph is identified by multiple independent classifiers. Each classifier is trained by a sample image with a specific attribute, and mainly focuses on the identification of the specific attribute for learning. For example, classifier F1 is used to analyze the features of an umbrella, and classifier F2 is used to analyze the features of a mask. Then, each classifier obtains the identification result of the attribute of interest corresponding to the classifier, and then the identification results obtained by each classifier are aggregated to obtain a structured attribute storage result.

[0004] However, in traditional technologies, when each classifier is identifying the corresponding attribute, different classifiers perform classification analysis on the attributes they are concerned about. Due to insufficient sharing of features required for identification of different attributes or poor adaptability of shared features to certain attributes, for example, there is a lack of effective interaction and information flow between different classifiers, or the identification of a certain attribute depends on low-level features such as RGB values, while the identification of other attributes may depend on high-level features such as texture, which affects the recognition accuracy of the corresponding attribute of the classifier and leads to poor accuracy in attribute recognition. In addition, the iterative training and orchestration of multiple classifiers also pose considerable challenges to resources. Summary of the invention

[0005] Based on this, it is necessary to provide an image recognition method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical problems.

[0006] In a first aspect, the present application provides an image recognition method, comprising:

[0007] Acquire an image to be detected and an image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit;

[0008] Extracting features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determining a scheduling coefficient based on the gating unit and the optimized feature map;

[0009] According to the plurality of expert units, the optimized feature graph is linearly transformed to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature graph;

[0010] Determining a prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and performing feature fusion on the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word;

[0011] A recognition result of the image to be detected is determined based on the feature word.

[0012] In one embodiment, determining the prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation includes:

[0013] The mapping feature representations corresponding to the expert units are weighted and summed according to the scheduling coefficients, and converted through a preset activation function to obtain prompt features.

[0014] In one embodiment, the step of fusing the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word unit includes:

[0015] The feature fusion unit performs dot product summation on the prompt feature and the optimized feature map to obtain a feature word corresponding to the optimized feature map.

[0016] In one embodiment, the image recognition model further includes an attribute recognition unit, the attribute recognition unit includes a plurality of multi-layer perceptrons, each of the multi-layer perceptrons is used to recognize each target attribute;

[0017] The determining the recognition result of the image to be detected based on the feature word comprises:

[0018] According to the mapping relationship between the feature word unit and the target attribute in each of the multi-layer perceptrons, the feature word unit is mapped and transformed to obtain an intermediate feature representation;

[0019] Performing probability normalization transformation on the intermediate feature representation output by each of the multilayer perceptrons to obtain a probability distribution result corresponding to each of the multilayer perceptrons; the probability distribution result represents the probability prediction of the target attribute of the multilayer perceptron;

[0020] A recognition result corresponding to the image to be detected is determined based on the probability distribution result.

[0021] In one embodiment, the image recognition model further includes a target detection unit; and the extracting unit extracts features from the image to be detected to obtain an optimized feature map, including:

[0022] Acquire an image to be detected, perform target detection on the image to be detected according to the target detection unit, and obtain a target calibration frame of a target object in the image to be detected;

[0023] Cropping the image to be detected according to the target calibration frame to obtain a target image corresponding to the target object;

[0024] Based on the extraction unit, feature extraction is performed on the target image to obtain an optimized feature map.

[0025] In one embodiment, the extraction unit includes a feature extraction unit and an information extraction unit; the step of extracting features from the target image based on the extraction unit to obtain an optimized feature map includes:

[0026] Extracting features from the target image using the feature extraction unit to obtain an original feature map corresponding to the target image;

[0027] The information extraction unit performs local information extraction on the original feature map to obtain an optimized feature map.

[0028] In one embodiment, the information extraction unit includes a cascaded convolution module, a batch normalization layer, an activation layer and a pooling layer; the information extraction unit performs local information extraction on the original feature map to obtain an optimized feature map, including:

[0029] According to the cascade convolution module, batch normalization layer and activation layer, the original feature map is subjected to local spatial information enhancement and refinement to obtain an initial optimized feature map;

[0030] The initial optimized feature map is subjected to dimensionality reduction sampling according to the pooling layer to obtain an optimized feature map.

[0031] In a second aspect, the present application also provides an image recognition device, comprising:

[0032] An acquisition module, used to acquire an image to be detected and an image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit;

[0033] A feature extraction module, used to extract features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determine a scheduling coefficient based on the gating unit and the optimized feature map;

[0034] A linear transformation module, used to perform a linear transformation on the optimized feature map according to the plurality of expert units to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature map;

[0035] A feature fusion module, used to determine the prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and perform feature fusion on the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word;

[0036] A determination module is used to determine a recognition result of the image to be detected based on the feature word.

[0037] In one of the embodiments, the feature fusion module is specifically used to perform weighted summation on the mapping feature representations corresponding to each of the expert units according to the scheduling coefficient, and convert them through a preset activation function to obtain a prompt feature.

[0038] In one of the embodiments, the feature fusion module is specifically configured to perform a dot product summation on the prompt feature and the optimized feature map according to the feature fusion unit to obtain a feature word corresponding to the optimized feature map.

[0039] In one embodiment, the image recognition model further includes an attribute recognition unit, the attribute recognition unit includes a plurality of multi-layer perceptrons, each of which is used to recognize each target attribute; the determination module is specifically used to perform mapping transformation on the feature word element according to the mapping relationship between the feature word element and the target attribute in each of the multi-layer perceptrons to obtain an intermediate feature representation;

[0040] Performing probability normalization transformation on the intermediate feature representation output by each of the multilayer perceptrons to obtain a probability distribution result corresponding to each of the multilayer perceptrons; the probability distribution result represents the probability prediction of the target attribute of the multilayer perceptron;

[0041] A recognition result corresponding to the image to be detected is determined based on the probability distribution result.

[0042] In one embodiment, the image recognition model further includes a target detection unit; the feature extraction module is specifically used to obtain an image to be detected, perform target detection on the image to be detected according to the target detection unit, and obtain a target calibration frame of a target object in the image to be detected;

[0043] Cropping the image to be detected according to the target calibration frame to obtain a target image corresponding to the target object;

[0044] Based on the extraction unit, feature extraction is performed on the target image to obtain an optimized feature map.

[0045] In one embodiment, the extraction unit includes a feature extraction unit and an information extraction unit; the feature extraction module is specifically used to extract features of the target image according to the feature extraction unit to obtain an original feature map corresponding to the target image;

[0046] The information extraction unit performs local information extraction on the original feature map to obtain an optimized feature map.

[0047] In one of the embodiments, the information extraction unit includes a cascaded convolution module, a batch normalization layer, an activation layer and a pooling layer; the feature extraction module is specifically used to perform local spatial information enhancement and extraction on the original feature map according to the cascaded convolution module, the batch normalization layer and the activation layer to obtain an initial optimized feature map;

[0048] The initial optimized feature map is sampled by dimensionality reduction according to the pooling layer to obtain an optimized feature map.

[0049] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0050] Acquire an image to be detected and an image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit;

[0051] Extracting features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determining a scheduling coefficient based on the gating unit and the optimized feature map;

[0052] According to the plurality of expert units, the optimized feature graph is linearly transformed to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature graph;

[0053] Determining a prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and performing feature fusion on the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word;

[0054] A recognition result of the image to be detected is determined based on the feature word.

[0055] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0056] Acquire an image to be detected and an image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit;

[0057] Extracting features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determining a scheduling coefficient based on the gating unit and the optimized feature map;

[0058] According to the plurality of expert units, the optimized feature graph is linearly transformed to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature graph;

[0059] Determining a prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and performing feature fusion on the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word;

[0060] A recognition result of the image to be detected is determined based on the feature word.

[0061] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0062] Acquire an image to be detected and an image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit;

[0063] Extracting features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determining a scheduling coefficient based on the gating unit and the optimized feature map;

[0064] According to the plurality of expert units, the optimized feature graph is linearly transformed to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature graph;

[0065] Determining a prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and performing feature fusion on the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word;

[0066] A recognition result of the image to be detected is determined based on the feature word.

[0067] The above-mentioned image recognition method, device, computer equipment, computer-readable storage medium and computer program product obtain an image to be detected and an image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit; based on the extraction unit, feature extraction is performed on the image to be detected to obtain an optimized feature map, and based on the gating unit and the optimized feature map, a scheduling coefficient is determined; based on the multiple expert units, the optimized feature map is linearly transformed to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature map; based on the scheduling coefficient and the mapping feature representation, the prompt feature corresponding to the optimized feature map is determined, and based on the feature fusion unit, the prompt feature and the optimized feature map are feature fused to obtain feature words; based on the feature words, the recognition result of the image to be detected is determined. By adopting this method, by combining the gating unit and multiple expert units, the scheduling coefficient can be used to dynamically determine the expert unit involved in feature processing among multiple expert units, and the feature processing of different expert units can be associated and information shared. Then, the feature fusion unit fuses the prompt feature and the optimized feature map to obtain the feature word. By optimizing the detailed information of the feature map and the global information of the prompt feature, the feature expression ability of the feature word is enhanced, and the understanding of the image to be detected by the image recognition model is improved. Finally, the recognition result is determined by the feature word, which can improve the recognition ability of each attribute recognition in complex real scenes, thereby improving the accuracy of attribute recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0069] Figure 1 is a flowchart of an image recognition method in one embodiment;

[0070] Figure 2 is a schematic diagram of the model structure of a gating unit, an expert unit, and a feature fusion unit in an image recognition model in one embodiment;

[0071] Figure 3 A schematic diagram of a process of determining a recognition result by a recognition unit in one embodiment;

[0072] Figure 4 A schematic diagram of a model structure of a recognition unit in an image recognition model in one embodiment;

[0073] Figure 5A schematic diagram of a process of extracting features from an image to be detected in one embodiment;

[0074] Figure 6 is a schematic diagram of a process of extracting features from a cropped target image in one embodiment;

[0075] Figure 7 It is a schematic diagram of a process of extracting local spatial information from an original feature map by an information extraction unit in one embodiment;

[0076] Figure 8 is a schematic diagram of a model structure of an information extraction unit in an image recognition model in one embodiment;

[0077] Fig. 9 is a flowchart of an example of an image recognition method in an embodiment;

[0078] Fig.10 is a structural block diagram of an image recognition device in one embodiment;

[0079] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0081] In one embodiment, Figure 1 As shown, an image recognition method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0082] Step 102: Obtain an image to be detected and an image recognition model.

[0083] Among them, the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit.

[0084] In the embodiments of the present application, the image recognition model can be deployed in terminals in scenarios such as smart cities and smart production. For example, the monitoring systems at the entrances of residential areas, stations, etc. need to identify multiple attributes of people entering and leaving. The terminal where the monitoring system is located can distinguish people entering and leaving according to their characteristics, including attribute recognition of gender, clothing color, and items carried by people entering and leaving. In the above scenario, the image recognition model can be deployed in the terminal where the monitoring system is located, and the image recognition model can be loaded. Furthermore, when the image recognition model is applied, the video stream captured by the monitoring system is obtained, and then the terminal extracts frames from the video stream to obtain multiple frames of images in the video stream as images to be detected.

[0085] Step 104: extract features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determine a scheduling coefficient based on the gating unit and the optimized feature map.

[0086] In an embodiment of the present application, after the terminal completes loading the image recognition model and acquiring the image to be detected, the image to be detected is input into the image recognition model in sequence, and features are extracted from the image to be detected by the extraction unit in the image recognition model. Meaningful feature information is extracted from the image to be detected, such as edges, corner points, textures, etc., to obtain an optimized feature map corresponding to the image to be detected.

[0087] After obtaining the optimized feature map, the terminal inputs the optimized feature map into the gating unit, analyzes the optimized feature map through the learning parameters of the gating unit, determines the correlation between the optimized feature map and each expert unit, and then the gating unit gives a scheduling coefficient G under the current feature map according to the correlation. The scheduling coefficient G represents the contribution of each expert unit to the final decision.

[0088] The gating unit uses the following formula:

[0089]

[0090] Where x represents the optimized feature map input to the gated unit, is the weight matrix, is the bias term for noise correction, and are all learnable parameters, is the normalized exponential function, is the activation function, is a standard normal distribution.

[0091] Furthermore, Specifically:

[0092]

[0093] Here, k is adjusted according to the specific task situation and takes a value of 2 in this embodiment.

[0094] Step 106: linearly transform the optimized feature map according to the multiple expert units to obtain a mapping feature representation corresponding to each expert unit.

[0095] Each mapping feature representation may reflect global information of different dimensions in the optimized feature map. For example, global information of different dimensions may be texture features, color features, or key position features.

[0096] In the embodiment of the present application, the expert unit is a first-level linear layer. Since the dimension of the optimized feature map is , , , are the embedding vector dimensions, height and width of the optimized feature map Y respectively, then the specific input dimension of the expert unit is the embedding vector dimension corresponding to the optimized feature map , the output dimension is 1. Each expert unit realizes the global information of a certain dimension corresponding to each expert unit by adjusting the model parameters during the training process. Each expert unit performs a linear transformation on the input optimized feature map to obtain the mapping feature representation of the corresponding dimension of the expert unit, and the mapping feature representation is a scalar matrix. The mapping feature representation output by each expert unit represents the mapping result of the input optimized feature map from a specific perspective of the expert unit corresponding to the dimension. It can be the learning of specific information in the optimized feature map, or the conversion result of the input optimized feature map embedding vector to capture global information of different dimensions. The terminal linearly changes the optimized feature map through multiple expert units respectively, maps the global information of different dimensions in the optimized feature map, and obtains mapping feature representations under multiple dimensions.

[0097] Each expert unit can also enhance the optimized feature map in the dimension corresponding to the expert, and strengthen the expression of the dimension in the optimized feature map. For example, the corresponding dimension of a certain expert unit can be texture information, then the expert unit enhances the texture information of the optimized feature map.

[0098] Step 108, determining the prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and performing feature fusion on the prompt feature and the optimized feature graph based on a feature fusion unit to obtain a feature word.

[0099] In an embodiment of the present application, the terminal combines each mapping feature representation with the scheduling coefficient according to the correspondence between the scheduling coefficient and the expert unit, adjusts the contribution of each mapping feature representation to the recognition result, and summarizes each feature representation to obtain a prompt feature of the optimized feature graph. The prompt feature can reflect the key information in the optimized feature graph, highlight and emphasize the global information that has a prominent contribution to the recognition result, and serves as a feature representation that concentrates on the corresponding attribute information of the optimized feature graph.

[0100] After the terminal obtains the prompt features of the optimized feature map according to the image recognition model, the prompt features are applied to the optimized feature map according to the feature fusion unit, and the optimized feature map and the prompt features are feature fused to obtain the final feature word unit, which is used for subsequent attribute recognition tasks. The feature word unit combines the information of the optimized feature map and the prompt features to provide the image recognition model with the final feature input for the final decision.

[0101] Step 110: determining a recognition result of the image to be detected based on the feature word.

[0102] In the embodiment of the present application, after the terminal obtains the feature word element through the feature fusion unit of the image recognition model, the terminal performs post-processing on the image to be detected according to the feature word element, maps the features contained in the feature word element, and obtains the recognition result of the image to be detected. Among them, the recognition result includes multiple attributes corresponding to multiple target objects in the image to be detected. Specifically, the image to be detected contains multiple characters, each of which has its specific attributes, including static features and dynamic features of each character. For example, the static feature root can be the gender, clothing color, etc. corresponding to each character, and the dynamic feature can include the behavior and posture of each character.

[0103] In the above-mentioned image recognition method, by combining the gating unit and multiple expert units, the scheduling coefficient can be used to dynamically determine the expert unit involved in feature processing among multiple expert units, and the feature processing of different expert units can be associated and information shared, and then the feature fusion unit fuses the prompt feature and the optimized feature map to obtain the feature word, and by optimizing the detailed information of the feature map and the global information of the prompt feature, the feature expression ability of the feature word is enhanced, and the understanding of the image to be detected by the image recognition model is improved. Finally, the recognition result is determined by the feature word, which can improve the recognition ability of each attribute recognition in complex real scenes, thereby improving the accuracy of attribute recognition.

[0104] In an exemplary embodiment, a method for training an image recognition model is provided, which is applied before step 102 and includes:

[0105] Step 1011, obtaining a target image set and an image recognition model to be trained.

[0106] Among them, each target image contained in the target image set includes a manually annotated sample label and a sample image containing a target object, and the extraction unit in the image recognition model to be trained is a frozen pre-trained model.

[0107] Step 1012, input each sample image set into the image recognition model to be trained to obtain a training recognition result, and determine the loss value of the image recognition model to be trained according to the training recognition result, the sample label and the preset loss function.

[0108] In the embodiment of the present application, the preset loss function is as follows:

[0109]

[0110] in, is the balance coefficient, and in this embodiment, the default value is 1e-3. Using cross entropy loss (CE, cross entropy), which is the sum of the losses of each task:

[0111]

[0112] Among them, ∑ represents sum, M represents the number of tasks in the image recognition model, and M also represents the number of attribute recognition units.

[0113] Furthermore, It represents the balance loss of the hybrid expert unit composed of a gating unit and multiple expert units, which is specifically:

[0114]

[0115] Furthermore, It is expressed as:

[0116]

[0117] Where M represents the number of tasks, and CV represents the squared coefficient of variation. The specific formula is:

[0118]

[0119] Among them, var represents variance, mean represents mean, Represents the allowable deviation and acts as a smoothing term to avoid the denominator being zero.

[0120] Furthermore, It is expressed as:

[0121]

[0122] Where M represents the number of tasks and CV represents the squared coefficient of variation.

[0123] Furthermore, combined with the formula of the gate control unit , Specifically:

[0124]

[0125] Among them, CDF (cumulative distribution function) represents the cumulative distribution function of the normal distribution. Indicates the gating result excluding the jth bit.

[0126] Step 1013, adjusting the model parameters of the image recognition model to be trained based on the loss value until the loss value meets the preset loss condition, thereby obtaining a trained image recognition model.

[0127] The preset loss condition may be that the loss value satisfies a preset loss threshold or the change in the loss value is less than a preset change threshold.

[0128] In an exemplary embodiment, step 108 includes step 1081. In which:

[0129] Step 1081, weighted sum is performed on the mapping feature representations corresponding to each expert unit according to the scheduling coefficient, and converted through a preset activation function to obtain a prompt feature.

[0130] The preset activation function may be Sigmoid (an activation function), or Tanh or Relu. This embodiment does not limit the preset activation function.

[0131] In the embodiment of the present application, the terminal may combine the scheduling coefficient with the mapping feature representation corresponding to each expert unit by weighted summation to obtain the prompt feature P. The specific formula is as follows:

[0132]

[0133] Wherein, x is the feature input, which is the optimized feature graph Y in this embodiment, and N is the number of expert units, which is 4 in this embodiment. are expert units, each of which has a corresponding scheduling coefficient.

[0134] like Figure 2 As shown, Figure 2 Contains gate control unit ( ) and the model structure of the expert unit (Linear). The inputs of the gating unit and the expert unit are both the optimized feature map Y. The scheduling coefficient G output by the gating unit is applied to the mapping feature representation output by each expert unit (Linear), and the weighted summation of each mapping feature representation is performed. Then, the nonlinear transformation is added through the Sigmoid function to determine the importance of each expert unit for the recognition result, and the prompt feature P is obtained.

[0135] In this embodiment, by using the scheduling coefficient to perform weighted summation on the mapping feature representation of each expert unit, the global information mapped by each expert unit on its corresponding dimension can be screened and summarized, and the important information contained in the optimized feature map can be concentrated. By increasing the nonlinear relationship between the mapping feature representation and the prompt feature learning through the preset activation function, the image recognition model focuses on the features most relevant to the recognition results, enhances the sensitivity of the image recognition model to the global information corresponding to each expert unit, and improves the recognition accuracy of the image recognition model.

[0136] In an exemplary embodiment, step 108 includes step 1082. In which:

[0137] Step 1082, perform dot product summation on the prompt feature and the optimized feature map according to the feature fusion unit to obtain the feature word corresponding to the optimized feature map.

[0138] In the embodiment of the present application, after the information extraction unit completes the information extraction of the original feature map and obtains the optimized feature map, the terminal uses the optimized feature map as the input of the gating unit and the feature fusion unit respectively, wherein the optimized feature map obtains the prompt feature after being processed by the gating unit and the feature fusion unit, that is, the same optimized feature map is input into the gating unit and the feature fusion unit respectively, wherein the optimized feature map first obtains the prompt feature after being processed by the gating unit and the expert unit, and then the terminal directly inputs the original optimized feature map output by the information extraction unit into the feature fusion unit at the same time, and then, the terminal combines the prompt feature and the optimized feature map through the feature fusion unit of the image recognition model, such as Figure 2 As shown, the feature fusion unit (composed of Figure 2 The prompt feature is applied to the optimized feature map by dot multiplication and summation, and each feature element in the optimized feature map is weighted according to the weight in the prompt feature to obtain the feature word T. The specific formula is as follows:

[0139]

[0140] Among them, Figure 2 As shown, sum represents matrix summation and * represents matrix element multiplication.

[0141] The hint feature is used to assign different importance to different feature elements in the optimized feature map, so as to achieve the fusion of the hint feature and the information in the optimized feature map. The hint feature contains the importance information corresponding to different feature elements, and the optimized feature map contains the actual feature values ​​corresponding to the image to be detected. Then, the importance information is combined with the actual feature values ​​through dot product summation to obtain a new feature representation, namely, feature word.

[0142] In this embodiment, through the dot product operation, the image recognition model can pay more attention to the feature elements emphasized by the prompted features, so that the feature words can effectively represent the key information contained in the image to be detected, and then perform attribute recognition through the feature words, thereby improving the accuracy of attribute recognition by the image recognition model.

[0143] In an exemplary embodiment, Figure 3 As shown, the image recognition model further includes an attribute recognition unit, which includes a plurality of multi-layer perceptrons, each of which is used to recognize each target attribute, and step 110 includes steps 302 to 306. Among them:

[0144] Step 302, mapping transformation is performed on the feature word-units according to the mapping relationship between the feature word-units and the target attributes in each multi-layer perceptron to obtain an intermediate feature representation.

[0145] In the embodiment of the present application, after the terminal obtains the feature word corresponding to the image to be detected through the feature fusion unit of the image recognition model, the feature word is input into the attribute recognition unit. The attribute recognition unit is mainly composed of multi-layer perceptrons with the same structure and independent tasks. The intermediate feature representation of the image to be detected is first obtained through each multi-layer perceptron. Each multi-layer perceptron corresponds to a target attribute.

[0146] The target attributes can be pre-set and trained based on the task of specific attribute recognition. For example, in the scenario where the specific attribute recognition task is applied to monitor the characteristics of pedestrians entering and leaving, the target attributes that need to be recognized by the multi-layer perceptron may include the gender, clothing color, objects carried, etc. of the person. Then, according to the pre-set target attributes, each multi-layer perceptron learns the mapping relationship between the feature word element of the sample image and the target attribute during the training process of the image recognition model. Then, in the application process of the image recognition model, each multi-layer perceptron performs mapping transformation on the feature word element according to the mapping relationship between the feature word element and the target attribute to obtain an intermediate feature representation.

[0147] Step 304, performing probability normalization transformation on the intermediate feature representation output by each multi-layer perceptron to obtain a probability distribution result corresponding to each multi-layer perceptron.

[0148] Among them, the probability distribution result represents the probability prediction of the target recognition attribute of the multi-layer perceptron.

[0149] In the present application embodiment, Figure 4 As shown in the figure, a multilayer perceptron (MLP) is connected to a normalization layer. After the multilayer perceptron outputs the intermediate feature representation, the terminal performs probability normalization transformation on the intermediate feature representation output by each multilayer perceptron according to the normalization layer, and outputs the probability distribution result for the corresponding intermediate feature representation, wherein the normalization layer can be a softmax layer. Specifically, the multilayer perceptron includes an input layer, a first-level hidden layer, and an output layer. The dimension of the first-level hidden layer of each multilayer perceptron is 1 / 4 of the input dimension. The intermediate features are processed by the input layer, the first-level hidden layer, and the output layer of the multilayer perceptron. The obtained analysis results are then passed through the softmax layer to obtain the final recognition results. .

[0150] For each multilayer perceptron, the multilayer perceptron corresponds to a target attribute. For example, the target attribute of the current multilayer perceptron is to identify the color of the target object in the image to be detected. After the intermediate feature representation of the current multilayer perceptron is subjected to probability normalization transformation, the probability of the target object corresponding to each color is obtained, which constitutes the probability distribution result of the current multilayer perceptron.

[0151] Step 306: Determine the recognition result corresponding to the image to be detected based on the probability distribution result.

[0152] In the embodiment of the present application, each multi-layer perceptron obtains a probability distribution result of the target attribute of each dimension, and the probability distribution result includes classification probabilities corresponding to multiple candidate results. Based on the probability distribution result, the terminal can determine the candidate result with the highest probability as the output result corresponding to the multi-layer perceptron, and then obtain the recognition results of the target attributes of multiple dimensions corresponding to the image to be detected. The terminal outputs the recognition results corresponding to the target attributes of multiple dimensions as the recognition results of the image to be detected.

[0153] In this embodiment, each multilayer perceptron has the same structure but different recognition tasks, so that when each multilayer perceptron maps feature words, since the recognition of different target attributes may depend on different information of feature words, each multilayer perceptron can focus on different feature patterns to improve the recognition accuracy of each target attribute.

[0154] In an exemplary embodiment, Figure 5 As shown, the image recognition model also includes a target detection unit, and step 104 includes steps 502 to 506. Among them:

[0155] Step 502 , obtaining an image to be detected, performing target detection on the image to be detected according to a target detection unit, and obtaining a target calibration frame of a target object in the image to be detected.

[0156] In an embodiment of the present application, the target detection unit of the image recognition model may be an existing target detection model, such as YOLO (You Only Look Once, a target detection model). After the image recognition model acquires the image to be detected, the terminal performs target detection on the target object in the image to be detected through the target detection unit, and obtains a target calibration frame corresponding to the target object. For example, in a scenario of monitoring the characteristics of pedestrians entering and exiting, the image to be detected may include one or more pedestrians. At this time, the target object is a pedestrian. The target detection unit can perform target detection on the pedestrians contained in the image to be detected, and use the calibration frame to mark the pedestrians in the image to be detected to obtain a target calibration frame for the pedestrians.

[0157] Step 504 , cropping the image to be detected according to the target calibration frame to obtain a target image corresponding to the target object.

[0158] In an embodiment of the present application, the terminal crops the image to be detected according to the target calibration frame, eliminates the background and other information of the image to be detected, and only leaves the information in the target calibration frame containing the target object, thereby obtaining a target image containing only the target object, so as to avoid interference of background and other information on subsequent image recognition.

[0159] Step 506: extract features from the target image based on the extraction unit to obtain an optimized feature map.

[0160] In the embodiment of the present application, the terminal performs feature extraction on the target image obtained after cropping according to the same principle as in step 104 to obtain an optimized feature map. The specific process of feature extraction will not be described in detail in the embodiment of the present application.

[0161] In this embodiment, the target image is obtained by cropping the image to be detected through the target calibration frame corresponding to the target object, which can eliminate the interference of background information and other information on image recognition, so that only relevant features of the target object are included in the optimized feature map, thereby improving the accuracy of attribute recognition of the image to be detected by the image recognition model.

[0162] In an exemplary embodiment, Figure 6 As shown, the extraction unit includes a feature extraction unit and an information extraction unit, and step 506 includes steps 602 to 604. Among them:

[0163] Step 602: extract features from the target image using a feature extraction unit to obtain an original feature map corresponding to the target image.

[0164] In the embodiment of the present application, the structure of the feature extraction unit of the image recognition unit can be a feature extraction network using a pre-trained visual self-attention model (vision transformer) in a deep neural network. Taking the visual self-attention model as an example, the terminal extracts features of the target image through the backbone network (backbone) of the feature extraction unit to obtain the original feature Figure X ,in , D, H, and W are the original features Figure X The vector dimensions, height and width.

[0165] Step 604: perform local information extraction on the original feature map according to the information extraction unit to obtain an optimized feature map.

[0166] In the embodiment of the present application, the information extraction unit may be a predefined filter or neural network layer. The terminal inputs the original feature map into the information extraction unit, extracts the local information of the original feature map by sliding the convolution kernel in the convolution layer, and extracts specific information from each local area of ​​the original feature map, for example, by sliding the convolution kernel on the original feature map, performing a dot product operation on each local area, generating a new feature value, and then forming an optimized feature map.

[0167] In this embodiment, the feature extraction unit is first used to extract features of the target image, and the terminal can obtain the original feature map of the target image, and then the information extraction unit extracts local information from these original feature maps to obtain optimized feature maps. By extracting local information from the original feature map, the quality of the optimized feature map can be effectively improved, and more representative and discriminative local features can be extracted, thereby enhancing the accuracy of subsequent expert units mapping the optimized feature map, thereby improving the accuracy of attribute recognition by the image recognition model.

[0168] In an exemplary embodiment, Figure 7 As shown, the information extraction unit includes a cascaded convolution module, a batch normalization layer, an activation layer and a pooling layer, and step 604 includes steps 702 to 704. Wherein:

[0169] Step 702: extract and enhance the local spatial information of the original feature map according to the cascaded convolution module, batch normalization layer and activation layer to obtain an initial optimized feature map.

[0170] In the embodiment of the present application, the terminal inputs the original feature map into the information extraction unit for information extraction. Figure 8 As shown in Figure 1, the original feature map is first input into two cascaded convolution modules, which consist of a convolution kernel with a size of The convolution layer (Conv) is used, and the batch normalization layer (BN) and activation layer (ReLU) are used to gradually enhance the refinement of local information on the original feature map to obtain the initial optimized feature map. Among them, the use of the batch normalization layer (BN) can speed up the network convergence speed of the image recognition model and appropriately reduce overfitting. Then, the activation layer (ReLU) is used to activate the normalized original feature map and add nonlinearity to the initial optimized feature map.

[0171] Step 704, performing dimensionality reduction sampling on the initial optimized feature map according to the pooling layer to obtain an optimized feature map.

[0172] In the embodiment of the present application, the terminal inputs the initial optimized feature map into the pooling layer (MaxPool2D), and further simplifies the information of the initial optimized feature map through the pooling layer to reduce the amount of calculation for subsequent image recognition, thereby making the image recognition model converge better. The pooling layer is composed of a maximum pooling (MaxPool2D) with a kernel size of 2 and a step size of 2, and outputs the optimized feature map Y, where , , , , Optimize the vector dimension, height and width of the feature map Y respectively.

[0173] In this embodiment, the local information of the original feature map is refined through the cascade convolution module, batch normalization layer and activation layer to obtain the initial optimized feature map, which can highlight the features in the original feature map that have a stronger correlation with the target attribute, and improve the training convergence speed of the image recognition model and reduce overfitting. Finally, the pooling layer is used for dimensionality reduction sampling to simplify the feature information contained in the initial optimized feature map to obtain the optimized feature map. Using the optimized feature map for subsequent attribute recognition can not only improve the efficiency of the image recognition model in attribute recognition processing, but also improve the accuracy of attribute recognition.

[0174] In an exemplary embodiment, an example of an image recognition method is provided, such as Fig. 9 As shown, including:

[0175] Step 901, inputting the target image into the image recognition model;

[0176] Step 902, extracting features from the target image using a feature extraction unit to obtain an original feature map corresponding to the target image;

[0177] Step 903, extracting local information from the original feature map according to the information extraction unit to obtain an optimized feature map, and inputting the optimized feature map into the gating unit and the feature fusion unit respectively;

[0178] Step 904, determining a scheduling coefficient based on the gating unit and the optimized characteristic graph;

[0179] Step 905, performing a linear transformation on the optimized feature map according to the plurality of expert units to obtain a mapping feature representation corresponding to each expert unit;

[0180] Step 906, determining a prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and performing feature fusion on the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word;

[0181] Step 907, mapping the characteristic word-units according to the recognition unit to obtain a probability distribution result;

[0182] Step 908: Determine the recognition result corresponding to the image to be detected based on the probability distribution result.

[0183] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0184] Based on the same inventive concept, the embodiment of the present application also provides an image recognition device for implementing the above-mentioned image recognition method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image recognition device embodiments provided below can refer to the limitations on the image recognition method above, and will not be repeated here.

[0185] In an exemplary embodiment, Fig.10 As shown, an image recognition device 1000 is provided, comprising: an acquisition module 1001, a feature extraction module 1002, a linear transformation module 1003, a feature fusion module 1004 and a determination module 1005, wherein:

[0186] The acquisition module 1001 is used to acquire the image to be detected and the image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit;

[0187] The feature extraction module 1002 is used to extract features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determine the scheduling coefficient based on the gating unit and the optimized feature map;

[0188] A linear transformation module 1003 is used to perform a linear transformation on the optimized feature map according to multiple expert units to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature map;

[0189] A feature fusion module 1004 is used to determine the prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and perform feature fusion on the prompt feature and the optimized feature graph based on a feature fusion unit to obtain a feature word unit;

[0190] The determination module 1005 is used to determine the recognition result of the image to be detected based on the feature word.

[0191] In one of the embodiments, the feature fusion module 1004 is specifically used to perform weighted summation on the mapping feature representations corresponding to each expert unit according to the scheduling coefficient, and convert them through a preset activation function to obtain the prompt feature.

[0192] In one embodiment, the feature fusion module 1004 is specifically configured to perform dot product summation on the prompt feature and the optimized feature graph according to the feature fusion unit to obtain a feature word corresponding to the optimized feature graph.

[0193] In one embodiment, the image recognition model further includes an attribute recognition unit, the attribute recognition unit includes a plurality of multi-layer perceptrons, each of which is used to recognize each target attribute; the determination module 1005 is specifically used to perform mapping transformation on the feature word element according to the mapping relationship between the feature word element and the target attribute in each multi-layer perceptron to obtain an intermediate feature representation;

[0194] Perform probability normalization transformation on the intermediate feature representation output by each multi-layer perceptron to obtain the probability distribution result corresponding to each multi-layer perceptron; the probability distribution result represents the probability prediction of the target attribute of the multi-layer perceptron;

[0195] The recognition result corresponding to the image to be detected is determined based on the probability distribution result.

[0196] In one embodiment, the image recognition model further includes a target detection unit; the feature extraction module 1002 is specifically used to obtain an image to be detected, perform target detection on the image to be detected according to the target detection unit, and obtain a target calibration frame of a target object in the image to be detected;

[0197] The image to be detected is cropped according to the target calibration frame to obtain the target image corresponding to the target object;

[0198] Based on the extraction unit, features of the target image are extracted to obtain an optimized feature map.

[0199] In one embodiment, the extraction unit includes a feature extraction unit and an information extraction unit; the feature extraction module 1002 is specifically used to extract features of the target image according to the feature extraction unit to obtain an original feature map corresponding to the target image;

[0200] The original feature map is locally refined according to the information extraction unit to obtain an optimized feature map.

[0201] In one embodiment, the information extraction unit includes a cascade convolution module, a batch normalization layer, an activation layer, and a pooling layer; the feature extraction module 1002 is specifically used to perform local spatial information enhancement and extraction on the original feature map according to the cascade convolution module, the batch normalization layer, and the activation layer to obtain an initial optimized feature map;

[0202] The initial optimized feature map is sampled by dimension reduction according to the pooling layer to obtain the optimized feature map.

[0203] Each module in the above-mentioned image recognition device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0204] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store images to be detected and recognition results. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image recognition method is implemented.

[0205] Those skilled in the art will understand that Fig.11The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0206] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0207] Obtaining an image to be detected and an image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit;

[0208] Extract features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determine a scheduling coefficient based on the gating unit and the optimized feature map;

[0209] According to the plurality of expert units, a linear transformation is performed on the optimized feature graph to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature graph;

[0210] Determine the prompt feature corresponding to the optimized feature map according to the scheduling coefficient and the mapping feature representation, and perform feature fusion on the prompt feature and the optimized feature map based on the feature fusion unit to obtain the feature word unit;

[0211] The recognition result of the image to be detected is determined based on the feature word.

[0212] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0213] The mapping feature representations corresponding to each expert unit are weighted and summed according to the scheduling coefficient, and converted through a preset activation function to obtain the prompt feature.

[0214] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0215] According to the feature fusion unit, the prompt feature and the optimized feature map are dot-producted and summed to obtain the feature word corresponding to the optimized feature map.

[0216] In one embodiment, the image recognition model further includes an attribute recognition unit, the attribute recognition unit includes a plurality of multi-layer perceptrons, each of which is used to recognize each target attribute; when the processor executes the computer program, the following steps are also implemented:

[0217] According to the mapping relationship between the feature word unit and the target attribute in each multi-layer perceptron, the feature word unit is mapped and transformed to obtain the intermediate feature representation;

[0218] Perform probability normalization transformation on the intermediate feature representation output by each multi-layer perceptron to obtain the probability distribution result corresponding to each multi-layer perceptron; the probability distribution result represents the probability prediction of the target attribute of the multi-layer perceptron;

[0219] The recognition result corresponding to the image to be detected is determined based on the probability distribution result.

[0220] In one embodiment, the image recognition model further includes a target detection unit; when the processor executes the computer program, the following steps are also implemented:

[0221] Acquire an image to be detected, perform target detection on the image to be detected according to the target detection unit, and obtain a target calibration frame of the target object in the image to be detected;

[0222] The image to be detected is cropped according to the target calibration frame to obtain the target image corresponding to the target object;

[0223] Based on the extraction unit, features of the target image are extracted to obtain an optimized feature map.

[0224] In one embodiment, the extraction unit includes a feature extraction unit and an information extraction unit; when the processor executes the computer program, the processor also implements the following steps:

[0225] Extract features of the target image according to the feature extraction unit to obtain an original feature map corresponding to the target image;

[0226] The original feature map is subjected to local information extraction according to the information extraction unit to obtain an optimized feature map.

[0227] In one embodiment, the information extraction unit includes a cascaded convolution module, a batch normalization layer, an activation layer, and a pooling layer; when the processor executes the computer program, the following steps are also implemented:

[0228] According to the cascaded convolution module, batch normalization layer and activation layer, the original feature map is enhanced and refined in local space to obtain the initial optimized feature map;

[0229] The initial optimized feature map is sampled by dimension reduction according to the pooling layer to obtain the optimized feature map.

[0230] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0231] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0232] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0233] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0234] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0235] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image recognition method, characterized in that: The method comprises: Acquire an image to be detected and an image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit; Extracting features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determining a scheduling coefficient based on the gating unit and the optimized feature map; According to the plurality of expert units, the optimized feature graph is linearly transformed to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature graph; Determining a prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and performing feature fusion on the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word; A recognition result of the image to be detected is determined based on the feature word.

2. The method according to claim 1, characterized in that The determining, according to the scheduling coefficient and the mapping feature representation, a prompt feature corresponding to the optimization feature graph comprises: The mapping feature representations corresponding to the expert units are weighted and summed according to the scheduling coefficients, and converted through a preset activation function to obtain prompt features.

3. The method according to claim 1, characterized in that The step of fusing the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word unit includes: The feature fusion unit performs dot product summation on the prompt feature and the optimized feature map to obtain a feature word corresponding to the optimized feature map.

4. The method according to claim 1, characterized in that: The image recognition model further includes an attribute recognition unit, wherein the attribute recognition unit includes a plurality of multi-layer perceptrons, each of which is used to recognize each target attribute; The determining the recognition result of the image to be detected based on the feature word comprises: According to the mapping relationship between the feature word unit and the target attribute in each of the multi-layer perceptrons, the feature word unit is mapped and transformed to obtain an intermediate feature representation; Performing probability normalization transformation on the intermediate feature representation output by each of the multilayer perceptrons to obtain a probability distribution result corresponding to each of the multilayer perceptrons; the probability distribution result represents the probability prediction of the target attribute of the multilayer perceptron; A recognition result corresponding to the image to be detected is determined based on the probability distribution result.

5. The method according to claim 1, characterized in that: The image recognition model further includes a target detection unit; the feature extraction of the image to be detected based on the extraction unit to obtain an optimized feature map includes: Acquire an image to be detected, perform target detection on the image to be detected according to the target detection unit, and obtain a target calibration frame of a target object in the image to be detected; Cropping the image to be detected according to the target calibration frame to obtain a target image corresponding to the target object; Based on the extraction unit, feature extraction is performed on the target image to obtain an optimized feature map.

6. The method according to claim 5, characterized in that The extraction unit includes a feature extraction unit and an information extraction unit; the feature extraction of the target image based on the extraction unit to obtain an optimized feature map includes: Extracting features from the target image using the feature extraction unit to obtain an original feature map corresponding to the target image; The information extraction unit performs local information extraction on the original feature map to obtain an optimized feature map.

7. The method according to claim 6, characterized in that The information extraction unit includes a cascade convolution module, a batch normalization layer, an activation layer and a pooling layer; the information extraction unit performs local information extraction on the original feature map to obtain an optimized feature map, including: According to the cascaded convolution module, batch normalization layer and activation layer, the original feature map is subjected to local spatial information enhancement and refinement to obtain an initial optimized feature map; The initial optimized feature map is sampled by dimensionality reduction according to the pooling layer to obtain an optimized feature map.

8. An image recognition device, characterized in that: The device comprises: An acquisition module, used to acquire an image to be detected and an image recognition model; the image recognition model includes an extraction unit, a gating unit, multiple expert units and a feature fusion unit; A feature extraction module, configured to extract features of the image to be detected based on the extraction unit to obtain an optimized feature map, and determine a scheduling coefficient based on the gating unit and the optimized feature map; A linear transformation module, used to perform a linear transformation on the optimized feature map according to the plurality of expert units to obtain a mapping feature representation corresponding to each expert unit; each mapping feature representation reflects global information of different dimensions in the optimized feature map; A feature fusion module, used to determine the prompt feature corresponding to the optimized feature graph according to the scheduling coefficient and the mapping feature representation, and perform feature fusion on the prompt feature and the optimized feature graph based on the feature fusion unit to obtain a feature word; A determination module is used to determine a recognition result of the image to be detected based on the feature word.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images