Object recognition method and related apparatus

Abstract

The application provides an object recognition method and related device, and relates to the technical field of object recognition. In the application, the illumination compensation result and the texture enhancement result of the initial image are linearly superimposed and fused, the problem of uneven illumination and texture weakening when collecting an object image can be solved through superposition of the illumination compensation and the texture enhancement operation, and the recognition accuracy of the object category is improved. In addition, the intermediate image is subjected to a convolution operation to obtain an output feature map, a preset attention weight function and the output feature map are used to obtain an attention weight map, the output feature map and the attention weight map are multiplied element by element to obtain an attention weighted feature map, the recognition ability of high-frequency details is improved, and the recognition accuracy of the object category is further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of object recognition, and in particular to an object recognition method and related device. BACKGROUND

[0002] With the development of science and technology, object type recognition operations are becoming more and more important. Taking cultural relics as an example, with the promotion of digital cultural relic protection and smart museum construction, cultural relic image recognition technology shows a wide application prospect in the fields of cultural relic digitization, automatic classification, virtual display, etc. However, cultural relic images usually have high complexity of texture features, diversification of material performance and subtle differences between classes, etc., which makes it difficult to accurately identify the type of cultural relics. SUMMARY

[0003] In view of the above problems, the present application provides an object recognition method and related device to accurately identify the type of objects. The specific scheme is as follows:

[0004] The first aspect of the present application provides an object recognition method, comprising:

[0005] obtaining an initial image of a target object;

[0006] linearly superimposing and fusing the illumination compensation result and the texture enhancement result of the initial image to obtain an intermediate image;

[0007] performing convolution operation on the intermediate image to obtain an output feature map, and using a preset attention weight function and the output feature map to obtain an attention weight map;

[0008] element-wise multiplying the output feature map and the attention weight map to obtain an attention weighted feature map, and performing classification operation on the attention weighted feature map to obtain an object type recognition result of the target object.

[0009] The second aspect of the present application provides an object recognition device, comprising:

[0010] an image acquisition module configured to acquire an initial image of a target object;

[0011] a fusion module configured to linearly superimpose and fuse the illumination compensation result and the texture enhancement result of the initial image to obtain an intermediate image;

[0012] a processing module configured to perform convolution operation on the intermediate image to obtain an output feature map, and use a preset attention weight function and the output feature map to obtain an attention weight map;

[0013] The identification module is configured to perform element-wise multiplication on the output feature map and the attention weight map to obtain an attention weighted feature map, and perform a classification operation on the attention weighted feature map to obtain an object category identification result of the target object.

[0014] The third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0015] The memory is configured to store a computer program.

[0016] The processor is configured to execute the computer program to enable the electronic device to implement the object identification method.

[0017] The fourth aspect of the present application provides a computer storage medium, the storage medium carries one or more computer programs, when the one or more computer programs are executed by an electronic device, the electronic device can implement the object identification method.

[0018] By the above technical solution, the present application provides an object identification method and related devices. In the present application, the linear superposition fusion of the illumination compensation result and the texture enhancement result of the initial image can solve the problems of uneven illumination and weakened texture when collecting an object image through the superposition of illumination compensation and texture enhancement operations, and improve the identification accuracy of object categories. In addition, the intermediate image is subjected to a convolution operation to obtain an output feature map, a preset attention weight function and the output feature map are used to obtain an attention weight map, element-wise multiplication is performed on the output feature map and the attention weight map to obtain an attention weighted feature map, and the identification ability of high-frequency details is improved, thereby improving the identification accuracy of object categories. BRIEF DESCRIPTION OF DRAWINGS

[0019] The above and other features, advantages, and aspects of the present disclosure will become more apparent by describing in detail the following specific embodiments with reference to the attached drawings. Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0020] Figure 1 A flowchart of an object identification method provided by the present application is shown in FIG. 1;

[0021] Figure 2 A schematic diagram of an initial image provided by the present application is shown in FIG. 2;

[0022] Figure 3 A flowchart of a method for obtaining an intermediate image provided by the present application is shown in FIG. 3;

[0023] Figure 4A schematic diagram of a processed image after illumination compensation provided by the present application;

[0024] Figure 5 A schematic diagram of a texture-enhanced image provided by the present application;

[0025] Figure 6 A schematic diagram of an intermediate image provided by the present application;

[0026] Figure 7 A performance comparison chart provided by the present application;

[0027] Figure 8 A flowchart of a method for determining initial weights provided by the present application;

[0028] Figure 9 Another performance comparison chart provided by the present application;

[0029] Figure 10 A schematic diagram of an output feature map provided by the present application;

[0030] Figure 11 A schematic diagram of a texture-enhanced feature heat map provided by the present application;

[0031] Figure 12 A structural schematic diagram of an object recognition device provided by the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application are described below in conjunction with the accompanying drawings of the embodiments of the present application. The terms used in the implementation part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. As known by those skilled in the art, with the development of technology and the appearance of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0033] With the advancement of digital cultural relic protection and the construction of smart museums, cultural relic image recognition technology has shown wide application prospects in the fields of digitalization of cultural relics, automatic classification, virtual display, etc. However, cultural relic images usually have high complexity of texture features, diversified material performance, and subtle differences between classes, etc. At the same time, they are often affected by factors such as uneven illumination, changes in viewing angle, background interference, etc. during actual collection, resulting in problems such as low recognition accuracy, unstable feature extraction, and poor robustness when traditional image enhancement methods and general recognition models are used to process cultural relic images.

[0034] In addition, in conventional technologies, image processing methods such as histogram equalization and global normalization are used, but cultural relic images are sensitive to changes in illumination and texture, which can easily cause loss of texture or over-enhancement.

[0035] In addition, random initialization or global mean initialization can also be used in conventional techniques, ignoring the texture distribution of cultural relic images, resulting in unstable training and inaccurate feature extraction.

[0036] In addition, in conventional techniques, only relying on cross-entropy cannot effectively enhance the inter-class discrimination and intra-class cohesion, especially in high-similarity cultural relic recognition, the accuracy is low.

[0037] Therefore, in the embodiments of the present application, in the aspect of image preprocessing, a multi-scale Retinex (Retina + Cortex, retinal and cerebral cortex theory) and adaptive direction Gabor filter combined enhancement algorithm is introduced, which effectively solves the problems of uneven illumination and texture weakening. In terms of model structure, a weight initialization strategy that integrates local texture statistical features and an adaptive texture attention mechanism are designed to improve the recognition ability of high-frequency details. In terms of training mechanism, by integrating the total discriminative loss function of cross-entropy, center loss and adaptive margin loss, the discriminability and cohesion of the feature space are strengthened.

[0038] In addition, in the application of recognition results, the classification output is first connected with the cultural relic knowledge graph to support cultural interpretation and three-dimensional interactive display, so as to build a full-process closed-loop system from recognition to popularization.

[0039] On the basis of the above, an embodiment of the present application provides an object recognition method, referring to Figure 1 may include:

[0040] S11, obtaining an initial image of a target object.

[0041] Among them, the target object in the embodiment can be cultural relics, antiques, treasures, curios, antiques, etc. Taking cultural relics as an example, cultural relics can be porcelain, bronze ware, calligraphy and painting, etc. In a standardized photographic studio environment, a high-resolution digital camera is used to shoot clear images of cultural relics from one or more fixed angles (such as front view, overhead view, side view), ensuring that the main feature surface of the cultural relics is covered.

[0042] The collected clear image is the initial image in the present application, and the initial image can focus on key visual features that can reflect class differences, such as the overall shape of the object, typical decorative patterns, material surface characteristics, etc.

[0043] In an example, the initial image shot can be a celadon four-system jar included in a museum, as shown in Figure 2 .

[0044] S12, linearly superimposes and fuses the illumination compensation result and the texture enhancement result of the initial image to obtain an intermediate image.

[0045] Specifically, the cultural relic image data usually has problems such as uneven illumination, low contrast, blurred details and background noise interference, which easily leads to difficulty in effectively extracting key texture features. Conventional global normalization techniques such as Min-Max scaling cannot adaptively process local feature differences, are sensitive to illumination changes and detail loss, and are easy to cause loss of subtle texture information, thereby reducing the model recognition accuracy.

[0046] To this end, in the embodiments of the present application, first, a multi-scale Retinex method is used to compensate for the illumination of the initial image, the illumination correction results under different scales are fused through dynamic weights, the uneven illumination of the image is eliminated and the overall contrast is improved, and the illumination compensation result is obtained. Then, the image is convolved by adaptive directional Gabor filtering, the key texture features in a specific direction are strengthened and the background noise is suppressed, and the texture enhancement result is obtained. Finally, the illumination compensation result and the texture enhancement result are linearly superimposed and fused to generate the final intermediate image.

[0047] In an implementation manner, referring to Figure 3 , step S12 can include:

[0048] S21, decompose the illumination component of the initial image, and obtain the illumination compensation result by fusing the illumination correction results under each scale through dynamic weights.

[0049] Specifically, when performing multi-scale Retinex illumination compensation, the initial image is decomposed into an illumination component by multi-scale Gaussian convolution, and the illumination correction results under each scale are fused through dynamic weights to eliminate uneven illumination of the image and improve overall contrast. The specific calculation formula is:

[0050] ;

[0051] Wherein, represents the Retinex output under the s-th scale, representing the illumination correction result, which can be an illumination normalization result;

[0052] is the s-th initial image; if there is only one initial image, then the value of s is 1.

[0053] is the standard deviation of the Gaussian kernel of the s-th scale, which controls the extraction range of the illumination component;

[0054] is a Gaussian filter kernel with a standard deviation of ;

[0055] represents convolution operation, which is used to extract the illumination component; ​

[0056] This is a logarithmic transformation function, with the default base being the natural constant.

[0057] Retinex is an image enhancement and color constancy theory. Specifically, it is an image enhancement framework that simulates the human visual mechanism. By separating the illumination and reflection components, it achieves dynamic range compression, color constancy, and detail enhancement.

[0058] In one example, for Figure 2 The initial image in the image is processed after multi-scale Retinex illumination compensation. (Image reference) Figure 4 As shown.

[0059] S22. Based on the weight decay coefficient and the difference between the gradient matrix of the initial image and the gradient clustering centers at each scale, the adaptive weights are calculated.

[0060] Specifically, after obtaining the illumination compensation result, it is also necessary to calculate the adaptive weight of the illumination compensation result. The adaptive weight specifically refers to the adaptive multi-scale weight, and its specific calculation process is as follows:

[0061] Based on the difference between the gradient matrix of the initial image and the gradient cluster centers at each scale, the sensitivity is controlled by a weight decay coefficient, and the weights at each scale are dynamically allocated to preserve local detail features. The specific calculation formula is as follows:

[0062] ;

[0063] in, The adaptive weights for the s-th scale determine the contribution of that scale to the fusion.

[0064] For the initial image The gradient matrix represents the edge intensity distribution; if there is only one initial image, then The value is 1. The gradient matrix is ​​obtained by applying the initial image... The image was obtained through standard edge detection, specifically by using the Sobel operator to calculate the gradient components in the horizontal direction. and gradient components in the vertical direction Then through The gradient magnitude is calculated pixel by pixel, and finally a gradient matrix representing the intensity distribution of image edges is generated.

[0065] The gradient cluster centers corresponding to the s-th scale are obtained by clustering the sample gradients using the K-means algorithm.

[0066] For example, the weight decay coefficient. =1.5, to control the sensitivity to gradient differences;

[0067] The total number of multiple scales is automatically determined by cluster analysis based on the complexity of the illumination distribution;

[0068] It is an L2 norm;

[0069] It is an exponential function.

[0070] S23. Filter the initial image using an adaptive orientation angle to obtain the texture enhancement result.

[0071] Specifically, adaptive directional Gabor texture enhancement can be used for filtering. Specifically, the adaptive directional angle is determined by maximizing the variance of the Gabor filter response. A Gabor filter kernel with this adaptive directional angle is then used to perform a convolution operation on the initial image, enhancing texture features in a specific direction and suppressing noise. The specific calculation formula is as follows:

[0072] ;

[0073] ;

[0074] in, For the first The adaptive orientation angle of an initial image is determined by maximizing the variance of the filter response; if there is only one initial image, then... The value is 1;

[0075] For the first There is one initial image; if there is only one initial image, then The value is 1;

[0076] For adaptive direction angle Gabor filter kernel;

[0077] For adaptive direction angle Gabor filter kernel;

[0078] is the variance function, which measures the significance of texture direction after filtering;

[0079] This indicates selecting the adaptive orientation angle that maximizes the variance. ;

[0080] The Gabor filter output image, i.e. the texture enhancement result, is used to enhance the texture features in a specific direction.

[0081] In one embodiment, the image after adaptive directional Gabor texture enhancement can refer to Figure 5 as shown.

[0082] In this embodiment, the texture direction is automatically selected by maximizing the variance of the filter response, the key direction texture is strengthened, the background noise is suppressed, and the key details for identifying cultural relics are preserved.

[0083] S24, using adaptive weights, linearly superimposes and fuses the illumination compensation result and the texture enhancement result to obtain an intermediate image.

[0084] Specifically, the illumination compensation result and the texture enhancement result are linearly superimposed by multi-modal enhancement fusion. Specifically, the multi-scale illumination compensation result of dynamic weight fusion is linearly superimposed with the texture enhancement result of adaptive directional Gabor texture enhancement. The texture item intensity is adjusted by the texture enhancement coefficient, and the final intermediate image after preprocessing is generated by fusion. The specific calculation formula is:

[0085] ;

[0086] Wherein, is the intermediate image after preprocessing of the first initial image;

[0087] is the texture enhancement coefficient, such as =0.8, the intensity of the Gabor item is controlled to suppress the background noise;

[0088] is the total number of multi-scale, which is automatically determined by the complexity of the illumination distribution through clustering analysis;

[0089] is the adaptive weight of the s scale;

[0090] Retinex output under the s scale, representing the illumination correction result;

[0091] is the Gabor filter output image, i.e. the texture enhancement result.

[0092] In one embodiment, the intermediate image after preprocessing can refer to Figure 6 as shown.

[0093] In one embodiment, refer to Figure 7, comparative analysis of the pretreatment effect is carried out, the effect of different pretreatment methods on the quality improvement of cultural relic image is evaluated, so as to verify the advantages of the multi-scale Retinex light compensation and adaptive direction Gabor texture enhancement fusion technology proposed in the application. The application is compared with global normalization (conventional technology), histogram equalization (conventional technology), Retinex + traditional Gabor (improved conventional technology), and the three core indicators are shown in the column chart:

[0094] Contrast enhancement rate (measure of light compensation effect): the column of contrast enhancement rate in the application is significantly higher than that of other methods, which shows that the multi-scale dynamic weight fusion effectively eliminates uneven illumination;

[0095] Detail retention degree (measure of texture feature retention ability): the column of detail retention degree in the application is outstanding, which proves that the adaptive direction Gabor filter accurately strengthens the key texture;

[0096] Noise suppression rate (measure of background interference elimination effect): the column of noise suppression rate in the application is the highest, which reflects that the texture enhancement coefficient effectively suppresses noise.

[0097] The experimental results show that the conventional method has large data fluctuation (column error reflection), which shows that it has poor adaptability to different cultural relics, while the data stability of the application is high (the column error is the smallest), which proves that the method has strong robustness, therefore, the application is superior in the three indicators, especially the noise suppression rate advantage is the most significant.

[0098] In this embodiment, by fusing light compensation and texture enhancement, the image contrast, definition and texture resolution are improved, providing high-quality input for subsequent identification. In addition, multi-scale fusion and gradient perception adaptive weight are used to dynamically adjust the light compensation effect and improve the local contrast of the image.

[0099] S13, the intermediate image is subjected to convolution operation to obtain an output feature map, and the attention weight function and the output feature map are used to obtain an attention weight map.

[0100] In the embodiment of the present application, the cultural relic image has high detail richness and inter-class similarity. The forward propagation of the conventional convolutional neural network only uses standard convolutional layers, which is difficult to effectively distinguish subtle features, resulting in a lack of discriminability of the feature map, the key texture region cannot be focused, and irrelevant background noise is retained, thereby reducing the classification accuracy. In addition, the traditional method ignores the dynamic feature selection mechanism and cannot adaptively enhance high-frequency details, which performs poorly in the scene of blurred details and inter-class similarity. Therefore, in the embodiment of the present application, when performing forward propagation calculation of the convolutional neural network, a standard convolution operation is performed on the input intermediate image to obtain an output feature map. Then, a dynamic preset attention weight function is calculated, which combines the global average information and local texture energy information of the output feature map, and generates an attention weight map that can focus on the high-frequency detail region through a Sigmoid activation function.

[0101] S14, element-wise multiplication is performed on the output feature map and the attention weight map to obtain an attention weighted feature map, and a classification operation is performed on the attention weighted feature map to obtain an object category recognition result of the target object.

[0102] Specifically, the output feature map of the standard convolution is multiplied element-wise with the dynamically generated attention weight map to generate an attention weighted feature map, which can dynamically suppress irrelevant background noise and enhance the texture features of the cultural relic subject region, thereby improving the feature discriminability. The attention weighted feature map is then output to a subsequent network layer for cultural relic category recognition to obtain an object category recognition result of the target object.

[0103] In the embodiment, the illumination compensation result and the texture enhancement result of the initial image are linearly superimposed and fused, which can solve the problems of uneven illumination and texture weakening when collecting the object image through the superposition of the illumination compensation and the texture enhancement operation, thereby improving the recognition accuracy of the object category. In addition, the intermediate image is convolved to obtain an output feature map, and the preset attention weight function and the output feature map are used to obtain an attention weight map. Element-wise multiplication is performed on the output feature map and the attention weight map to obtain an attention weighted feature map, so as to improve the recognition ability of high-frequency details and further improve the recognition accuracy of the object category.

[0104] On the basis of any of the above embodiments, in an implementation, steps S13-S14 are implemented by using an object recognition model. The object recognition model can be a cultural relic recognition model, and the object recognition model can use a hierarchical convolutional neural network architecture. The object recognition model includes a feature extraction backbone network, an adaptive attention module, and a classification output layer.

[0105] The feature extraction backbone network includes multiple groups of convolution-pooling cascade units (convolution-pooling cascade units). The first group of convolution-pooling cascade units is used to capture morphological features of a specified view, and the other groups of convolution-pooling cascade units are used to extract textures of a specified granularity level by level. The feature extraction backbone network is configured to perform convolution operations on the intermediate image by using the multiple groups of convolution-pooling cascade units to obtain an output feature map.

[0106] In an example, the feature extraction backbone network includes four groups of convolution-pooling cascade units. The first layer uses a 7*7 large convolution kernel to capture morphological features of a specified view, specifically macro-morphological features. The subsequent three layers use 5*5, 3*3, and 3*3 convolution kernels to extract textures of a specified granularity level by level, specifically fine-grained textures. Each group of convolution is followed by a max-pooling layer to compress the spatial dimension and retain significant features.

[0107] In this application, the convolution layer in the convolution-pooling cascade unit performs a standard convolution operation to obtain an output feature map. After the standard convolution operation, a max-pooling operation can be performed to obtain the output feature map.

[0108] The adaptive attention module is embedded after the convolution-pooling cascade unit of a specified group. The adaptive attention module is configured to use a preset attention weight function and an output feature map to obtain an attention weight map, multiply the output feature map and the attention weight map element by element, and obtain an attention weighted feature map.

[0109] Specifically, the adaptive attention module is embedded after the third convolution layer. The adaptive attention module is configured to strengthen high-frequency texture responses by a feature map channel weighting mechanism, use a preset attention weight function and an output feature map to obtain an attention weight map, multiply the output feature map and the attention weight map element by element, and obtain an attention weighted feature map.

[0110] The classification output layer includes a multi-level fully connected network. The multi-level fully connected network is configured to perform classification operations on the attention weighted feature map by dimension reduction and object classification operations to obtain an object category recognition result of the target object.

[0111] Specifically, the classification layer is implemented by a two-level fully connected network. The first level fully connected network reduces the attention feature vector to a 256-dimensional discriminative feature vector. The second level fully connected network uses the 256-dimensional discriminative feature vector to measure the distance from each class feature center formed in the training phase, outputs a classification probability distribution corresponding to the number of cultural relic categories, and finally selects the cultural relic type corresponding to the class center with the smallest Euclidean distance as the recognition result, i.e., the object category recognition result of the target object is obtained.

[0112] Based on the embodiment, in an implementation, with reference to Figure 8 , the convolution layer in the convolution-pooling cascade unit is configured with initial weights. The determination process of the initial weights includes:

[0113] S31, taking the average value of the gradient matrix of the enhanced sample as a reference weight component.

[0114] The enhanced sample is obtained by linearly superimposing and fusing the illumination compensation result and the texture enhancement result of the image sample.

[0115] Specifically, the image sample is a high-quality basic data set constructed during training of the object recognition model. The image sample acquisition process is as follows: for a target cultural relic category such as porcelain, bronze ware, calligraphy and painting, a high-resolution digital camera is used to capture a clear image of each cultural relic from multiple fixed angles (such as front view, top view, and side view) in a standardized photography studio environment, so as to ensure that the main feature surface of the cultural relic is covered. The acquisition content focuses on key visual features that can reflect the differences between categories, such as the overall shape of the object, typical decorative patterns, and material surface characteristics.

[0116] In one implementation, to cope with possible problems in the actual environment, such as partial occlusion and slight angle change, limited natural background or slight non-uniform illumination samples are appropriately introduced during image acquisition to enhance the data robustness.

[0117] Further, the collected data is labeled. According to the knowledge of cultural relic typology, professional cultural relic and museum personnel accurately label the cultural relic category label of each image, such as “Mei vase with winding branch peony pattern in blue and white of Yuan Dynasty” and “Bronze Ding with Taote pattern of Warring States”.

[0118] After obtaining the image sample, an enhancement algorithm combining multi-scale Retinex and adaptive directional Gabor filtering is used to linearly superimpose and fuse the illumination compensation result and the texture enhancement result of the image sample, to obtain the enhanced sample.

[0119] In actual scenarios, in the face of high-dimensional cultural relic image features, random weight initialization of conventional convolutional neural networks can easily lead to slow training convergence or falling into local optimal solution, and it is difficult to adapt to the complex spatial distribution of cultural relic textures. The global mean initialization method cannot handle the problem of uneven distribution of local texture density or sparsity, and is prone to failure in complex texture areas, reducing the sensitivity of the model to subtle features.

[0120] To this end, this embodiment calculates the initial weights of the convolutional layers in the convolutional neural network based on the global gradient statistical characteristics and local texture clustering information of the preprocessed image. First, the arithmetic mean of the gradient matrices of all enhanced samples is calculated as the baseline weight component, representing the global edge distribution. Then, a texture weighting factor is calculated based on the difference between the sample texture enhancement result and the center of its respective texture cluster, and the texture correction increment is obtained by weighted summation of the sample texture enhancement results accordingly. Finally, the baseline weight component is added to the texture correction increment to generate the initial weight matrix of the convolutional layer, which integrates global edge features and local texture adaptability, allowing the model to focus on key texture features from the initial training stage. The specific steps are as follows:

[0121] Calculate the arithmetic mean of the gradient matrices of all enhanced samples to establish the baseline weight components characterizing the global edge feature distribution. The specific calculation formula is as follows:

[0122] ;

[0123] in, Provides a global edge feature distribution for the baseline weight components;

[0124] For the first The intermediate image after preprocessing the initial image specifically refers to the first image during model training. Augmented samples of image samples;

[0125] for The gradient matrix represents the edge features;

[0126] This represents the total number of training samples.

[0127] S32. Calculate the texture weighting factor of the image sample by using the difference between the texture enhancement result of the image sample and the texture cluster center to which the texture enhancement result of the image sample belongs.

[0128] Specifically, based on the difference between the texture enhancement result of an image sample after adaptive directional Gabor texture enhancement and the center of its respective texture cluster, a texture weighting factor for the image sample is generated by controlling the sensitivity through texture influence hyperparameters to address the uneven distribution problem. The specific calculation formula is as follows:

[0129] ;

[0130] in, The texture weighting factor function outputs... Range values;

[0131] for The center of the texture cluster;

[0132] For texture influence hyperparameters, such as, = 1.2, control the sensitivity to texture density difference;

[0133] For Gabor filter output image, that is, texture enhancement result.

[0134] S33, using the texture weighting factor of the image sample to correct the texture enhancement result of the image sample, and obtaining a texture correction increment.

[0135] Specifically, the texture enhancement result of the image sample is weighted and summed according to the texture weighting factor, the response strength of the adjustment rate is adjusted, the texture correction increment compensating for the local texture difference is generated, and the specific calculation formula is:

[0136] ;

[0137] Wherein, is the texture correction increment, compensating for the local texture difference;

[0138] is the adjustment rate, such as, = 0.05, control the response strength to clustering information;

[0139] is the total number of training samples;

[0140] For Gabor filter output image, that is, texture enhancement result.

[0141] S34, adding the reference weight component and the texture correction increment to obtain the initial weight of the convolution layer in the convolution pooling cascaded unit.

[0142] Specifically, the reference weight component and the texture correction increment are added to generate a convolution layer initialization weight matrix that fuses global edge features and local texture adaptability, which is represented as:

[0143] ;

[0144] Wherein, is the convolution layer initialization weight matrix, which is the starting point of network training;

[0145] is the reference weight component, capturing global edge features;

[0146] is the texture correction increment, enhancing local texture adaptability.

[0147] In this embodiment, based on the global gradient statistical characteristics and local texture clustering information of the pre-processed image, the initial weights of the convolution layer in the convolutional neural network are calculated, so that the model focuses on key texture features at the initial stage of model training.

[0148] In addition, by using the global gradient statistics and local texture clustering to correct the weight initialization, the sensitivity of the model to subtle features is enhanced, and the training efficiency and recognition accuracy are improved.

[0149] It should be noted that the remaining trainable parameters of the object recognition model can be initialized randomly according to the conventional method.

[0150] On the basis of any of the above embodiments, after the initial values of the trainable parameters of the object recognition model are known, the parameters need to be updated in the subsequent training process, that is, the parameters in the object recognition model need to be updated in the training process. In one implementation, the updating process of the parameters in the object recognition model includes:

[0151] 1) Calculate the historical cumulative value of the variance of the attention weighted feature map of the enhanced sample, and based on the historical variance decay factor and the historical cumulative value, obtain the variance cumulative term for adaptive scaling of the learning rate.

[0152] Specifically, to avoid the risk of uneven distribution of cultural relic image training data and the model being easily trapped in local optimum, the conventional SGD (Stochastic Gradient Descent, Stochastic Gradient Descent) optimizer with fixed learning rate will cause oscillation or stagnation in the convergence process, and the traditional Adagrad (Adaptive Gradient, also known as adaptive gradient algorithm) method only accumulates the square of the gradient and ignores the volatility of the feature space, making it difficult to adaptively balance the parameter update amplitude in different texture complexity regions.

[0153] Therefore, the embodiments of the present application use an optimizer variant based on feature variance to update the convolution layer weights, iteratively accumulate the historical value of the variance of the attention weighted feature map, generate a variance cumulative term for adaptive scaling of the learning rate, and use the variance cumulative term to construct an adaptive scaling factor when updating the parameters, scale the gradient of the total discriminative loss function, and effectively reduce the learning rate when the feature space volatility increases, and vice versa. Then, according to the feature stability, the parameter update step is dynamically adjusted, which helps to balance the convergence speed and stability, and reduces the risk of falling into local optimum. The specific steps are as follows:

[0154] Iteratively update the historical cumulative value of the variance of the attention weighted feature map, balance the new and old statistics through the historical variance decay factor, and generate a variance cumulative term for adaptive scaling of the learning rate. The specific calculation formula is:

[0155] ;

[0156] wherein, is a variance cumulative term, iteratively updated for scaling learning rate, reducing update step when feature variance increases;

[0157] is a history variance decay factor, controlling the proportion of old variance value to be reserved, the larger the value, the more dependent on long-term statistics;

[0158] is a variance calculation function, calculating the variance of attention weighted feature map of all elements (such as attention weighted feature map of enhanced sample), obtaining a history cumulative value, representing the fluctuation intensity of feature space;

[0159] represents an iterative assignment operation.

[0160] 2) Construct an adaptive scaling factor using the variance cumulative term and the smoothing constant.

[0161] Specifically, the calculation formula of the adaptive scaling factor is:

[0162] ;

[0163] wherein, is a variance cumulative term, is a smoothing constant, such as, .

[0164] 3) Perform parameter update operation on the convolutional layer weight gradient after scaling using the adaptive scaling factor, to obtain the updated parameters of the object recognition model.

[0165] Specifically, perform parameter update on the convolutional layer weight gradient after scaling using the adaptive scaling factor, to balance feature stability and convergence speed, and the specific calculation formula is:

[0166] ;

[0167] wherein, is the updated weight, i.e. the updated parameters of the object recognition model;

[0168] is the current weight, i.e. the parameters of the object recognition model before updating;

[0169] is an initial learning rate, controlling the basic update step, such as, ;

[0170] is a smoothing constant, preventing the denominator from being zero, such as, ;

[0171] is a total discriminative loss function the gradient of the current weight ;

[0172] is an adaptive scaling factor, the greater the effective learning rate is smaller.

[0173] The model parameter updating process is provided with a stop iteration condition, and the convergence index needs to be monitored synchronously in the model training process to determine whether to execute the stop decision. First, the validation set accuracy is tracked, and when the accuracy fluctuation amplitude is less than 0.2% in 10 consecutive training cycles, the primary stop condition is triggered. Secondly, the decay rate of the total discriminative loss function is calculated, and if the loss value decreases by less than 0.1% in 20 cycles, it is determined to enter the plateau.

[0174] When the above conditions are met at the same time, or the total training cycle reaches the preset upper limit (such as 3000 rounds), the iteration is terminated.

[0175] It should be noted that each trainable parameter in the model is updated using the above method.

[0176] In this embodiment, the initialization method of the initial weight of the convolutional layer is improved, and the adaptive optimizer using the parameter adaptive learning rate optimization is used for parameter updating operation, which can effectively avoid the local optimal trap, speed up the training, and improve the robustness in complex texture scenes.

[0177] In an example, after the model is trained, the model can be used to realize cultural relic recognition through an end-to-end inference process. Specifically, after inputting the image of the cultural relic to be recognized (such as the initial image described above), first, multi-scale Retinex light compensation and adaptive directional Gabor texture enhancement preprocessing are performed to generate a standardized enhanced image (such as the intermediate image described above). Then, it is input to the initial object recognition model for forward propagation, and after convolution feature extraction, the key texture area is dynamically weighted by the attention module, and finally a 256-dimensional discriminative feature vector is generated; the vector is distance measured with each class feature center formed in the training stage, and the cultural relic type corresponding to the class center with the smallest Euclidean distance is selected as the recognition result.

[0178] In an embodiment, a classification accuracy comparison analysis is performed to evaluate the performance advantage of the overall architecture of the embodiment in the cultural relic fine-grained classification task. The recognition accuracy of the five types of cultural relics of the conventional convolutional neural network, the deep residual network (ResNet50, Residual Network 50, 50 layers of residual network) and the model in the embodiment of the present application is compared, such as Figure 9As shown in FIG. 6, the horizontal axis is the artifact category (bronze / porcelain / painting / jade / gold and silver / average), and the vertical axis is the classification accuracy (unit: percentage, presented in decimal form). The experimental results show that the painting category has the largest gap, and the application has a significant advantage. The ink texture of the painting needs to be adaptively enhanced in the direction of Gabor, and the jade category has a significant advantage. The light compensation of the semi-transparent material depends on the multi-scale Retinex. In addition, the conventional method has large fluctuations in different categories (high for bronze and low for painting), and the column height of each category in the application is uniform, proving that the model in the application has strong generalization ability.

[0179] On the basis of any of the above embodiments, the preset attention weight function and the output feature map are used to obtain an attention weight map, including:

[0180] The preset attention weight function is calculated using global average information and local texture energy information, and the output feature map is processed using the preset attention weight function to obtain the attention weight map.

[0181] Specifically, the preset attention weight function compresses the global features extracted by the average pooling through the multi-layer perception, superimposes the texture energy term extracted by the high-pass filter, and generates the attention weight of the dynamic focusing high-frequency detail area through the Sigmoid activation function. The specific calculation formula is:

[0182] ;

[0183] wherein, represents the feature map of the preset attention weight function; may be the output feature map output by the convolution layer; in one embodiment, the output feature map can refer to Figure 10 as shown.

[0184] represents the Sigmoid activation function, and the output [0, 1] range weight ensures the weight interpretability and nonlinearity;

[0185] represents the multi-layer perception, which is used to compress the feature dimension and reduce the calculation amount;

[0186] represents the average pooling operation, which extracts the global average feature (also referred to as global average information) of , and reduces the spatial dimension;

[0187] represents the texture energy term (also referred to as local texture energy information), which represents the local texture saliency, and enhances the high-frequency details through the high-pass filter. The calculation method is ;

[0188] wherein, is The number of channels;

[0189] For the first Feature maps of each channel;

[0190] It is a high-pass filter core used to extract high-frequency details;

[0191] This indicates an absolute value operation, ensuring that the energy value is positive.

[0192] Once the preset attention weight function is known, it can be used to... The value is the output feature map of the convolutional layer, and the calculation method is expressed as follows: ,Will Substituting the values ​​into the preset attention weight function yields the attention weight map. The attention weight map is... .in, This represents the standard convolution operation. This is the output feature map obtained after standard convolution operation. Initialize the weights for the convolutional layer. For the first An intermediate image after preprocessing an initial image.

[0193] Subsequently, the output feature map and the attention weight map can be multiplied element-wise to obtain an attention-weighted feature map. Specifically, when calculating the attention-weighted feature map, the output feature map obtained from the standard convolution operation is multiplied element-wise with the attention weight map to generate an attention-weighted feature map that dynamically suppresses background noise and enhances the main area of ​​the cultural relic. This map is then output to subsequent network layers. The specific calculation formula is as follows:

[0194] ;

[0195] in, Representing the attention-weighted feature map, such as the first... The attention-weighted feature map of the initial image is used to suppress irrelevant backgrounds and enhance the main body area of ​​the cultural relics through dynamic weights, thus solving the problem of inter-class similarity.

[0196] This represents the standard convolution operation;

[0197] Initialize the weights for the convolutional layer;

[0198] This indicates a preset attention weight function used to generate dynamic weights;

[0199] This represents element-wise multiplication, applying attention weights to the feature map;

[0200] is an image index;

[0201] is a pre-processed intermediate image for the first initial image.

[0202] In this embodiment, the adaptive texture attention mechanism is used to accurately focus on the high-frequency region and strengthen the texture difference, thereby solving the problem of blurred inter-class boundary.

[0203] On the basis of any of the above embodiments, the total discriminative loss function of the object recognition model is:

[0204] ;

[0205] wherein, the total discriminative loss function of the object recognition model, the cross-entropy loss, the center loss, the adaptive margin loss, the center loss weight coefficient, the margin loss weight coefficient.

[0206] In specific implementation, for the high similarity between cultural relic categories, the conventional cross-entropy loss only focuses on the matching degree of the predicted probability and the true label, ignores the distribution characteristics of the feature space, and leads to dispersed intra-class features and blurred inter-class boundary, which is easy to produce misclassification between highly similar categories. The traditional method lacks explicit constraints on the compactness and discriminativeness of the feature space, and cannot effectively cope with the inter-class similarity challenge.

[0207] The embodiment of the present application adopts a total discriminative loss function that fuses triple constraints. First, the basic cross-entropy loss is calculated to ensure that the model prediction matches the true label. Then, the center loss is calculated to measure the distance between the sample feature and the class center to which it belongs, and to force the aggregation of features of the same class in the embedding space. Finally, the adaptive margin loss is calculated to dynamically adjust the inter-class interval threshold according to the sample gradient information, and to force the expansion of the feature center distance of similar categories. The center loss and the adaptive margin loss are weighted and added to the cross-entropy loss to form the total discriminative loss function, which jointly constrains the compactness and discriminativeness of the feature space. The specific steps are as follows:

[0208] 1) Calculate the cross-entropy loss

[0209] Based on the difference between the model prediction probability distribution and the true label, the basic cross-entropy loss value that ensures correct classification is calculated, which is represented as:

[0210] ;

[0211] wherein, represents the cross-entropy loss value, which is a basic component of the total loss, ensuring that the model correctly matches the label;

[0212] is the total number of cultural relic categories, reflecting the complexity of the classification task;

[0213] is the true label of the th sample in the th category, which is 1 when the sample belongs to the category , otherwise 0;

[0214] is the probability of the th sample belonging to the th category predicted by the model, representing the classification confidence, calculated by the Softmax function of the classification layer.

[0215] 2) Calculate the center loss

[0216] measure the distance between the attention-weighted feature map and its class center, filter the same-class samples through the indicator function, and force the same-class features to aggregate in the embedding space, represented as:

[0217] ;

[0218] wherein represents the center loss value, which constrains the features of the same-class samples to be close to the class center;

[0219] is the attention-weighted feature map of the th sample, representing the feature representation after texture enhancement; in an embodiment, the texture-enhanced feature heat map can refer to Figure 11 .

[0220] is the feature center of the th category, initialized as the mean of the in-class sample features and dynamically updated during training;

[0221] is the indicator function, which outputs 1 when the sample belongs to the category , otherwise outputs 0;

[0222] is the true category label of the th sample;

[0223] is the category index.

[0224] 3) Calculate the adaptive margin loss

[0225] An adaptive adjustment factor is generated based on the Frobenius norm of image gradient, and the threshold of the distance between feature centers of different classes is dynamically adjusted, and the hinge function is used to force the expansion of the feature distance of similar classes, which is represented as:

[0226] ;

[0227] wherein, represents the adaptive margin loss value, which forces the expansion of the feature distance of similar classes;

[0228] is a reference margin constant, such as, =1.0, which sets the minimum inter-class distance;

[0229] is the feature center of the class ;

[0230] represents the distance between the feature centers of different classes;

[0231] is an adaptive adjustment factor, and the larger the value, the greater the inter-class distance required by the sample, and the calculation method is represented as ;

[0232] is the Frobenius norm, which calculates the overall intensity of the image gradient;

[0233] is the hinge function, which sets the loss to zero when the inter-class distance is greater than the adjusted threshold.

[0234] 4) Calculate the total discriminative loss

[0235] The cross-entropy loss, weighted center loss and weighted adaptive margin loss are added to generate a total discriminative loss function that coordinates the classification accuracy and feature distribution optimization, which is represented as:

[0236] ;

[0237] wherein, is the total discriminative loss function, which is output to the optimizer for parameter update;

[0238] is the center loss weight coefficient, such as, =0.01, which controls the intra-class aggregation strength;

[0239] is the margin loss weight coefficient, such as, =0.005, which controls the inter-class separation strength.

[0240] In this embodiment, by integrating the total discriminative loss function of cross-entropy, center loss and adaptive margin loss, the discriminability and aggregability of the feature space are strengthened, and the structure of the feature space is optimized.

[0241] On the basis of any of the above embodiments, in an implementation, after obtaining the object category recognition result of the target object, the object information corresponding to the object category recognition result can be matched out by using the knowledge graph corresponding to the target object.

[0242] Specifically, the object category recognition result is input into the large model knowledge engine in a structured data format. The object category recognition result can include three core attributes of cultural relics category, age, material, and feature vector fingerprint.

[0243] The large model first retrieves the cultural relic knowledge graph constructed internally, and matches out the authoritative object information such as the historical background, process characteristics and cultural value of the cultural relic.

[0244] On the basis of any of the above embodiments, in an implementation, after obtaining the object category recognition result of the target object, similar objects of the target object can be located.

[0245] Specifically, similar museum cultural relics can be located in combination with the texture fingerprint in the feature vector of the intermediate image, and the similar museum cultural relic cases are compared and explained. Subsequently, multi-modal popular science content can be generated: the text end adopts a "core fact + story expression" structure, and focuses on explaining the symbolic meaning and manufacturing process of the decorative symbols.

[0246] On the basis of any of the above embodiments, in an implementation, after obtaining the object category recognition result of the target object, a three-dimensional model of the object category recognition result can be generated.

[0247] Specifically, the visual end calls a three-dimensional reconstruction module to generate a three-dimensional model of the cultural relic, and then dynamically demonstrates the cross section of the cultural relic, and focuses on interpretation in association with the key texture area enhanced in the preprocessing stage. After the output content is verified by the cultural relic expert knowledge base, the depth question and answer and virtual repair experience functions are provided through the interactive interface.

[0248] In this embodiment, the cultural relic recognition result is fused with the knowledge graph, supports human-computer interaction applications such as three-dimensional display, story-telling explanation and virtual repair, expands the technical value, and meets the deep needs of public popular science and cultural and educational education, so as to improve the cultural communication ability of cultural relics.

[0249] To sum up, in the embodiment of the application, aiming at the actual problems of complex collection environment, delicate texture features and small differences between categories of cultural relic images, a comprehensive technical solution of image enhancement, identification modeling and semantic linkage is proposed, which aims to improve image quality through multi-scale Retinex light compensation and adaptive directional Gabor texture enhancement, construct a deep neural network based on texture statistical characteristics to enhance the sensitivity of the model to detailed features, and optimize the feature space structure by introducing a total discriminative loss function, and at the same time, the identification result is structured and output and connected to the cultural relic knowledge graph, realizing the closed-loop processing of cultural relic images from high-precision classification to cultural semantic popularization, and comprehensively improving the identification accuracy, robustness and cultural value conveying ability of the system.

[0250] On the basis of the above-mentioned embodiment of the object recognition method, another embodiment of the application provides an object recognition device, which refers to Figure 12 , comprising:

[0251] The image acquisition module 11 is configured to acquire an initial image of a target object.

[0252] The fusion module 12 is configured to linearly superimpose and fuse the light compensation result and the texture enhancement result of the initial image to obtain an intermediate image.

[0253] The processing module 13 is configured to perform convolution operation on the intermediate image to obtain an output feature map, and use a preset attention weight function and the output feature map to obtain an attention weight map.

[0254] The recognition module 14 is configured to perform element-by-element multiplication on the output feature map and the attention weight map to obtain an attention weighted feature map, and perform classification operation on the attention weighted feature map to obtain an object category recognition result of the target object.

[0255] It should be noted that the working processes of each module and sub-module in the embodiment are described above with reference to the corresponding description in the above-mentioned embodiment, and will not be described here again.

[0256] The embodiment of the application further provides an electronic device, which comprises at least one processor and a memory connected to the processor, wherein:

[0257] The memory is configured to store a computer program.

[0258] The processor is configured to execute the computer program, so that the electronic device can implement the above-mentioned object recognition method.

[0259] The embodiment of the application further provides a computer program product comprising computer readable instructions, which, when executed on an electronic device, enable the electronic device to implement any one of the object recognition methods provided by the embodiments of the application.

[0260] The embodiment of the present application further provides a computer readable storage medium, the storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can realize any object recognition method provided by the embodiment of the present application.

Claims

1. A method of object recognition, characterized by, The method comprises the following steps: obtaining an initial image of a target object; performing linear superposition fusion on the illumination compensation result and the texture enhancement result of the initial image to obtain an intermediate image; performing convolution operation on the intermediate image to obtain an output feature map, and using a preset attention weight function and the output feature map to obtain an attention weight map; performing element-by-element multiplication on the output feature map and the attention weight map to obtain an attention weighted feature map, and performing classification operation on the attention weighted feature map to obtain an object category recognition result of the target object; wherein the feature extraction backbone network in the object recognition model comprises a plurality of convolutional pooling cascade units, the convolutional layer in the convolutional pooling cascade unit is configured with an initial weight, and the determination process of the initial weight comprises: taking the average value of the gradient matrix of the enhanced sample as a reference weight component; wherein the enhanced sample is obtained by performing linear superposition fusion on the illumination compensation result and the texture enhancement result of the image sample; using the difference between the texture enhancement result of the image sample and the texture cluster center to which the texture enhancement result belongs to calculate a texture weighting factor of the image sample; performing correction operation on the texture enhancement result of the image sample using the texture weighting factor to obtain a texture correction increment; adding the reference weight component and the texture correction increment to obtain the initial weight of the convolutional layer in the convolutional pooling cascade unit.

2. The object recognition method of claim 1, wherein, performing linear superposition fusion on the illumination compensation result and the texture enhancement result of the initial image to obtain an intermediate image, comprising: performing illumination component decomposition on the initial image, and obtaining an illumination compensation result by dynamically fusing the illumination correction results under each scale; based on the weight attenuation coefficient, and the difference between the gradient matrix of the initial image and the gradient clustering center of each scale, an adaptive weight is calculated; performing filtering operation on the initial image using the adaptive direction angle to obtain a texture enhancement result; using the adaptive weight, performing linear superposition fusion on the illumination compensation result and the texture enhancement result to obtain an intermediate image.

3. The object recognition method of claim 1, wherein, The object recognition model comprises: a feature extraction backbone network, an adaptive attention module, and a classification output layer; The feature extraction backbone network comprises a plurality of convolutional pooling cascade units; the feature extraction backbone network is used to perform convolution operation on the intermediate image using the plurality of convolutional pooling cascade units to obtain an output feature map; The adaptive attention module is embedded after the convolutional pooling cascade unit of a specified group, and is used to obtain an attention weight map using a preset attention weight function and the output feature map, and perform element-by-element multiplication on the output feature map and the attention weight map to obtain an attention weighted feature map; The classification output layer comprises a multi-level fully connected network, and is used to perform classification operation on the attention weighted feature map through dimension reduction and object classification operation to obtain an object category recognition result of the target object.

4. The object recognition method of claim 3, wherein, The parameters in the object recognition model need to be updated in the training process, and the parameter updating process comprises: The historical cumulative value of the attention-weighted feature map variance of the enhanced sample is calculated, and a variance cumulative term for adaptive scaling of the learning rate is obtained based on a historical variance decay factor and the historical cumulative value; An adaptive scaling factor is constructed using the variance cumulative term and a smoothing constant; After scaling the convolution layer weight gradient using the adaptive scaling factor, a parameter update operation is performed to obtain updated parameters of the object recognition model.

5. The object recognition method of claim 1, wherein, An attention weight map is obtained using a preset attention weight function and the output feature map, including: The preset attention weight function is calculated using global average information and local texture energy information; The output feature map is processed using the preset attention weight function to obtain an attention weight map.

6. The object recognition method of claim 1, wherein, After obtaining the object category recognition result of the target object, further including: Using the knowledge graph corresponding to the target object, matching the object information corresponding to the object category recognition result; And / or, positioning similar objects of the target object; And / or, generating a three-dimensional model of the object category recognition result.

7. An object recognition apparatus characterized by comprising: Including: An image acquisition module is configured to acquire an initial image of a target object; A fusion module is configured to perform linear superposition fusion on the illumination compensation result and the texture enhancement result of the initial image to obtain an intermediate image; A processing module is configured to perform convolution operation on the intermediate image to obtain an output feature map, and obtain an attention weight map using a preset attention weight function and the output feature map; An identification module is configured to multiply the output feature map and the attention weight map element by element to obtain an attention-weighted feature map, and perform classification operation on the attention-weighted feature map to obtain an object category recognition result of the target object. In the feature extraction backbone network of the object recognition model, a plurality of convolution-pooling cascade units are included, and the convolution layers in the convolution-pooling cascade units are configured with initial weights. The determination process of the initial weights includes: The average value of the gradient matrix of the enhanced sample is taken as the reference weight component; wherein the enhanced sample is obtained by linear superposition fusion on the illumination compensation result and the texture enhancement result of the image sample; The texture weighting factor of the image sample is calculated using the difference between the texture enhancement result of the image sample and the texture cluster center to which the texture enhancement result belongs; The texture enhancement result of the image sample is modified using the texture weighting factor of the image sample to obtain a texture correction increment; The initial weights of the convolution layers in the convolution-pooling cascade unit are obtained by adding the reference weight component and the texture correction increment.

8. An electronic device, comprising: The electronic device includes at least one processor and a memory connected to the processor, wherein: The memory is used to store a computer program; The processor is used to execute the computer program, so that the electronic device can implement the object recognition method according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that The storage medium carries one or more computer programs, which can make the electronic device implement the object recognition method according to any one of claims 1 to 6 when executed by the electronic device.

Images