A cultural and museum image processing method and system based on image recognition technology

Abstract

The present invention discloses a method and system for processing cultural and historical images based on image recognition technology. The method comprises performing preliminary preprocessing operations on the original cultural and historical images to obtain a first image; using the VGG16‑Unet cultural and historical image preprocessing model based on deep learning to perform noise recognition and removal operations on the first image to obtain a second image; using the cultural and historical image segmentation model based on the RT‑DETR model to perform segmentation processing on the second image to obtain a third image containing target information; using the cultural and historical image target information extraction model based on the improved ResNet‑101 to identify the target information of the third image, and determine the target information contained in the original cultural and historical images. The scheme of the present invention not only improves the clarity of cultural and historical images, but also greatly improves the accuracy of target information segmentation and recognition, and is suitable for a variety of application scenarios such as cultural relics protection, digital archiving, and academic research.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a cultural and museum image processing method and system based on image recognition technology. Background Art

[0002] Through the digital archiving and analysis of cultural relics, we can effectively protect cultural relics and improve the efficiency of academic research. As an important carrier for recording and displaying historical relics, cultural relics images carry a large amount of information with academic and cultural value, such as ancient characters, patterns, and structural details.

[0003] In the process of collecting original cultural and museum images, noise interference often exists due to the influence of various factors such as equipment performance, light conditions, and the surface condition of cultural relics. For example, the text in the image has insufficient contrast with the background, or the texture information is blurred due to the wear of the surface of the cultural relics. These problems seriously affect the accuracy of subsequent image processing and analysis. Therefore, how to effectively remove image noise, enhance target information, and realize automatic recognition and extraction has become an important technical problem that needs to be solved in the cultural and museum industry. This is not only related to the scientific research of cultural relics, but also has important practical significance for the digital preservation and display of cultural relics.

[0004] At present, although there are some algorithms and tools for image denoising and target detection, most of them have not been optimized for the specific needs of cultural and museum images. In particular, when dealing with problems such as complex backgrounds, low contrast, and subtle textures, the results of existing methods are still unsatisfactory. Summary of the invention

[0005] In order to solve the technical problem that the existing image processing methods cannot meet the specific requirements of cultural and museum images, the embodiments of the present invention provide a method and system for processing cultural and museum images based on image recognition technology.

[0006] The technical solution of the embodiment of the present invention is achieved as follows:

[0007] An embodiment of the present invention provides a cultural and historical image processing method based on image recognition technology, the method comprising: acquiring an original cultural and historical image; performing preliminary preprocessing operations on the original cultural and historical image to obtain a first image; using a VGG16-Unet cultural and historical image preprocessing model based on deep learning to perform noise identification and removal operations on the first image to obtain a second image; using a cultural and historical image segmentation model based on an RT-DETR model to perform segmentation processing on the second image to obtain a third image containing target information; using a cultural and historical image target information extraction model based on an improved ResNet-101 to identify target information on the third image, and determine the target information contained in the original cultural and historical image.

[0008] In one embodiment, the original cultural and museum image is subjected to preliminary preprocessing operations, including: using image enhancement technology to process the original cultural and museum image to increase the range of pixel brightness values ​​in the original cultural and museum image; using a Gaussian filter to process the original cultural and museum image after the brightness value range is increased to complete preliminary denoising, smoothing and edge detection of the original cultural and museum image; setting the pixel grayscale value of the original cultural and museum image to 0 or 255 to convert the original cultural and museum image into a black and white effect image.

[0009] In one embodiment, the VGG16-Unet cultural image preprocessing model based on deep learning includes an encoder and a decoder, the encoder includes 5 convolutional layers and 4 maximum pooling layers, and the decoder includes an upsampling module and a convolutional layer; the training process of the VGG16-Unet cultural image preprocessing model based on deep learning includes: obtaining a training data set, the training data set includes a clean image with noise removed and a noisy image without noise removed; using image subtraction technology to perform image subtraction processing on the clean image and the noisy image to obtain a cultural image noise map; flipping, rotating, randomly cropping and / or translation transformation processing on the cultural image noise map to complete the expansion of the cultural image noise map; synthesizing the expanded cultural image noise map with the clean image to obtain the expanded training data set; using the training data set to train the VGG16-Unet cultural image preprocessing model based on deep learning;

[0010] Among them, the training loss function of the VGG16-Unet cultural image preprocessing model based on deep learning is:

[0011]

[0012] in, express Pixels in the clean image, express Pixels in a noisy image.

[0013] In one embodiment, the VGG16-Unet cultural image preprocessing model based on deep learning is trained using the training data set, including: in the process of using the VGG16-Unet cultural image preprocessing model based on deep learning to train the training data set, an evaluation index value is calculated, and the evaluation index value includes a peak signal-to-noise ratio and a structural similarity; when the peak signal-to-noise ratio is higher than a preset first threshold, it is determined that the processing quality of the VGG16-Unet cultural image preprocessing model based on deep learning after training is qualified; and / or, when the structural similarity is greater than a preset second threshold, it is determined that the denoising effect of the VGG16-Unet cultural image preprocessing model based on deep learning after training is qualified;

[0014] The peak signal-to-noise ratio is calculated as:

[0015]

[0016] in, is the peak signal-to-noise ratio, represents the maximum value of pixels in the image, and MSE represents the loss function of the VGG16-Unet cultural image preprocessing model based on deep learning;

[0017] The calculation formula for the structural similarity is:

[0018]

[0019] in, , and Respectively represent brightness contrast function, contrast contrast function and structure contrast function, where,

[0020]

[0021]

[0022]

[0023] in, , Represent the average values ​​of the clean image and the noisy image respectively, , denote the variance of the clean image and the noisy image respectively, Represent the standard deviation of the clean image and the noisy image respectively, , and represents a constant, , is the covariance of the image pair consisting of the clean image and the noisy image, and its formula is:

[0024]

[0025] Where N is the total number of pixels in the image, represents the pixel value of the clean image at the i-th pixel, represents the pixel value of the noise image at the i-th pixel, and i represents the i-th pixel in the image.

[0026] In one embodiment, the cultural and museum image segmentation model based on the RT-DETR model includes a backbone network, a neck network and a decoder; the backbone network is used for feature extraction; the neck network is used to further aggregate multi-layer feature maps and enhance the multi-scale representation capability of the image through upsampling and downsampling operations; the decoder is used to use a fully connected layer and non-maximum suppression technology to process key information in the feature map, identify the target information and complete the segmentation; the loss function of the cultural and museum image segmentation model based on the RT-DETR model during the training process is:

[0027]

[0028] in, is the cross entropy loss, For adversarial loss, and are the hyperparameters that control the weights of the cross entropy loss and adversarial loss respectively.

[0029] In one embodiment, the evaluation indicators of the cultural and museum image segmentation model based on the RT-DETR model during the training process include accuracy, recall, F1, AP and mAP; wherein,

[0030]

[0031]

[0032]

[0033]

[0034]

[0035] Among them, TP is the correct detection of the real detection box, FP is the wrong detection of the non-existent object or the wrong detection of the position of the existing object, and FN is the real detection box that has not been detected; is the number of all predicted boxes, is the number of all true detection boxes, P is the precision value, R is the recall value, is the recall rate of the n+1th sample, is the recall rate of the nth sample, is the average accuracy, which refers to the average accuracy under different recall rates, and mAP is the average accuracy of all categories The average value of N is the number of samples. It is the value of the maximum accuracy corresponding to the recall rate exceeding the n+1th sample.

[0036] In one embodiment, the cultural and museum image target information extraction model based on the improved ResNet-101 includes multiple residual blocks and dual attention network modules: each of the residual blocks includes a convolution layer and an activation function layer, and the residual block is used to pass the output of the convolution layer to the next layer, and add the input data to the data processed by the convolution and activation function layers; the dual attention network module includes a spatial attention module and a channel attention module; the spatial attention module is used to selectively aggregate the features of each position by the weighted sum of the features at all positions; the channel attention module is used to use the self-attention mechanism to capture the channel dependency between any two channel maps, and update each channel map by the weighted sum of all channel maps; the loss function of the training process of the cultural and museum image target information extraction model based on the improved ResNet-101 includes:

[0037]

[0038] in, is the probability of the i-th category of the true label, is the probability of the i-th category of the model prediction output.

[0039] An embodiment of the present invention also provides a cultural and museum image processing system based on image recognition technology, comprising: a processor and a memory for storing a computer program that can be run on the processor; wherein the processor executes the steps of the above-mentioned method when running the computer program.

[0040] An embodiment of the present invention further provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0041] An embodiment of the present invention also provides a computer program product, which includes computer instructions. The computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps of the above-mentioned method.

[0042] The cultural and historical image processing method and system based on image recognition technology provided by the embodiment of the present invention effectively solves the problems of noise interference and information extraction difficulties in cultural and historical images through three major steps: image denoising preprocessing, automatic image segmentation, and target information recognition. The method of this embodiment not only improves the clarity of cultural and historical images, but also greatly improves the accuracy of target information segmentation and recognition, and is suitable for a variety of application scenarios such as cultural relics protection, digital archiving, and academic research. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the flow of a cultural and museum image processing method based on image recognition technology according to an embodiment of the present invention;

[0044] Figure 2 It is a schematic diagram of the overall process of the cultural and museum image denoising and information extraction method based on image recognition technology in an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram showing partial noise of a certain cultural and museum image according to an embodiment of the present invention;

[0046] FIG4 (a) is a schematic diagram of an original image of a cultural and museum image before preprocessing and a grayscale comparison diagram according to an embodiment of the present invention;

[0047] FIG4( b ) is a schematic diagram and a grayscale comparison diagram of a certain cultural and museum image after contrast enhancement processing is performed on the original image according to an embodiment of the present invention;

[0048] FIG4 (c) is a schematic diagram and a grayscale comparison diagram of a cultural and museum image after the original image is binarized according to an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of image enhancement and data set expansion operations for a certain cultural and museum image according to an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of a cultural and museum image preprocessing model based on VGG16-Unet according to an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of the network structure of a cultural and museum image target information segmentation model based on RT-DETR according to an embodiment of the present invention;

[0052] Figure 8 (a) is a schematic diagram of the loss change during the YOLOv5 model training process;

[0053] Figure 8 (b) is a schematic diagram of the loss changes during the training process of the YOLOv8 model;

[0054] Figure 8 (c) is a schematic diagram of the loss changes during the training process of the RT-DETR model;

[0055] Fig. 9This is a schematic diagram of the IOU effect of an embodiment of the present invention;

[0056] Fig.10 Schematic diagram of the PR curve of the RT-DETR model of an embodiment of the present invention;

[0057] Fig.11 This is a schematic diagram of the segmentation result of the target information in the image by the RT-DETR model according to an embodiment of the present invention;

[0058] Fig.12 : is a schematic diagram of the residual unit structure of the ResNet-101 model according to an embodiment of the present invention; wherein, Fig.12 (a) is a schematic diagram of the structure of the full-line residual unit; Fig.12 (b) is a schematic diagram of the structure of the dotted residual unit;

[0059] Fig.13 Schematic diagram of the spatial attention module of the ResNet-101 model according to an embodiment of the present invention;

[0060] Fig.14 Schematic diagram of the channel attention module of the ResNet-101 model according to an embodiment of the present invention;

[0061] Fig.15 This is a schematic diagram of the application effect of an embodiment of the present invention;

[0062] Fig.16 This is a schematic diagram of the model testing process flow of an embodiment of the present invention;

[0063] Fig.17 1 is a diagram showing the internal structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0065] The embodiment of the present invention provides a method for processing cultural and museum images based on image recognition technology, such as Figure 1 As shown, the method includes:

[0066] Step 101: Acquire an original cultural and historical image; perform a preliminary preprocessing operation on the original cultural and historical image to obtain a first image;

[0067] Step 102: Use the VGG16-Unet cultural image preprocessing model based on deep learning to perform noise recognition and removal operations on the first image to obtain a second image;

[0068] Step 103: using a cultural and museum image segmentation model based on the RT-DETR model to perform segmentation processing on the second image to obtain a third image containing target information;

[0069] Step 104: Use the cultural and historical image target information extraction model based on the improved ResNet-101 to identify the target information of the third image, and determine the target information contained in the original cultural and historical image.

[0070] The method of this embodiment is used to perform preprocessing operations on original cultural and historical images, and automatically identify and extract required information. It can efficiently realize denoising, target detection and information extraction of original cultural and historical images, and has important technical value and application prospects.

[0071] This embodiment includes the following steps:

[0072] S1. Obtain original cultural and museum image data;

[0073] S2, perform preliminary image preprocessing operations;

[0074] This embodiment uses CLAHE image enhancement technology to increase the range of pixel brightness values ​​in cultural and historical images, thereby improving the visual effect and quality of the images; uses a Gaussian filter to achieve preliminary denoising, smoothing and edge detection of cultural and historical images; performs binarization processing to set the pixel grayscale value of the image to 0 or 255, thereby converting the image into an obvious black and white effect, which is convenient for distinguishing between text and background and is beneficial to subsequent model feature extraction work.

[0075] S3. Establish a VGG16-Unet cultural image preprocessing model based on deep learning. After manually removing the noise of the original image to obtain a group of clean and noisy images, obtain a large number of model training sets through image subtraction, noise classification, noise enhancement, image synthesis and other operations. With the help of the VGG16-Unet framework, extract noise features, accurately identify and remove image noise, and realize automatic preprocessing of image noise.

[0076] This embodiment first collects and enhances an image data set, and after completing the manual processing of the original data set, extracts a cultural and museum image noise map based on an image subtraction operation based on the Python-OpenCV library.

[0077] Since the composition of cultural and museum image noise is extremely complex and there are few typical combination rules, in order to simulate the real cultural and museum image noise to the maximum extent, this embodiment enhances the noise. The extracted noise image is cropped, and the Python-OpenCV library is used to perform geometric transformations such as flipping, rotating, random cropping, and translation transformation on the noise image, thereby enhancing the noise image and greatly expanding the cultural and museum image noise dataset.

[0078] The noise image is obtained by random combination, and the new noisy image is synthesized with the clean image to expand the original data pair and successfully complete the construction of the model training set.

[0079] Secondly, the preprocessing model is constructed. The VGG16-Unet network used in this embodiment consists of two parts: an encoder and a decoder. The encoder module adopts the VGG16 network part, including 5 convolutional layers and 4 maximum pooling layers. After encoding the cultural and museum images, 4 feature information of different scales and abstract levels from shallow to deep layers will be obtained, and then the feature images of the same scale in the decoder module are fused in the channel direction through the jump connection in Unet to make up for the edge information lost in the downsampling process of the image.

[0080] The decoder is mainly composed of an upsampling module and a convolutional layer. The upsampling module uses bilinear interpolation, and the number of upsampling is equal to the number of pooling layers in the encoder. After the input image is feature extracted by the encoder, it is passed to the decoder and then goes through multiple upsampling processes to finally obtain a segmentation prediction image of the same size as the input image.

[0081] Furthermore, the model should determine a suitable loss function and evaluation index, and the selections in this embodiment are as follows:

[0082] The loss function is used to describe the difference between the predicted value and the true value. By continuously optimizing the model, the model converges, thereby minimizing the model loss and obtaining an ideal model prediction result. For the cultural image preprocessing model, the denoising is relatively complex and it is necessary to ensure that the model has a good fitting effect. Therefore, the present invention selects the mean square error as the loss function to complete the model training, which is expressed as MSE. Clean image of and its noisy image For example:

[0083]

[0084] in, express Pixels in the clean image, express Pixels in a noisy image.

[0085] In addition, during the model training process, in addition to calculating the model loss, it is also necessary to select appropriate indicators to evaluate the performance of the model. In the field of image denoising, evaluation indicators can be divided into two categories: subjective evaluation and objective evaluation. Subjective evaluation indicators refer to the subjective and intuitive feelings and judgments given by people when observing the denoised images. Objective evaluation indicators include peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).

[0086] Peak signal-to-noise ratio is the most common indicator for evaluating image quality in image denoising tasks. In this embodiment, it is expressed as PSNR:

[0087]

[0088] in, Indicates the maximum value of pixels in the image. Taking into account the particularity of cultural and museum images, this embodiment stipulates that a PSNR higher than 15dB is a qualified image, and the higher the PSNR value, the higher the image processing quality.

[0089] Structural similarity evaluates image quality from three aspects: image brightness, contrast, and structure. The comparison function of the three can be expressed as follows:

[0090]

[0091]

[0092]

[0093] in , and They represent brightness contrast function, contrast contrast function and structure contrast function respectively. , Represent the average value of the two images, , They represent the variance of the two images respectively, Represent the standard deviation of the clean image and the noisy image respectively, represents the covariance of the two images, , and represents a constant, is the covariance of the image pair consisting of the clean image and the noisy image, and its formula is:

[0094]

[0095] Where N is the total number of pixels in the image, represents the pixel value of the clean image at the i-th pixel, represents the pixel value of the noise image at the i-th pixel, and i represents the i-th pixel in the image.

[0096] Based on the above three functions, let , the structural similarity SSIM can be expressed as:

[0097]

[0098] The value range of SSIM is (0,1]. The closer the SSIM value is to 1, the closer the denoised image is to the original image, and the better the denoising effect.

[0099] Furthermore, in order to minimize the overfitting phenomenon, this embodiment uses a dropout layer in the model. After the dropout layer is introduced, the network will randomly discard hidden units, and each unit has a 1 / 2 probability of being hidden. After the hidden unit is discarded, it will no longer participate in forward and reverse propagation. The dropout layer provides a simple and effective method for training and predicting convolutional neural networks, which helps to improve the accuracy of the model.

[0100] S4. Modeling is performed based on the RT-DETR network as the basic framework of the model, and a backbone network, neck network and decoder based on network features are constructed. The denoised cultural and museum images are used as training sets for training to achieve automatic segmentation of target information.

[0101] This embodiment uses the improved DETR algorithm to build an image segmentation model. For the backbone network, RT-DETR uses a CNN network, and the target detector extracts three scale outputs from the backbone network, with output steps of 8, 16, and 32 respectively. Considering that cultural relics images usually contain complex detailed textures and diverse graphic patterns, through multi-scale output, the model can better capture the detailed features in the cultural relics images, such as the relief lines or the strokes of ancient characters, thereby achieving accurate image segmentation effects.

[0102] This embodiment reduces the dimensionality of the two-dimensional S5 feature into a feature vector, passes it to the AIFI module for processing, and re-two-dimensionalizes the output, denoted as S5, to complete the subsequent cross-scale feature fusion. In the process of cultural and museum image processing, the high-level semantic features of the S5 feature are particularly important, because many cultural relics have complex historical symbols, textures or traces on their surfaces, and this information may be difficult to clearly distinguish in shallow features. Dimensionality reduction simplifies the calculation, but retains key deep semantic information, which is crucial for the classification and feature extraction of cultural relics.

[0103] Furthermore, the AIFI of the RT-DETR model only processes the last S5 feature, which is based on the following two points:

[0104] Traditional DETR converts multi-scale features into one dimension and then concatenates them to form a very long vector. Although this allows the multi-scale features to fully interact, in the cultural and museum scene, since cultural relics images are often high-resolution and rich in details, directly processing all multi-scale features will significantly increase the computational cost. Choosing to process only S5 features can reduce the computational burden while ensuring that the model can still capture the most discriminative deep semantic information.

[0105] In addition, compared with the shallower S3 and S4 features, the S5 feature has deeper, more advanced, and richer semantic features, which is effective in distinguishing the features of different objects, while the shallow features have little effect due to the lack of good semantic features. In the field of cultural relics, the high-level semantic information of the S5 feature helps to conduct detailed analysis and classification of historical relics, especially in the subtle differences between different forms of cultural relics, such as distinguishing bronze or porcelain from different ages.

[0106] Furthermore, in terms of decoder selection, the RT-DETR model uses the Transformer decoder. Specifically, it uses the DINO decoder part, namely the DINO Head, and uses DINO's "denoising idea" to improve the sample quality of bilateral matching and speed up the convergence of training.

[0107] Based on the image segmentation model constructed above, the denoised cultural and museum images are used for model training until the model converges.

[0108] Furthermore, the evaluation indicators that focus on model training include:

[0109] IOU is a metric for evaluating the proximity of two bounding boxes. This metric takes into account both size and position, and represents the area of ​​intersection / area of ​​union, with a range of [0, 1]. The larger the IOU, the closer the two bounding boxes are, 1 represents complete overlap, and 0 represents non-intersection.

[0110] Specifically, IOU can be tested and verified based on indicators such as Precision, Recall, F1, AP, and mAP.

[0111]

[0112]

[0113] TP (True positive) refers to the correct detection of the real detection box (ground truths), at this time the IOU detection is greater than or equal to the threshold; FP (False positive) refers to the wrong detection of a non-existent object or the wrong detection of the position of an existing object, at this time the IOU detection is less than the threshold; FN (False negative) refers to ground truths that have not been detected. is the number of all predicted boxes, is the number of all true detection boxes.

[0114] The threshold depends on the indicators of the specific task and is usually 50%, 75% or 95%.

[0115] Precision is equal to the proportion of correctly predicted detection values ​​among all detection values, reflecting the ability of the model to identify only relevant objects. Recall is equal to the proportion of correctly predicted detection values ​​among all ground truths, reflecting the ability of the model to distinguish all relevant objects.

[0116] Ideally, the model's FN = 0, that is, Recall (recall rate) is very high, and FP = 0, that is, Precision (accuracy) is very high.

[0117] F1 is an evaluation index that combines Precision and Recall. It can also be used to evaluate the model training effect. Its calculation formula is as follows:

[0118]

[0119] Among them, P is the value of precision and R is the value of recall.

[0120] Based on the above, the Precision Recall curve (PR curve) can be used to evaluate the trade-off between Precision and Recall of the model at different confidence thresholds. If the accuracy of the model remains high as Recall increases, it means that the model performs well, which means that if the confidence threshold is changed, Precision and Recall will still be high.

[0121] AP refers to the area under the PR curve (AUC). The higher the AP, the higher the Precision and Recall of the model. However, the PR curve is often jagged, so it is difficult to accurately measure the AUC of the curve. Before estimating the AUC, the PR curve needs to be processed to eliminate the jagged phenomenon. This embodiment uses the full-point interpolation method for processing. The full-point interpolation method does not use the Precision observed at only a few points, but obtains AP by interpolating the Precision of R at each level, taking the maximum Precision value where R is greater than or equal to Rn+1, as shown in the following formula:

[0122]

[0123] in, , is the recall rate of the n+1th sample, is the recall rate of the nth sample, is the average accuracy, which refers to the average accuracy under different recall rates. It is the value of the maximum accuracy corresponding to the recall rate exceeding the n+1th sample.

[0124] mAP is an indicator that measures the accuracy of the model on all categories, that is, the average of AP on all categories, as shown in the following formula:

[0125]

[0126] Among them, mAP is for all categories The average value of , N is the number of samples.

[0127] S5. Training is performed based on the improved ResNet-101 model. The structure of the improved model of the present invention is divided into an input layer, a residual block, a convolutional layer, a pooling layer, and a fully connected layer. The cross-extraction loss function, the Adam algorithm, and the dual attention network are used to effectively improve the recognition accuracy and efficiency of the target information. The existing data is used as a training set to train the ResNet-101 model, thereby realizing automatic recognition and presentation of the images in the test set.

[0128] Improvements and optimizations are made based on the ResNet-101 model, and model training is performed. ResNet-101 is a deep residual network that is obtained by adding more layers to ResNet-50. Its network structure is greatly improved compared to traditional deep convolutional neural networks, mainly reflected in the introduction of residual structures, which makes the network easier to train and has stronger generalization capabilities. In cultural and museum scenarios, the subtle differences between different cultural relics are crucial for classification and information extraction. The deep structure of ResNet-101 can effectively capture the complex features in cultural relic images, especially in the recognition of fine textures and historical traces on cultural relics. Due to the diversity of cultural relic types and materials, models with strong generalization capabilities such as ResNet-101 can better adapt to the image features of different cultural relics, thereby improving the applicability of the model to a variety of cultural and museum data sets. The network structure of ResNet-101 includes:

[0129] The residual block is the core structure of ResNet-101 in the present invention. Each residual block contains a convolution layer and an activation function layer, wherein the core of the residual block is not only to pass the output of the convolution layer to the next layer, but also to add the input data to the data processed by the convolution and activation functions, i.e., residual connection. This design helps to solve the gradient vanishing problem in the deep network training of cultural relics image processing, maintains sensitivity to the complex details of cultural relics images, and speeds up the recognition speed, making it possible to batch process massive high-precision cultural relics images.

[0130] The loss function of this embodiment is the cross entropy loss function, which is used to measure the difference between two probability distributions and is usually used to evaluate the difference between the model output and the true label. In the classification task, the smaller the cross entropy between the probability distribution of the model output and the probability distribution of the true label, the more accurate the model's prediction is. The formula is:

[0131]

[0132] in, is the probability of the i-th category of the true label, is the probability of the i-th category of the model's predicted output.

[0133] This embodiment uses a dual attention network (DANet) to adaptively integrate local features and global dependencies. Two types of attention modules are attached to the residual network to simulate semantic interdependencies in the spatial and channel dimensions respectively. Considering that cultural relics images often contain complex textures and historical traces, it is critical to capture the dependencies between these local details and the overall background. DANet processes local and global features simultaneously, so that important details and overall semantic information in cultural relics images can be obtained more accurately, thereby improving the accuracy of cultural relics classification and information extraction. This dual attention network (DANet) can improve the accuracy of target information recognition tasks, including:

[0134] The spatial attention module selectively aggregates the features at each position by weighted sum of the features at all positions. Similar features will be related to each other regardless of the distance. Since the features such as patterns, symbols or carvings on the cultural relics may appear in different areas, but they are highly related semantically. Through the spatial attention module, these similar features can be adaptively associated with each other, which helps to extract accurate historical information and cultural significance in the cultural relic images. The spatial attention module with the derivable characteristic does not need to be redundantly labeled, and can adaptively learn the spatial transformation methods of different types of data at any time. It can not only perform spatial transformation on all input signals, but also directly insert it as a network module into any layer of the existing network, thereby realizing the spatial transformation of different feature maps. In the context of the diversity and complexity of cultural relic images, this adaptive spatial transformation can effectively process different types of cultural relic images regardless of their shooting angle, lighting conditions or background environment. For the channel attention module, this embodiment uses a similar self-attention mechanism to capture the channel dependency between any two channel maps, and uses the weighted sum of all channel maps to update each channel map. By capturing the complex relationship between cultural and museum information, the recognition ability of complex cultural relic details can be effectively improved. Finally, the outputs of the two attention modules are fused to further enhance the feature representation. This fusion mechanism is particularly suitable for multi-scale and multi-channel information extraction of cultural relics images, helping the system to better identify and classify complex features in cultural relics images and improve overall processing accuracy.

[0135] Compared with the prior art, this embodiment has the following beneficial effects:

[0136] The improved VGG16-Unet image preprocessing model has good recognition effect on three types of noise, good denoising effect, and can process cultural and museum image data with complex noise conditions;

[0137] The cultural and museum image segmentation model based on RT-DETR has a high text segmentation success rate, strong stability, and faster training convergence speed, which is suitable for the segmentation operation of large-scale cultural and museum images;

[0138] The ResNet-101 target information recognition model that introduces a dual attention mechanism has high accuracy and stronger generalization ability.

[0139] In summary, the cultural and historical image denoising and information extraction method based on image recognition proposed in this embodiment effectively solves the problems of noise interference and difficulty in information extraction in cultural and historical images through three steps: image denoising preprocessing, automatic image segmentation, and target information recognition. This method not only improves the clarity of cultural and historical images, but also greatly improves the accuracy of target information segmentation and recognition, and is suitable for a variety of application scenarios such as cultural relics protection, digital archiving, and academic research.

[0140] Below, this solution will be described in detail based on a specific application embodiment.

[0141] See also Figure 2 , which is a flow chart of a method for denoising and extracting information from cultural and museum images based on image recognition technology provided in this embodiment. The method includes the following processes:

[0142] S1. Obtain original cultural and historical image data.

[0143] S2. Perform preliminary image preprocessing operations.

[0144] S3. Establish a VGG16-Unet cultural image preprocessing model based on deep learning to accurately identify and remove image noise and realize automatic preprocessing of image noise.

[0145] S4. Establish a cultural and museum image segmentation model based on the RT-DETR model, construct a backbone network, neck network and decoder based on network features, and use the denoised cultural and museum images as training sets for training to achieve automatic segmentation of target information.

[0146] S5. Establish a cultural and museum image target information extraction model based on the improved ResNet-101, and use the existing data as the training set for model training to achieve automatic recognition and presentation of images in the test set.

[0147] See also Figure 3 , which is an original image of a specific embodiment of the present invention. The cultural image itself has the problems of unclear structure and irregular noise. The traditional image preprocessing operation based on filters and morphology is difficult to accurately identify and process the interference elements in this embodiment. Specifically, the interference factors of the cultural image are divided into three categories, namely point noise, artificial texture and inherent texture.

[0148] Please refer to Figure 4 (a), Figure 4 (b) and Figure 4 (c), based on Figure 3 The noise type analysis of the original cultural and museum images, the preliminary image preprocessing operation of this embodiment includes:

[0149] (1) Perform color correction on the image to remove color deviation caused by changes in lighting conditions, making the image color more realistic;

[0150] (2) The brightness and contrast of the image are adjusted by using histogram equalization technology to highlight the detail information in the cultural and historical images; the edge of the cultural and historical images is enhanced by using morphological filtering technology to ensure that the details at the edge of the image are clearer.

[0151] (3) Perform binarization processing and set the pixel grayscale value of the image to 0 or 255, thereby converting the image to an obvious black and white effect.

[0152] It should be noted that in the specific implementation process, 6150 original cultural and museum pictures were collected, and the noise of 80 images was removed by using image processing tools, resulting in 80 pairs of images (each image pair includes an image with noise and an image with noise removed). Considering that deep learning relies on data-driven, the quantity and quality of data affect the quality of the model to a certain extent. In order to prevent the underfitting problem caused by the small size of the data set and to improve the model's recognition ability for point noise, artificial textures and inherent textures, this embodiment uses data set enhancement to provide the model with a better training set for iterative training.

[0153] See also Figure 5 Based on the above ideas, this embodiment extracts three types of image noise based on the image subtraction operation of the Python-OpenCV library, and performs geometric transformations such as flipping, rotating, randomly cropping, and translation transformation on the noise map, thereby enhancing the noise map and greatly expanding the oracle bone rubbing noise data set. Figure 5 The synthesis operation of the noisy image in this embodiment is demonstrated. Specifically, by randomly combining three types of noisy images and synthesizing the randomly combined noise images with the clean images, a new noisy image can be obtained, thereby expanding the original data pair. Figure 5 From left to right, they are the clean image obtained after artificial denoising and the synthetic image with random noise added. So far, this embodiment has obtained a total of about 800 noisy cultural and museum images through the collection, processing and enhancement operations of the image data set, and successfully completed the construction of the model training set.

[0154] See also Figure 6 , which shows the model framework of step S3 of the present invention, that is, the model framework of the VGG16-Unet cultural and historical image preprocessing model based on deep learning. This embodiment uses the 800 noisy cultural and historical images completed in the above steps as the model training set for training, uses the mean square error (MSE) as the loss function of this model, and uses the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) as the evaluation indicators of the output results of this model, and the formula is as follows:

[0155]

[0156]

[0157]

[0158] In this step, in order to minimize overfitting, this model uses a dropout layer. After the dropout layer is introduced, the network will randomly discard hidden units, and each unit has a 1 / 2 chance of being hidden. After the hidden unit is discarded, it will no longer participate in forward and backward propagation. The dropout layer provides a simple and effective method for training and predicting convolutional neural networks, which helps to improve the accuracy of the model.

[0159] The following table shows the differences in PSNR and SSIM indicators under different operations in the specific implementation process. It can be found that the image quality obtained by using the VGG16-Unet preprocessing model is significantly higher than the traditional denoising result, and the denoising quality is higher.

[0160]

[0161] See also Figure 7 , which shows a schematic diagram of the model of the cultural and museum image segmentation model based on the RT-DETR model in step S4 of the present invention. The main structural framework of the model is as follows:

[0162] (1) Backbone network: In this embodiment, the backbone network adopts a CNN network, and the target detector extracts outputs of three scales from the backbone network, and the output step sizes are 8, 16, and 32 respectively.

[0163] (2) Neck network: The neck network of this embodiment includes reducing the two-dimensional S5 feature into a feature vector, handing it over to the AIFI module for processing, and re-two-dimensionalizing the output, denoted as S5, to complete the subsequent cross-scale feature fusion. Compared with the shallower S3 and S4 features, the S5 feature has deeper, more advanced, and richer semantic features, which is effective in distinguishing the features of different objects, while the shallow features have little effect due to the lack of good semantic features.

[0164] (3) Decoder: In this embodiment, the decoder of Transformer is selected. Specifically, the decoder part of DINO, namely DINO Head, is selected, and the "denoising idea" of DINO is used to improve the sample quality of bilateral matching and accelerate the convergence speed of training.

[0165] (4) Loss function: The loss function setting of this embodiment is also the same as DINO, that is, DINO loss, which is a specific loss function that combines cross entropy loss and adversarial loss. The formula is as follows:

[0166]

[0167] in, and are hyperparameters that control the weights of the cross entropy loss and adversarial loss, respectively.

[0168] In this step, RT-DETR, YOLOv8, and YOLOv5 models are used for training respectively, and the training set is the preprocessed cultural and museum images. The training configuration GPU of this embodiment is INVDIA Tesla P100, the learning rate is 0.001667, the Adam optimization algorithm is used, the epoch is 20, and the batch size is 8. Please refer to Figures 8 (a), 8 (b), and 8 (c), which show the curves of the loss values ​​of the three models during the training process as the epoch changes. It can be found that the loss values ​​of the three models are in a downward trend in a total of 20 epochs in this embodiment, and the verification set gradually converges. In addition, the loss value of the RT model is significantly lower than that of the other two models, and the model training effect is better.

[0169] It should be noted that, in order to select the best target detection model, this embodiment uses IOU as an important evaluation indicator for target segmentation effect evaluation, and its effect diagram is shown in the following figure: Fig. 9 Specifically, Precision, Recall, F1 value, AP (Average Precision) and mAP (mean AP) are used to quantify IOU and evaluate the model effect.

[0170] See also Fig.10 , which shows the PR curve of the RT-DETR model. Generally speaking, the larger the area wrapped by the curve, the larger the balance point (the value of P=R), and the better the performance. The PR curves of the three models all have large areas, but in comparison, the RT-DETR model performs better. The results of other evaluation indicators in this embodiment are shown in the following table. It can be seen that the RT-DETR model not only performs better in various indicators, but also can realize automatic target information segmentation in a short time, which can provide reliable decision support for scientific researchers in the field of cultural relics.

[0171]

[0172] See also Fig.11 , which shows the segmentation results of the target information (the target information in this embodiment is oracle bone script) by the RT-DETR model in this embodiment. The recognition box indicates that the model has recognized the target information, the label on the recognition box indicates whether it is an oracle bone script, and the number next to the label indicates the confidence level of the oracle bone script.

[0173] See also Figure 12-14These figures show the specific network structure of the cultural and museum image target information extraction model based on ResNet-101 in step S5 of the present invention. Specifically, the structure of the model mainly includes:

[0174] (1) Residual block: Each residual block contains a convolution layer and an activation function layer. The core of the residual block is not only to pass the output of the convolution layer to the next layer, but also to add the input data to the data processed by the convolution and activation functions, that is, the residual connection. This design helps to solve the gradient vanishing problem in deep network training. The residual unit structure of this model is as follows: Fig.12 shown.

[0175] (2) Loss function: The loss function used in this embodiment is the cross entropy loss function, which is used to measure the difference between two probability distributions and is usually used to evaluate the difference between the model output and the true label. In classification tasks, the smaller the cross entropy between the probability distribution of the model output and the probability distribution of the true label, the more accurate the model's prediction. The formula is:

[0176]

[0177] (3) Attention mechanism module: This embodiment uses a dual attention network (DANet) to adaptively integrate local features and global dependencies. Two types of attention modules are attached to the residual network to simulate semantic interdependencies in the spatial and channel dimensions respectively. Fig.13 We present a spatial attention module that selectively aggregates features at each position via a weighted sum of features at all positions. Fig.14 The channel attention module is presented, which uses a similar self-attention mechanism to capture the channel dependency between any two channel maps and updates each channel map using the weighted sum of all channel maps. Figure 13-14 The outputs of these two attention modules are combined to further enhance the feature representation.

[0178] In the specific implementation process, the preprocessed images are used as training sets for training, the batchsize value is set to 12, the learning rate is set to 0.0001, and the epoch is 20. In general, the improved ResNet-101 model performs well in the test and can recognize most of the target information in the image, with an accuracy of 0.919, a Recall value of 0.918, and an F1 value of 0.807, which shows that the model of the present invention is highly feasible in this embodiment. Fig.15 The whole process of target information extraction in this embodiment is shown. Fig.16 The training and testing processes of the above three models are demonstrated.

[0179] In order to implement the method of an embodiment of the present invention, an embodiment of the present invention also provides a cultural and museum image processing system based on image recognition technology, including: a processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, it executes the steps of the above-mentioned method.

[0180] The above-mentioned system provided in this embodiment belongs to the same concept as the above-mentioned method embodiment. The specific implementation process thereof is detailed in the method embodiment and will not be repeated here.

[0181] In order to implement the method of the embodiment of the present invention, the embodiment of the present invention also provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the above method.

[0182] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present invention, the embodiment of the present invention further provides an electronic device (computer device). Specifically, in one embodiment, the computer device may be a terminal, and its internal structure diagram may be as follows: Fig.17 As shown. The computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05 and a memory (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A06. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor A01, the method of any one of the above embodiments is implemented. The display screen A04 of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device A05 of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0183] Those skilled in the art will understand that Fig.17 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0184] The device provided by the embodiment of the present invention includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the method of any one of the above embodiments is implemented.

[0185] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0186] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0187] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0188] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0189] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0190] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0191] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0192] It can be understood that the memory of the embodiment of the present invention can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0193] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0194] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for processing cultural and museum images based on image recognition technology, characterized in that: The method comprises: Acquire an original cultural and historical image; perform a preliminary preprocessing operation on the original cultural and historical image to obtain a first image; Use the VGG16-Unet cultural image preprocessing model based on deep learning to perform noise recognition and removal operations on the first image to obtain a second image; Using a cultural and museum image segmentation model based on the RT-DETR model to segment the second image to obtain a third image containing target information; Using a cultural and historical image target information extraction model based on an improved ResNet-101 to identify target information of the third image, and determine the target information contained in the original cultural and historical image; The VGG16-Unet cultural and museum image preprocessing model based on deep learning includes an encoder and a decoder, the encoder includes 5 convolutional layers and 4 maximum pooling layers, and the decoder includes an upsampling module and a convolutional layer; the training process of the VGG16-Unet cultural and museum image preprocessing model based on deep learning includes: Acquire a training data set, wherein the training data set includes a clean image from which noise has been removed and a noisy image from which noise has not been removed; Perform image subtraction processing on the clean image and the noise image using an image subtraction technique to obtain a cultural and museum image noise map; Flipping, rotating, randomly cropping and / or translating the cultural and historical image noise map to complete the expansion of the cultural and historical image noise map; The expanded cultural and museum image noise map is synthesized with the clean image to obtain an expanded training data set; Using the training data set to train the VGG16-Unet cultural and museum image preprocessing model based on deep learning; Among them, the training loss function of the VGG16-Unet cultural image preprocessing model based on deep learning is: in, express Pixels in the clean image, express Pixels in a noisy image.

2. The method for processing cultural and museum images based on image recognition technology according to claim 1, characterized in that: The original cultural and museum images are pre-processed, including: Using image enhancement technology to process the original cultural and historical image, thereby increasing the range of pixel brightness values ​​in the original cultural and historical image; The original cultural and museum images with increased brightness value range are processed using a Gaussian filter to complete preliminary denoising, smoothing and edge detection of the original cultural and museum images; The pixel grayscale value of the original cultural and historical image is set to 0 or 255, so that the original cultural and historical image is converted into a black and white effect image.

3. The method for processing cultural and museum images based on image recognition technology according to claim 1, characterized in that: The VGG16-Unet cultural and museum image preprocessing model based on deep learning is trained using the training data set, including: In the process of training the training data set using the VGG16-Unet cultural image preprocessing model based on deep learning, calculating evaluation index values, wherein the evaluation index values ​​include peak signal-to-noise ratio and structural similarity; When the peak signal-to-noise ratio is higher than a preset first threshold, it is determined that the processing quality of the trained VGG16-Unet cultural image preprocessing model based on deep learning is qualified; and / or, When the structural similarity is greater than a preset second threshold, it is determined that the denoising effect of the trained VGG16-Unet cultural image preprocessing model based on deep learning is qualified; The peak signal-to-noise ratio is calculated as: in, is the peak signal-to-noise ratio, represents the maximum value of pixels in the image, and MSE represents the loss function of the VGG16-Unet cultural image preprocessing model based on deep learning; The calculation formula for the structural similarity is: in, , and Respectively represent brightness contrast function, contrast contrast function and structure contrast function, where, in, , Represent the average values ​​of the clean image and the noisy image respectively, , denote the variance of the clean image and the noisy image respectively, Represent the standard deviation of the clean image and the noisy image respectively, , and represents a constant, , is the covariance of the image pair consisting of the clean image and the noisy image, and its formula is: Where N is the total number of pixels in the image, represents the pixel value of the clean image at the i-th pixel, represents the pixel value of the noise image at the i-th pixel, and i represents the i-th pixel in the image.

4. The method for processing cultural and museum images based on image recognition technology according to claim 1, characterized in that: The cultural and museum image segmentation model based on the RT-DETR model includes a backbone network, a neck network and a decoder; the backbone network is used for feature extraction; the neck network is used to further aggregate multi-layer feature maps and enhance the multi-scale representation capability of the image through upsampling and downsampling operations; the decoder is used to process the key information in the feature map using a fully connected layer and non-maximum suppression technology, identify the target information and complete the segmentation; the loss function of the cultural and museum image segmentation model based on the RT-DETR model during the training process is: in, is the cross entropy loss, For adversarial loss, and are the hyperparameters that control the weights of the cross entropy loss and adversarial loss respectively.

5. The method for processing cultural and museum images based on image recognition technology according to claim 4 is characterized in that: The evaluation indicators of the cultural and museum image segmentation model based on the RT-DETR model during the training process include accuracy, recall, F1, AP and mAP; in, Among them, TP is the correct detection of the real detection box, FP is the wrong detection of the non-existent object or the wrong detection of the position of the existing object, and FN is the real detection box that has not been detected; is the number of all predicted boxes, is the number of all true detection boxes, P is the precision value, R is the recall value, is the recall rate of the n+1th sample, is the recall rate of the nth sample, is the average accuracy, which refers to the average accuracy under different recall rates, and mAP is the average accuracy of all categories The average value of N is the number of samples. It is the value of the maximum accuracy corresponding to the recall rate exceeding the n+1th sample.

6. The method for processing cultural and museum images based on image recognition technology according to claim 1, characterized in that: The cultural and museum image target information extraction model based on the improved ResNet-101 includes multiple residual blocks and dual attention network modules: each of the residual blocks includes a convolution layer and an activation function layer, and the residual block is used to pass the output of the convolution layer to the next layer, and add the input data to the data processed by the convolution and activation function layers; the dual attention network module includes a spatial attention module and a channel attention module; the spatial attention module is used to selectively aggregate the features of each position by the weighted sum of the features at all positions; the channel attention module is used to capture the channel dependency between any two channel maps using a self-attention mechanism, and update each channel map by the weighted sum of all channel maps; the loss function of the training process of the cultural and museum image target information extraction model based on the improved ResNet-101 includes: in, is the probability of the i-th category of the true label, is the probability of the i-th category of the model prediction output.

7. A cultural and museum image processing system based on image recognition technology, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, the steps of the method according to any one of claims 1 to 6 are executed.

8. A storage medium storing a computer program, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the method according to any one of claims 1 to 6.

Images