Target detection method and system for non-perpetual cultural image

Abstract

The invention provides a target detection method and system for a non-perpetual cultural image, and relates to the field of image intelligent recognition. The method comprises the following steps: pre-processing a non-perpetual culture image, and extracting high-order semantic features of the pre-processed image by using a feature extractor; on the basis of the high-order semantic features of the preprocessed images, calculating the joint similarity between the different preprocessed images; mapping the high-order semantic features of the preprocessed image to a low dimension based on joint similarity to obtain a dimension-reduced image; performing clustering operation on the dimension-reduced image to generate a clustering label; randomly extracting dimension-reduced images in the clustering labels, comparing non-perpetual and cultural images, confirming names corresponding to the clustering labels, generating a mapping table of the labels and the names, and obtaining non-perpetual and cultural labeled images; and training a target detection model based on the non-missing labeled image, and outputting a target detection result through the target detection model. The problem that a traditional detection model is low in recognition accuracy when facing diversified non-perpetual culture images is solved.

Description

Technical Field

[0001] This application relates to the field of intelligent image recognition, and in particular to a target detection method and system for images of intangible cultural heritage. Background Technology

[0002] With the popularization of high-precision imaging equipment and the continuous development of digital acquisition methods, a large number of high-resolution painting images are being generated and entering the digital management and application system. Against the backdrop of the large-scale accumulation and refined utilization of the aforementioned cultural resources, the systematic protection and digital inheritance of intangible cultural heritage through intelligent technology has become one of the main directions for the digital and intelligent development of cultural resources.

[0003] Existing object detection systems for intangible cultural heritage (ICH) images generally rely on manual prior definition of categories and individual annotation of training samples. This results in a labeling process that is highly dependent on expert experience, involves a huge workload, and is difficult to guarantee consistency. Furthermore, when sample categories are incomplete or category boundaries are ambiguous, the training performance and generalization ability of the object detection model are significantly limited. Therefore, traditional detection models based on direct supervised learning cannot effectively utilize the overall semantic similarity structure among ICH images using only the above methods. This makes model training highly sensitive to the number of samples and the quality of manual annotation, leading to low recognition accuracy when faced with diverse ICH images.

[0004] Therefore, there is an urgent need for a target detection method and system for intangible cultural heritage images. Summary of the Invention

[0005] This application provides a target detection method and system for intangible cultural heritage images, which solves the problem that traditional detection models based on direct supervised learning cannot effectively utilize the overall semantic similarity structure between intangible cultural heritage images and have low recognition accuracy when faced with diverse intangible cultural heritage images.

[0006] The first aspect of this application provides a target detection method for intangible cultural heritage images. The method includes: acquiring intangible cultural heritage images; preprocessing the intangible cultural heritage images to obtain preprocessed images; extracting high-order semantic features of the preprocessed images using a feature extractor; calculating the direct similarity and indirect similarity between different preprocessed images based on the high-order semantic features of the preprocessed images, and averaging the direct and indirect similarities to obtain joint similarity; defining a first loss function based on joint similarity, minimizing the loss value through gradient descent, and mapping the high-order semantic features of the preprocessed images to a low dimension to obtain a dimensionality-reduced image; performing clustering operations on the dimensionality-reduced image to generate cluster labels; randomly selecting dimensionality-reduced images from the cluster labels, comparing them with intangible cultural heritage images, confirming the names corresponding to the cluster labels, generating a mapping table between labels and names to obtain intangible cultural heritage labeled images; training a target detection model based on the intangible cultural heritage labeled images, and processing the intangible cultural heritage objects to be detected using the target detection model, and outputting the target detection results.

[0007] A second aspect of this application provides a target detection system for intangible cultural heritage images, the system comprising an acquisition module and a processing module, wherein, The acquisition module is used to collect images of intangible cultural heritage (ICH) and preprocess them to obtain preprocessed images. A feature extractor is used to extract high-order semantic features from the preprocessed images. Based on these high-order semantic features, direct and indirect similarities between different preprocessed images are calculated, and the direct and indirect similarities are averaged to obtain joint similarity. Based on the joint similarity, a first loss function is defined, and gradient descent is used to minimize the loss value, mapping the high-order semantic features of the preprocessed images to a lower dimension to obtain a dimensionality-reduced image. Clustering is performed on the dimensionality-reduced image to generate cluster labels. Dimensionality-reduced images from the cluster labels are randomly selected, compared with the ICH images, and the names corresponding to the cluster labels are confirmed. A mapping table between labels and names is generated to obtain ICH-labeled images.

[0008] The processing module is used to train a target detection model based on the intangible cultural heritage labeled image, and to process the intangible cultural heritage object to be detected through the target detection model, and output the target detection result.

[0009] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described above.

[0010] A fourth aspect of this application provides a non-transitory computer-readable storage medium storing a computer program, the computer program being executed by a processor using any of the methods described above.

[0011] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. Preprocess the intangible cultural heritage (ICH) images, using a feature extractor to extract high-order semantic features from the preprocessed images; calculate the joint similarity between different preprocessed images based on the high-order semantic features; map the high-order semantic features of the preprocessed images to low dimensions based on the joint similarity to obtain dimensionality-reduced images; perform clustering operations on the dimensionality-reduced images to generate cluster labels; randomly select dimensionality-reduced images from the cluster labels, compare them with ICH images, confirm the names corresponding to the cluster labels, generate a mapping table between labels and names, and obtain ICH-annotated images; train an object detection model based on the ICH-annotated images, and input the data through the object detection model. The system outputs target detection results, enabling automatic discovery, semantic confirmation, and standardized annotation of target element categories in intangible cultural heritage images without the need for pre-completion manual annotation. The resulting highly consistent annotation results are effectively incorporated into the training process of the target detection model, allowing the model to stably output the category name, spatial location boundary, and corresponding confidence score of target elements in intangible cultural heritage scenes with complex forms, diverse styles, and uneven sample distribution. This significantly reduces the reliance on manual annotation, improves annotation efficiency and consistency, and enhances the target detection model's recognition accuracy and generalization ability for intangible cultural heritage target elements.

[0012] 2. Obtain the label information of the detection feature map and establish co-occurrence relationships between different label information to obtain associated features; flatten the associated features into a one-dimensional vector and map the one-dimensional vector to a new feature space through a linear layer to obtain the first spatial features; expand the first spatial features to a four-dimensional space to obtain the second spatial features; extract the classification branch features of the detection feature map through a preset convolutional layer and convert the classification branch features into a weight vector; based on the weight vector, fuse the classification branch features with the second spatial features to obtain the classification result. Thus, in the classification prediction process, the local discriminative semantics in the target detection feature map and the global co-occurrence relationship at the label level are jointly modeled, so that the classification result not only depends on a single spatial location or local texture features, but also comprehensively reflects the co-occurrence rules and semantic association strength of different target categories in intangible cultural heritage images, thereby suppressing semantic confusion between categories, improving the ability to distinguish similar morphological target elements, and enhancing the stability and accuracy of classification prediction in complex intangible cultural heritage scenarios. Attached Figure Description

[0013] Figure 1This is a flowchart illustrating a target detection method for intangible cultural heritage images provided in an embodiment of this application; Figure 2 This is a schematic diagram of a target detection system for intangible cultural heritage images provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0014] Explanation of reference numerals in the attached figures: 21. Acquisition module; 22. Processing module; 301. Processor; 302. Communication bus; 303. User interface; 304. Network interface; 305. Memory. Detailed Implementation

[0015] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0016] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0017] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0018] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0019] Please refer to Figure 1 The flowchart illustrates a target detection method for intangible cultural heritage images provided in this application embodiment. The flowchart mainly includes the following steps: S101 to S107.

[0020] Step S101: Collect images of intangible cultural heritage, preprocess the images to obtain preprocessed images.

[0021] Specifically, images of intangible cultural heritage are collected; in this application, these images can be Thangka paintings. To eliminate size and color differences, the collected intangible cultural heritage images need to be scaled, have edges filled, randomly flipped, and normalized to RGB.

[0022] In one possible implementation, step S101 further includes: scaling the intangible cultural heritage image to a specified size using a proportional scaling formula; filling the edges of the scaled intangible cultural heritage image so that the length and width of the filled intangible cultural heritage image can be divided by a preset parameter to form a filled image; randomly flipping the filled image with a preset flip probability to form a flipped image; and performing RGB normalization processing on the flipped image to obtain a preprocessed image.

[0023] Specifically, common sizes for intangible cultural heritage images are 50×70cm, 80×60cm, 100×70cm, etc. A proportional scaling formula can be used: scaling factor = min(target width / original width, target height / original height) to uniformly scale the collected images to the specified size, which can preserve the edge and texture details of the intangible cultural heritage images.

[0024] Fill the left and right / top and bottom edges of the scaled image with black pixels. For example, if the preset parameter is 64 and the preset size of the scaled image is 800×638 pixels, the width of 800 is divisible by 64, and the height of 638 needs to be padded with 2 pixels to 640, resulting in a final 800×640 filled image.

[0025] The target elements in intangible cultural heritage images are mostly symmetrically distributed from left to right. The preset flip probability can be set to 0.5, which only performs horizontal flipping and does not change the semantics after flipping. For example, randomly flipping an intangible cultural heritage image containing floral patterns can simulate intangible cultural heritage images drawn from different angles.

[0026] Divide the RGB value (0-255) of each pixel in the flipped image by 255 to map it to the (0,1) range. For example, the red pixel (255,0,0) of mineral pigment in an intangible cultural heritage image is mapped to (1.0,0.0,0.0), which can reduce the interference of numerical range on model training.

[0027] Step S102: Use a feature extractor to extract high-order semantic features from the preprocessed image.

[0028] Specifically, a feature extractor extracts multi-scale features of the target element layer by layer, from basic texture to core semantics, to achieve accurate representation of the target element features. The feature extractor mainly consists of a convolutional layer, a first residual group, a second residual group, and a third residual group. The target element can be a pattern element of a cultural artifact, a decorative pattern element, a symbolic pattern element, or an image object element with a stable morphological structure in an intangible cultural heritage image.

[0029] In one possible implementation, step S102 further includes: extracting low-level features of the preprocessed image through a convolutional layer; extracting local features of the preprocessed image through a first residual group based on the low-level features; extracting medium-scale semantic features of the preprocessed image through a second residual group based on the local features; and extracting high-order semantic features of the preprocessed image through a third residual group based on the medium-scale semantic features.

[0030] Specifically, the convolutional layer consists of 3×3 convolutional kernels with a stride of 1 and 64 output channels. For example, if the preprocessed image is 800×640×3, after inputting into the convolutional layer, it outputs a 64-channel low-level feature map: 800×640×64. The low-level features include line edges, basic color gradients, etc., which can capture the basic forms of multi-branch artifact patterns and petal edges of flower patterns in intangible cultural heritage images.

[0031] The first residual group consists of one residual block with a stride of 2 and two residual blocks with a stride of 1. By further processing the underlying feature map of the intangible cultural heritage image through the three residual blocks of the first residual group, local feature maps such as small patterns and local color combinations of the intangible cultural heritage image can be extracted: 400×320×128, and the spoke pattern of the wheel-shaped structure in the intangible cultural heritage image can be identified.

[0032] The second residual group consists of two residual blocks with a stride of 2 and two residual blocks with a stride of 1. By processing the local feature map through the four residual blocks of the second residual group, a medium-scale semantic feature map of local pattern combination and color level of intangible cultural heritage image can be extracted: 100×80×512. It can distinguish the petal combination structure of flower pattern in intangible cultural heritage image and the handle and end combination structure of multi-branched object pattern.

[0033] The third residual group consists of two residual blocks with a stride of 2 and three residual blocks with a stride of 1. By processing the medium-scale semantic features of the intangible cultural heritage image through the five residual blocks of the third residual group, high-order semantic feature maps such as the overall texture layout and core shape semantics of the intangible cultural heritage image can be extracted. This can accurately represent the overall form of the Thangka target elements, such as the circular shape of a wheel-like structure and the three-strand shape of a multi-branched structure. The Thangka bottom-level feature map passes through the three residual groups sequentially. Each time it passes through a residual block with a stride of 2, the spatial dimension of the feature map is halved and the number of channels is doubled, ultimately outputting a high-order semantic feature map with 2048 channels: 25×20×2048.

[0034] Step S103: Based on the high-order semantic features of the preprocessed images, calculate the direct similarity and indirect similarity between different preprocessed images, and average the direct and indirect similarities to obtain the joint similarity.

[0035] Specifically, based on high-order semantic feature maps, the direct similarity, indirect similarity, and joint similarity between different preprocessed images are calculated, which can measure the similarity between images of different intangible cultural heritage. The specific calculation process is as follows: Direct similarity: Calculating preprocessed images , The similarity of feature vectors is calculated using the following formula:

[0036] in, This represents the Gaussian kernel bandwidth, which can be set to 0.1; , Preprocessed images , High-order semantic feature vectors The feature vectors are the squared Euclidean distance; if two images contain the same target elements, then their feature vectors are similar. The value is close to 1.

[0037] Indirect similarity: Defined as reverse similarity, its calculation logic is the same as direct similarity, and the formula is as follows:

[0038] Joint similarity: The average of direct and indirect similarities is calculated using the following formula:

[0039] in, The total number of preprocessed images can balance the two-way similarity bias and improve the stability of similarity measurement.

[0040] Step S104: Based on joint similarity, define the first loss function, minimize the loss value through gradient descent, and map the high-order semantic features of the preprocessed image to a low dimension to obtain a dimensionality-reduced image.

[0041] Specifically, joint similarity is The matrix, the high-order semantic feature map is The matrix is ​​optimized using the first loss function to map the 2048-dimensional high-order semantic feature map to a 2-dimensional space. The implementation process is as follows:

[0042] Based on joint similarity, the first loss function is defined as follows:

[0043] in, For the first in higher-dimensional space , Joint similarity of images related to Zhang's intangible cultural heritage; For the first in low-dimensional space , Similarity of images related to intangible cultural heritage; similar Thangkas containing the same target elements in higher dimensions, Enlarge and force Also large, when Thangkas in higher dimensions contain different target elements, and are forced It is also small, which ultimately allows the low-dimensional distribution to approach the high-dimensional semantic distribution.

[0044] Initializing low-dimensional features helps avoid large initial coordinate differences that could lead to local optima in subsequent optimization. The initial low-dimensional feature matrix is ​​randomly generated using a normal distribution with a mean of 0 and a variance of 0.0001. , shape is For example, the first The initial low-dimensional coordinates of Zhang's intangible cultural heritage image are ,in, All obey .

[0045] Calculating low-dimensional similarity alleviates feature crowding after mapping high-dimensional features to low-dimensional ones. For example, in images of intangible cultural heritage, target elements have many categories, and they are prone to overlap in low-dimensional space. The formula is as follows:

[0046] in, , The first , The low-dimensional feature vectors of Zhang's intangible cultural heritage images all have a dimension of 2; This is a t-distribution mapping term with 1 degree of freedom, which converts distance into similarity. To normalize the denominator, iterate through all low-dimensional intangible cultural heritage image pairs with k=l and sum all t-distribution mapping values.

[0047] Calculate the gradient of the first loss function. For the low-dimensional feature matrix... Each element , Let L1 represent the two lower dimensions. The gradient of the first loss function L1 is calculated as follows:

[0048]

[0049] Among them, when This indicates that similar intangible cultural heritage images in higher dimensions are not close enough in lower dimensions. If positive, the gradient direction points to... ,Will Pull towards ;when This indicates that dissimilar intangible cultural heritage images in higher dimensions appear too similar in lower dimensions. If it is negative, then the gradient direction points to... ,Will ; As weights, they can prevent samples that are too far apart in low dimensions from having too much influence on the gradient.

[0050] The low-dimensional feature matrix is ​​updated iteratively using gradient descent. The update formula for each round is as follows:

[0051] in, For the first The low-dimensional feature matrix of the wheel; The learning rate is set to 500, which controls the step size of each gradient descent round. 1 represents the gradient matrix of the first loss function, with a shape of M×2; The momentum coefficient is set to 0.9; The amount of the previous update is used to introduce momentum, which can accelerate convergence and avoid oscillations; the iterative constraint is to normalize the low-dimensional features after each update (mean is 0, variance is 1), which can ensure numerical stability.

[0052] Stop iteration and output the dimensionality-reduced image. Stop iteration when the number of iterations reaches 1000 rounds or the change in the loss value tends to stabilize after 50 consecutive rounds; the final iterative result is... The shape is M×2, and each row corresponds to the 2D coordinates of one intangible cultural heritage image. .

[0053] Step S105: Perform clustering operation on the dimensionality-reduced image to generate cluster labels.

[0054] Specifically, the preset number of clusters K is set to 15, the preset similarity threshold θ is set to 0.85, and the maximum number of iterations T is set to 300. Through cluster center initialization, definition of the second loss function, and cluster center iterative optimization, the M×2 dimensionality-reduced image is divided into 15 non-overlapping clusters. Each cluster corresponds to a type of target element. Finally, a unique clustering label, such as 0-14, is assigned to each intangible cultural heritage image, so as to achieve the goal of clustering the same target elements into one type and different target elements into different clusters.

[0055] In one possible implementation, step S105 further includes: acquiring the target dimensionality-reduced image as the cluster center; randomly selecting the target dimensionality-reduced image from the dimensionality-reduced images; calculating the minimum distance from the unselected dimensionality-reduced images to the selected cluster centers to form clusters; selecting the next cluster center in order of increasing distance until the number of cluster centers meets the preset selection requirement; calculating the sum of squared distances based on the distance from the dimensionality-reduced image to each cluster center, and defining a second loss function based on the sum of squared distances; iteratively updating the selected cluster centers along the gradient direction of the second loss function until the cosine value between each pair of dimensionality-reduced images in each cluster is greater than a preset similarity value; outputting the target cluster centers that meet the iterative update conditions, and assigning corresponding labels to the target cluster centers to generate cluster labels.

[0056] Specifically, using a random number generator, such as Python's np.random.randint(0,M), an index i is selected to randomly draw one sample from the dimensionality-reduced image of M intangible cultural heritage images as the first cluster center. The corresponding 2D features ( ).

[0057] For all samples not selected as cluster centers, calculate their Euclidean distances to each selected cluster center, and take the minimum value as the minimum distance for that sample. For unselected samples and the selected cluster center The Euclidean distance formula is:

[0058]

[0059] minimum distance The formula is:

[0060] Where k is the number of currently selected cluster centers, initially k=1. Each unselected sample is temporarily assigned to the cluster corresponding to the nearest selected cluster center, forming a temporary cluster class.

[0061] Select the next cluster center in ascending order of distance, until K cluster centers are selected; alternatively, prioritize the sample with the largest minimum distance D(x) as the next cluster center to ensure that the cluster centers are uniformly distributed in the reduced-dimensional space. The implementation method is as follows: Calculate D(x) for all unselected samples and sort them in descending order; select the first sample after sorting, i.e., the sample farthest from the existing cluster center, as the new cluster center. Repeat the cluster center selection process until K=15 cluster centers have been selected. , ,..., In this way, the 15 cluster centers can respectively cover the dimensionality reduction feature distribution areas of 15 types of target elements, such as flower pattern elements, wheel structure elements, and multi-branch structure elements, avoiding the initial cluster centers being concentrated in the feature areas of a certain type of target element.

[0062] A second loss function, L2, is defined to construct an optimization objective that maximizes the similarity of samples within a cluster. Minimizing the loss value helps to concentrate samples within each cluster. Using the sum of squared Euclidean distances from all samples within a cluster to the cluster center as the core, the formula for the second loss function is as follows:

[0063]

[0064] in, The preset number of clusters is fixed at 15. For the first Each cluster contains a set of all dimension-reduced image samples; For the first The th cluster A dimensionality-reduced image sample, i.e., a 2D feature vector ( ); For the first The center of each cluster, i.e., the 2D feature vector ( , ); sample To the cluster center The squared Euclidean distance. By minimizing the second loss function L2, the intangible cultural heritage image samples within each cluster can be clustered around the cluster center as much as possible, achieving the clustering goal of small intra-cluster differences and large inter-cluster differences.

[0065] The cluster center is iteratively updated and the termination condition is verified. The cluster centers are optimized by gradient descent, and the consistency within the cluster is verified by cosine similarity. Finally, stable cluster labels are output. The specific process is as follows: Iterative update of cluster centers using gradient descent: Calculate the gradient with respect to the second loss function L2 with respect to the cluster centers. Taking the partial derivative and setting it to zero, we obtain the optimal update formula for the cluster centers:

[0066] in, This represents the number of iterations. For the first Round The number of samples in each cluster; essentially, it is to take the mean of the 2D coordinates of all samples in the cluster as the new cluster center, which is the solution that minimizes the sum of squares loss within the cluster.

[0067] Iterative updates, for the first Wheel, based on the current cluster center Recalculate the distances from all samples to the 15 cluster centers, assign each sample to the nearest cluster, and update the cluster set. And calculate the new cluster center according to the above formula. Calculate the new loss value. ,like If the loss value stabilizes, then proceed to consistency verification; otherwise, repeat the iteration and calculation of the new loss value until the maximum number of iterations T=300 is reached.

[0068] Perform intra-cluster consistency verification: for each cluster Calculate the cosine similarity between all pairs of samples within a cluster, ensuring that all pairwise similarities are ≥ a preset threshold θ = 0.85; perform cosine similarity calculation for the cluster. Any two samples and The similarity formula is as follows:

[0069] The similarity value ranges from [0,1]. The closer the value is to 1, the more similar the target element features of the two samples are.

[0070] Perform verification and iterative adjustments, if a certain cluster If there are samples with a pairwise similarity of <0.85, it indicates that different target elements are mixed in the cluster, such as samples of multi-branch structure elements mixed with samples of flower pattern elements, and it is necessary to continue iterating; if all 15 clusters satisfy the condition that the similarity of all pairs of samples in the cluster is ≥0.85, then the clustering converges and the iteration stops.

[0071] Output cluster labels, assigning a unique label (0-14) to each cluster, and setting the labels of all intangible cultural heritage image samples within the cluster as the cluster label. For example: cluster This is a multi-branch structural element with a label of 0; cluster The element is a floral pattern, labeled 1; cluster The elements are arranged in a wheel-like structure with 14 labels; the final output is an M×1 label array, with each intangible cultural heritage image corresponding to a cluster label.

[0072] Step S106: Randomly extract the dimensionality-reduced image from the cluster labels, compare it with the intangible cultural heritage image, confirm the name corresponding to the cluster label, generate a mapping table between the label and the name, and obtain the intangible cultural heritage labeled image.

[0073] Specifically, dimensionality-reduced images are randomly selected from the cluster labels, compared with intangible cultural heritage images, the names corresponding to the cluster labels are confirmed, and a mapping table between labels and names is generated to obtain the labeled image set. The specific steps are as follows: Dimensionality-reduced images are randomly selected from the cluster labels, compared with intangible cultural heritage images, the names corresponding to the cluster labels are confirmed, and a mapping table between labels and names is generated to obtain the labeled image set.

[0074] A target detection model is trained based on annotated images of intangible cultural heritage (ICH) and then processed using this model to output the target detection results. Specifically, random sampling by cluster is performed. Fifteen cluster labels (0-14) are traversed, and for each label, samples are extracted from the dimensionality-reduced images within the corresponding cluster using a random number generator. If the cluster corresponding to label 0 contains 450 dimensionality-reduced images, 45 images are extracted at a rate of 10%. If the cluster corresponding to label 1 contains 32 images, all 32 images are extracted. A total of approximately 500-600 samples are ultimately extracted, covering the different forms of the 15 target elements. After sampling, a "sample list" is generated, containing information such as cluster labels, dimensionality-reduced image indexes, original image paths, and sample numbers within clusters, facilitating subsequent traceability.

[0075] The dimensionality-reduced images are compared and associated with the original intangible cultural heritage images. Based on the index of the dimensionality-reduced images of the sampled samples, the corresponding original intangible cultural heritage images are retrieved in batches; a one-to-one correspondence between "sampled samples and original images" is established, and a visual comparison page is generated, which can support left and right split-screen display: the left side is the 2D coordinate scatter points of the dimensionality-reduced image, and the right side is the area where the target element of the corresponding original image is located;

[0076] Experts confirm the names of the target elements corresponding to the cluster labels. Experts conduct reviews, providing the review panel with a visual comparison page and an electronic version of the Intangible Cultural Heritage Target Element Knowledge Base, confirming the standard names of the target elements in each sampled image. Each expert reviews the images and, based on the feature descriptions in the knowledge base, labels each sample with the target element name, such as "floral pattern element," "wheel-shaped structure element," or "multi-branch structure element," and provides a confidence score (0-100 points). For all sampled images under the same cluster label, the expert labeling results are statistically analyzed. If the vote rate for a certain name is ≥60%, the target element name corresponding to that label is preliminarily determined. If multiple experts label a sample with different names, such as "multi-branch structure element" and "variant multi-branch structure element," the details of the target element in the original image of that sample, such as head shape and handle structure, are extracted. Combined with the morphological difference comparison chart in the knowledge base, experts are organized for centralized review. If more than 20% of the samples under a certain cluster label have discrepancies, the process returns to step S105 to re-optimize the clustering, such as adjusting the cosine similarity threshold and iterating the cluster centers again to ensure the purity of the target elements within the cluster.

[0077] Mapping table verification and calibration. Validation set extraction: Five additional samples not participating in expert review are extracted from each of the 15 clusters to form the validation set, containing 75 samples. The validation set samples are then automatically matched to target element names based on cluster labels using the initially generated "label-name" mapping relationship. The review team only examines the original images of the validation set samples and the automatically matched names to determine if the match is correct. If the verification accuracy is ≥98%, the mapping table takes effect directly. If the accuracy is <98%, an incorrect label is identified, and 10-15 additional samples from that cluster are added for re-review. The mapping relationship is then corrected and validated again until the threshold is met.

[0078] The final mapping table adopts a structured format (CSV / JSON); the mapping table is also stored as a callable configuration file, such as label_name_mapping.json, which can be directly called during subsequent automatic labeling to achieve a fast conversion from "cluster label" to "target element name".

[0079] Output an annotated image set. Based on the label-name mapping table, traverse all original intangible cultural heritage images, including sampled and unsampled samples, and replace the cluster label corresponding to each image with the specific target element name to complete the annotation of the intangible cultural heritage images; integrate all annotated images to form an annotated image set, which includes image files and annotation information, such as target element names and target element boundary information.

[0080] Step S107: Train a target detection model based on the intangible cultural heritage labeled images, and process the intangible cultural heritage objects to be detected using the target detection model to output the target detection results. Specifically, the above annotation method is used to annotate intangible cultural heritage (ICH) images, resulting in annotated ICH images. These annotated images are the fully annotated ICH images. An object detection model is trained using these annotated images. The core architecture of the target detection model consists of a sampler and a detector. The sampler's core function is to downsample the features of the annotated images at multiple scales, gradually improving the semantic expressive power of the features. For example, it captures more complex target element shapes and texture information, providing high-quality multi-scale features for subsequent classification and localization by the detector. The detector, based on the feature map output by the sampler, completes the classification prediction of the target element category and the bounding box prediction of its location. The sampler contains four sampling modules, arranged in a series. Through sampling rules that double the number of channels and halve the spatial dimension, it gradually achieves feature downsampling and semantic enhancement. The core of each module uses a combination of a convolutional layer with a stride of 2, batch normalization, and an activation function.

[0081] In one possible implementation, step S107 further includes: converting the intangible cultural heritage annotation image into an initial feature map; sampling the initial feature map using a first sampling module to double the number of channels and halve the spatial dimension to form a first sampled map; sampling the first sampled map using a second sampling module to double the number of channels and halve the spatial dimension to form a second sampled map; sampling the second sampled map using a third sampling module to double the number of channels and halve the spatial dimension to form a third sampled map; sampling the third sampled map using a fourth sampling module to double the number of channels and halve the spatial dimension to form a fourth sampled map; enhancing the spatial feature localization of the second, third, and fourth sampled maps; uniformly converting the channels of the second, third, and fourth sampled maps to the same dimension to form a first feature, a second feature, and a third feature; fusing the third feature with the second feature to obtain a fourth feature; fusing the fourth feature with the first feature to obtain a fifth feature; downsampling the third feature to obtain a sixth feature; and downsampling the sixth feature to obtain a seventh feature.

[0082] Specifically, the intangible cultural heritage (ICH) labeled image is transformed into an initial feature map. An 800×640×3 ICH labeled image is input into the feature transformation layer. The low-level texture features, such as the edges of target elements and basic color gradients, are extracted through convolution operations. After batch normalization and ReLU activation function, the initial feature map is output with a dimension of 800×640×64, that is, the spatial dimension is maintained at 800×640, and the number of channels is increased from 3 to 64, thus completing the transformation of pixel information into feature vectors.

[0083] The first sampling module samples and generates the first sampled image. The first sampling module adopts a 3×3 convolutional layer with a stride of 2 + batch normalization + ReLU activation structure, with the number of convolutional kernels set to 128. The initial feature map with dimensions of 800×640×64 is input into the first sampling module. The convolution operation with a stride of 2 halves the spatial dimension (width × height) from 800×640 to 400×320, and the 128 convolutional kernels double the number of channels from 64 to 128. After batch normalization and ReLU activation, the first sampled image is formed with dimensions of 400×320×128. At this time, the feature map can capture the local small patterns of the target element, such as the spoke outline of a wheel structure.

[0084] The second sampling module samples and generates a second sampling image. Consistent with the structure of the first sampling module, it employs a 3×3 convolutional layer with a stride of 2, batch normalization, and ReLU activation, with 256 convolutional kernels. The first sampling image, with an input dimension of 400×320×128, is halved in size to 200×160 by the 2-stride convolution, and the 256 convolutional kernels double the number of channels from 128 to 256. After activation and normalization, the second sampling image is formed, with dimensions of 200×160×256. The feature map begins to focus on local pattern combinations of target elements, such as the petal-like structure of a flower.

[0085] The third sampling module samples and generates the third sampled image. Continuing the unified structure of the first two sampling modules, a 3×3 convolutional layer with a stride of 2 + batch normalization + ReLU activation is used, with the number of convolutional kernels set to 512. The second sampled image with an input dimension of 200×160×256 is processed by a convolution with a stride of 2, which halves the spatial dimension to 100×80, and the 512 convolutional kernels double the number of channels from 256 to 512. After processing, the third sampled image is formed with a dimension of 100×80×512. At this point, the semantic expressive power of the feature map is significantly improved, and it can accurately capture the medium-scale semantic features of the target elements, such as the combination of handle and hammer head in a multi-branched object pattern.

[0086] The fourth sampling module samples and generates a fourth sampling map. Maintaining structural consistency, a 3×3 convolutional layer with a stride of 2, batch normalization, and ReLU activation are used, with the number of convolutional kernels set to 1024. The input dimension of the third sampling map is 100×80×512. A convolution with a stride of 2 halves the spatial dimension to 50×40, and 1024 convolutional kernels double the number of channels from 512 to 1024. After activation and normalization, the fourth sampling map is formed, with dimensions of 50×40×1024. This feature map focuses on the high-order semantic features of the target elements.

[0087] In one possible implementation, step S107 further includes: reducing the dimensionality of the second, third, and fourth sampled images into single-channel feature maps; performing horizontal average pooling on each row of pixel values ​​in all single-channel feature maps to obtain a first encoded feature; performing vertical average pooling on each column of pixel values ​​in all single-channel feature maps to obtain a second encoded feature; concatenating the first and second encoded features spatially to form a concatenated feature; expanding the channels of the concatenated feature and performing batch normalization and activation processing to obtain a spatial attention feature vector; segmenting the spatial attention feature vector to perform high-dimensional feature splitting to form vertical sub-features and horizontal sub-features; performing Sigmoid activation processing on the vertical and horizontal sub-features respectively to obtain vertical attention weights and horizontal attention weights; multiplying the vertical attention weights and horizontal attention weights to obtain spatial attention weights; and processing the second, third, and fourth sampled images sequentially using the spatial attention weights to complete the enhancement of spatial feature localization.

[0088] Specifically, horizontal average pooling is used to extract the first encoded feature. The input second sampled image (200×160×256), third sampled image (100×80×512), and fourth sampled image (50×40×1024) are all processed as single-channel feature maps. For example, the second sampled image is split into 256 single-channel images of 200×160 each. Horizontal global average pooling is used, with the pooling kernel size set to 1 × the width of the single-channel image. For example, the second sampled image has a single-channel width of 160, a kernel size of 1×160, a stride of 1, and no padding, ensuring that only one mean is output for each row of pixels. The average value is calculated for each row of pixels in each single-channel feature map, compressing the 2D single-channel image (e.g., 200×160) into a 1D vector (200×1). All single-channel 1D vectors are concatenated according to the channel dimension to obtain the first encoded feature. Taking the second sampled image as an example, the output dimension is 200×256, i.e., row dimension 200 and channel dimension 256. Its core function is to extract global spatial features in the horizontal direction and capture the positional distribution pattern of target elements in the horizontal direction.

[0089] Vertical average pooling is used to extract the second encoded feature. Symmetrical to horizontal pooling, vertical global average pooling is employed, with the pooling kernel size set to the height of the single-channel image × 1. For example, in the second sampled image, the single-channel height is 200, the kernel size is 200×1, the stride is 1, and there is no padding. The average value is calculated for each column of pixels in each single-channel feature map, compressing the 2D single-channel image (e.g., 200×160) into a 1D vector (1×160). All single-channel 1D vectors are concatenated according to channel dimensions to obtain the second encoded feature. Taking the second sampled image as an example, the output dimension is 160×256, i.e., column dimension 160 and channel dimension 256. Its core function is to extract global spatial features in the vertical direction, complementing the first encoded feature and covering the complete spatial dimension.

[0090] Spatial dimension splicing generates spliced ​​features that integrate global spatial information in both horizontal and vertical directions. Based on spatial dimension splicing, the height dimension of the first encoded feature (200×256) and the second encoded feature (160×256) are superimposed, while the channel dimension remains at 256. Taking the second sampled image as an example, after splicing the 200×256 first encoding and the 160×256 second encoding, a spliced ​​feature of (200+160)×256=360×256 is formed; the third / fourth sampled images are similar, i.e., the third sampled image: 100×512+80×512=180×512; the fourth sampled image: 50×1024+40×1024=90×1024.

[0091] The spatial attention feature vector is obtained through channel expansion, normalization, and activation. A 1×1 convolutional layer is used for channel matching, with the number of convolutional kernels matching the number of channels in the sampled image to be enhanced (256 for the second sampled image, 512 for the third, and 1024 for the fourth). Taking the second sampled image as an example, the 360×256 concatenated features are processed by a 1×1 convolution (256 kernels) to output a 360×256 feature map, completing channel dimension adaptation. Batch normalization with a mean of 0 and a variance of 1 is applied to the convolutional output to reduce the interference of numerical fluctuations on attention generation. The ReLU activation function is used to introduce non-linearity, enhancing the expressive power of the attention features, ultimately yielding the spatial attention feature vector of the 360×256 second sampled image.

[0092] By segmenting the feature vector, vertical and horizontal sub-features are generated. The feature vector is then split in reverse along the original pooling dimension, dividing its height dimension into two parts, corresponding to the vertical and horizontal attention bases respectively. Taking the second sampling image as an example, the 360×256 feature vector is split into a vertical sub-feature of 200×256 and a horizontal sub-feature of 160×256. The third and fourth sampling images are split similarly: third sampling image: 180×512 → 100×512 + 80×512; fourth sampling image: 90×1024 → 50×1024 + 40×1024. The fused spatial features are then split into vertical and horizontal dimensions, generating corresponding attention weights for each direction, achieving precise spatial localization.

[0093] Vertical and horizontal attention weights are obtained through Sigmoid activation. Sigmoid activation is applied to the vertical sub-features (200×256) and horizontal sub-features (160×256) respectively, mapping the feature values ​​to the (0,1) interval. Values ​​closer to 1 indicate greater importance of the feature at that spatial location. To accommodate subsequent element-wise multiplication with the sampled image, the activated weights are reshaped: the vertical attention weight (200×256) is reshaped to (200×1×256), and the horizontal attention weight (160×256) is reshaped to (1×160×256). That is, in the third sampled image: 100×512 → 100×1×512, 80×512 → 1×80×512; the same applies to the fourth sampled image. Finally, the vertical and horizontal attention weights are obtained, which can be directly applied to the sampled image. Their core function is to mark the key positions of target elements in the vertical and horizontal directions and suppress background features.

[0094] The weights are multiplied to generate spatial attention weights. The vertical attention weights (200×1×256) and horizontal attention weights (1×160×256) are multiplied element-wise to obtain spatial attention weights that are completely consistent with the spatial and channel dimensions of the original sampled image. Taking the second sampled image as an example, the output generates a spatial attention weight of 200×160×256; the third sampled image outputs 100×80×512, and the fourth sampled image outputs 50×40×1024. In the weight matrix, the values ​​in the target element's region are close to 1, while the background region is close to 0, achieving precise selection of spatial features.

[0095] Attention weight enhancement sampling map. Spatial attention weights are multiplied element-wise with the corresponding second, third, and fourth sampling maps. Positions with higher weights show more pronounced feature enhancement; positions with lower weights (background) show suppressed features. The second sampling map (200×160×256) is multiplied with its corresponding weight (200×160×256) to output the enhanced second sampling map. The spatial features of the target element region, such as the positional combination of flower petals and the morphological boundaries of multi-branch structural elements, are more prominent, while background noise is reduced. The enhanced sampling map significantly improves the accuracy of spatial feature localization.

[0096] In one possible implementation, step S107 further includes: concatenating the third, fourth, fifth, sixth, and seventh features by channels and dimensions to obtain a detection feature map; and performing classification prediction and localization prediction on the detection feature map using a detector to obtain classification results and bounding box prediction results.

[0097] Before feature fusion, the channel dimensions of the second, third, and fourth sampling images need to be unified to the same number of channels. Considering semantic expressiveness and computational efficiency, a target channel number of 512 is selected. Channel mapping can be achieved using a 1×1 convolutional layer, combined with batch normalization and ReLU activation. The specific configuration is as follows: Second sampling image (200×160×256): 1×1 convolutional kernel number = 512, input 256 channels → output 512 channels, spatial dimension remains 200×160, generating the first feature (200×160×512); Third sampling image (100×80×512): the number of channels is already... The kernel size is 512. Feature calibration is performed directly through 1×1 convolution (kernel number = 512) to generate the second feature (100×80×512). The fourth sampling map (50×40×1024): 1×1 convolution kernel number = 512, input 1024 channels → output 512 channels, spatial dimension is maintained at 50×40, generating the third feature (50×40×512). Among them, 1×1 convolution only changes the number of channels and does not destroy the spatial feature distribution. At the same time, the non-linear expression of the feature is enhanced by the activation function.

[0098] The third feature is fused with the second feature to generate the fourth feature. During fusion, the spatial dimensions need to be unified. The second feature (100×80×512) has twice the spatial dimension of the third feature (50×40×512), so it needs to be downsampled. Downsampling uses a 3×3 convolutional layer (stride=2, padding=1), maintaining 512 channels, combined with batch normalization and ReLU activation, downsampling the second feature from 100×80×512 to 50×40×512, aligning it with the spatial dimension of the third feature. The downsampled second feature is then added element-wise to the third feature to generate the fourth feature (50×40×512), which can fuse medium-scale semantic features (the second feature) with high-order semantic features (the third feature), strengthening the correlation between the target element's shape and local details.

[0099] The fourth feature is fused with the first feature to generate the fifth feature. During fusion, the spatial dimensions need to be unified. The spatial dimension of the first feature (200×160×512) is four times that of the fourth feature (50×40×512), requiring two downsampling operations on the first feature. The downsampling configuration involves consecutively using two 3×3 convolutions (stride=2, padding=1) + batch normalization + ReLU module. The first operation transforms 200×160 to 100×80, and the second transforms 100×80 to 50×40, ultimately resulting in a 50×40×512 downsampled version of the first feature. The downsampled first feature is then added element-wise to the fourth feature to generate the fifth feature (50×40×512). This fifth feature supplements the underlying spatial features (the first feature contains local pattern combinations), improves the detail capture capability of the fused features, and prevents higher-order features from losing edge information.

[0100] The third feature is downsampled to generate the sixth feature. Downsampling uses a 3×3 convolutional layer with a stride of 2, padding of 1 + batch normalization + ReLU activation, and the number of channels remains at 512. By inputting the third feature (50×40×512), the spatial dimension is halved (50×40→25×20) through a convolution with a stride of 2, while the number of channels remains at 512, generating the sixth feature (25×20×512). This further enhances the semantic concentration of the feature, capturing macroscopic features such as the overall layout of the target element, and forming a multi-scale complementarity with the preceding features.

[0101] The sixth feature is downsampled to generate the seventh feature. To maintain structural uniformity and ensure a smooth decrease in feature scale, the input sixth feature (25×20×512) is processed by a 3×3 convolution (stride of 2, padding of 1), halving the spatial dimension (25×20→12×10) while maintaining 512 channels, generating the seventh feature (12×10×512). This feature can focus on the highest-order semantic features and strengthen the representation of the core shape of the target element, such as the circular shape of a wheel structure or the three-strand shape of a multi-branch structure.

[0102] Multi-feature channels and dimensions are concatenated to generate a detection feature map. Since the spatial dimensions of the five features are inconsistent, low-resolution features need to be upsampled to a baseline of 50×40. Upsampling uses transposed convolution (3×3 kernels, stride 2, padding 1), changing only the spatial dimensions without altering the number of channels: the sixth feature (25×20×512) is upsampled once to 50×40×512; the seventh feature (12×10×512) is upsampled twice to 25×20 to 50×40×512.

[0103] By concatenating the five features after unifying the dimensions, all of which are 50×40×512, the channels are superimposed, resulting in a total of 512×5=2560 channels. Finally, a detection feature map of 50×40×2560 is generated. This feature map integrates multi-scale information from local details (first feature) to macro semantics (seventh feature).

[0104] In one possible implementation, step S107 further includes: acquiring label information of the detection feature map and establishing co-occurrence relationships between different label information to obtain associated features; transforming the associated features into a one-dimensional vector through a flattening operation, and mapping the one-dimensional vector to a new feature space through a linear layer to obtain first spatial features; expanding the first spatial features to a four-dimensional space to obtain second spatial features; extracting classification branch features of the detection feature map through a preset convolutional layer, and converting the classification branch features into weight vectors; and fusing the classification branch features with the second spatial features based on the weight vectors to obtain a classification result.

[0105] Specifically, the classification result obtained by classifying and predicting the detection feature map through the detector includes: Obtain a mapping table of label-target element names in the intangible cultural heritage labeled images. For example, label 0 represents multi-branch structure elements, label 1 represents floral pattern elements, and there are 15 label categories in total. These are encoded as integers from 0 to 14, and each detection feature map corresponds to one unique label. During batch processing, the batch size is set to N, and the input detection feature maps are N×50×40×2560 (dimensions represent: batch size × height × width × number of channels). Each feature map is associated with one label, forming a label vector. ( ∈{0,1,...,14}).

[0106] Construct a co-occurrence matrix, count the simultaneous occurrence frequency of all tags within a batch, and generate a K×K co-occurrence matrix (K=15, representing the total number of tags). Matrix elements... Indicates label and The percentage of times they appear together in the same batch. For example, if there are 30 feature maps in a batch containing multi-branch structural elements (label 0), and 20 of them appear simultaneously with floral pattern elements (label 1), then... =20 / 30≈0.67.

[0107] To avoid the co-occurrence matrix relying solely on statistical information, semantic features are integrated, combining the co-occurrence matrix with the global semantic features of the detection feature maps. For each detection feature map... Perform global average pooling on a (50×40×2560) vector to obtain a global semantic vector of 1×2560. Then, it is mapped to a 1×K label semantic vector through a 1×1 convolutional layer (with 15 kernels). , This indicates the semantic matching degree between the feature map and each label.

[0108] To generate association features, the K×K co-occurrence matrix is ​​concatenated with the label semantic vectors of all samples channel by channel to obtain the final association features. (Dimension meaning: tag) × Tag The semantic matching channel (×) includes both the label association strength and feature semantic matching information.

[0109] Perform a flattening operation and input associated features. The three-dimensional matrix is ​​converted into a one-dimensional vector by flattening. The calculation method is: 15×15×15=3375, and the flattened vector is output. This can transform spatial features into linearly processable vector forms. Linear layer mapping and activation are used to design a linear layer, Linear1, with an input dimension of 3375 and an output dimension of 512. The expression is: ;in, The weight matrix is ​​(3375×512). The bias vector (1×512) is initialized to a normal distribution with mean 0 and variance 0.0001. Nonlinear activation is applied, processed using the ReLU activation function, as shown in the formula below. Output the first spatial features By introducing nonlinearity, the problem of linear mapping being unable to fit complex correlated features is solved. At the same time, high-dimensional correlated features are reduced in dimensionality and mapped to a unified feature space, eliminating dimensional redundancy and enhancing the linear separability of features.

[0110] Expand the dimensions and input the first spatial features. The spatial dimension of the detection feature map is 50×40 (height×width). The one-dimensional vector needs to be expanded into a four-dimensional feature (batch size×height×width×channel size) that is consistent with the spatial dimension of the detection feature map. The N×512 vector can be repeated along the spatial dimension (height, width) to obtain the N×50×40×512 four-dimensional feature.

[0111] For example, for a single sample with a 1×512 feature, repeat 50 times along the high dimension and 40 times along the wide dimension to generate a 50×40×512 feature. During batch processing, the N dimension remains unchanged. The output second spatial feature is Feature_second=N×50×40×512, with the dimensions perfectly aligned with the features of subsequent classification branches.

[0112] The classification branch features are extracted, and the detection feature map is Feature_det=N×50×40×2560 with 2560 channels, containing multi-scale semantic features. The classification branch features are extracted through a 1×1 convolutional layer with 512 kernels, a stride of 1, and no padding. Batch normalization and ReLU activation are used to output the classification branch features Feature_cls=N×50×40×512, retaining only the semantic features related to classification, such as the core shape and texture features of the target element, which can filter out redundant information related to localization.

[0113] The features are converted into weight vectors and subjected to global average pooling. Global spatial pooling is then performed on the classification branch features (Feature_cls) with a 50×40 kernel, compressing the N×50×40×512 vector into N×1×1×512, resulting in the global classification features. Flattening and activation are then performed, resulting in an N×512 vector. The Sigmoid activation function maps the feature values ​​to the (0,1) interval, generating a weight vector of weight = N×512. Each element of the weight vector corresponds to a channel weight of the classification branch feature. The closer the value is to 1, the more important the classification feature of that channel is. For example, the contour channel weight of the target element approaches 1, while the background channel weight approaches 0, thus highlighting important local features and suppressing background noise.

[0114] Weight adaptation is performed by using the `reshape` function to transform the weight vector `Weight = N×512` into `N×1×1×512`, adapting it to the spatial dimensions of the classification branch feature `Feature_cls` (N×50×40×512) and the second spatial feature `Feature_second` (N×50×40×512). A weighted and summed approach is adopted, combining weighted local features and global correlation features, using the formula: `Feature_fuse = (Feature_cls × Weight) + Feature_second`, outputting the fused feature `Feature_fuse = N×50×40×512`. After weighting, the classification branch features highlight key local classification features; the sum of the classification branch features and the second spatial features incorporates global label correlation information, avoiding overfitting of local features.

[0115] Global feature extraction and classification prediction are performed, including global feature compression and global average pooling of Feature_fuse to obtain N×512 globally fused features. This eliminates spatial dimensional redundancy while retaining core classification information. A linear mapping is then implemented using a Linear2 layer with an input dimension of 512 and an output dimension of 15, as shown in the formula. ,in, It is 512×15, The value is 1×15. Probability normalization is performed, and the Logits are transformed into a class probability distribution using the Softmax activation function. ,in, The label corresponding to the highest probability is taken as the final classification result, i.e., Result = argmax(Prob). For example, if... If 0.92 is the maximum value, then the classification result is "flower shape" corresponding to label 1.

[0116] In one possible implementation, step S107 further includes: acquiring boundary information of the intangible cultural heritage processing object in the detection feature map; setting multiple anchor boxes near the intangible cultural heritage processing object based on the boundary information; selecting and defining candidate anchor boxes among the multiple anchor boxes; the center point of the candidate anchor box is within the ground truth box; the ground truth box is determined based on the boundary information of the intangible cultural heritage processing object; calculating the first intersection-union ratio between the candidate anchor box and the ground truth box, and establishing an alignment matrix between the first intersection-union ratio and the classification result; selecting candidate anchor boxes whose alignment matrix value is greater than a preset matrix value as positive samples.

[0117] Specifically, based on the annotation information in the intangible cultural heritage annotated image, the boundary information of each target element in the detection feature map is obtained, in the following format: ,in, The coordinates are the top-left corner coordinates of the target element in the original image (800×640 pixels). The coordinates are the bottom right corner, in pixels; the detection feature map is N×50×40×2560, where N is the batch size, and each detection feature map can be associated with the boundaries of multiple target elements.

[0118] The baseline scale of the anchor boxes on the detection feature map is set to [8, 16, 32], corresponding to the original image scales of 8×16=128, 16×16=256, and 32×16=512 pixels; the aspect ratio is set to [1:1, 1:2, 2:1] to ensure that the anchor boxes can match the shape features of different target elements. For example, floral pattern elements are close to circles, multi-branch structure elements are elongated, and wheel-shaped structure elements are circular. Each baseline scale is paired with 3 aspect ratios, generating a total of 9 basic anchor boxes. The specific generation method is as follows: The anchor boxes are generated with the center of the target element boundary B as the reference scale. Based on the center coordinates mapped onto the detection feature map ,in, =16 is the downsampling rate; the anchor frame center must be within the target element boundary mapping area. Within the range, The width of the target element boundary on the detection feature map To avoid the anchor boxes deviating too far from the target element boundary, for each basic anchor box, 10 variants are generated by center fine-tuning and scale scaling. The fine-tuning step size is 1 pixel on the detection feature map, and the scale scaling factors are 0.8, 1.0, and 1.2. Finally, 9×10=90 anchor boxes are generated for the boundary of a single target element, covering multiple locations and scales of the scene near the target element.

[0119] Candidate anchor boxes with their center points within the boundaries of the target element are selected. The center points of the anchor boxes are calculated, and the coordinates of each anchor box on the original image are... Its center point coordinates are ;Mapped to the center point of the detection feature map Filter candidate anchor boxes; the boundary of the target element on the original image is [range missing]. , Determine the center point of the anchor frame ( , Does it satisfy: ,and .

[0120] Anchor boxes that meet the criteria are candidate anchor boxes. Based on the target element boundary and the range of anchor box generation, 30-50 candidate anchor boxes can be retained after each batch of filtering, while invalid anchor boxes whose center point is completely outside the target element boundary are filtered out.

[0121] Calculate the Intersection over Union (IoU) and establish an alignment matrix. IoU is a measure of the overlap between the candidate anchor box and the target element's boundary, with the formula: IoU(A,B)=Area(A∩B) / Area(A∪B), where Area(A∩B) is the intersection area of ​​the two boxes, and Area(A∪B) is the union area of ​​the two boxes, i.e., Area(A∪B)=Area(A)+Area(B)-Area(A∩B). The specific calculation is as follows:

[0122] Intersection coordinates: , ;like or If so, Area(A∩B)=0, IoU=0; otherwise Each candidate anchor box corresponds to one IoU value, ranging from [0,1]. The closer the value is to 1, the higher the position matching degree.

[0123] Construct the alignment matrix. Let the number of candidate anchor boxes be M, the number of target element categories be K=15, and the alignment matrix Align be M×K in dimensions. ( ,in, For the m-th candidate anchor box, Let be the classification probability of the target element of class k corresponding to the m-th anchor box.

[0124] Positive samples are selected based on the alignment matrix. A preset matrix threshold τ = 0.5 is used for each candidate anchor box. Take its maximum value in the alignment matrix. =max(Align[m][0],Align[m][1],...,Align[m]

[14] ); if Max >τ, then It is determined to be a positive sample, and its best matching category is recorded. =argmax(Align[m][k]). The number of positive samples in each batch is controlled between 5 and 10 to ensure that each positive sample highly overlaps with the ground truth bounding box and highly matches the category.

[0125] In one possible implementation, step S107 further includes: reducing the length and width of each anchor frame by the same proportion to obtain auxiliary anchor frames; calculating the second intersection-union ratio between each auxiliary anchor frame and the ground truth frame; if it is confirmed that the second intersection-union ratio is greater than 0 and the corresponding anchor frame is not classified as a positive sample, then the corresponding anchor frame is classified as an edge sample; extracting features from the positive samples and edge samples to obtain the bounding box prediction result.

[0126] Specifically, an auxiliary anchor frame is generated. The reduction ratio is set to r = 0.8, and the center point of the auxiliary anchor frame is parallel to the center point of the original anchor frame. , Maintain consistency; the width of the original anchor frame ,high After reduction, the width w' = w × r and the height h' = h × r.

[0127] Update coordinates, auxiliary anchor frame coordinates are A'=( -w' / 2, -h' / 2, +w' / 2, +h' / 2), ensuring the center remains unchanged and the size is reduced proportionally.

[0128] Filter edge samples. Set edge sample filtering conditions: Condition 1, the IoU between the auxiliary anchor box and the ground truth box is >0, indicating that the auxiliary anchor box and the ground truth box overlap and are positionally related; Condition 2, the corresponding original anchor box is not judged as a positive sample, which can avoid duplication with positive samples.

[0129] Output edge samples by iterating through all auxiliary anchor boxes. Anchor boxes that satisfy the above two conditions are the corresponding original anchor boxes, which are the edge samples. The number of edge samples in each batch is 2-3 times that of positive samples. For example, if there are 5 positive samples, then there should be 10-15 edge samples. This ensures sample balance and avoids introducing too many noisy samples. Extract sample features and output the bounding box prediction results, as follows:

[0130] Sample feature extraction is performed by mapping the coordinates of the anchor boxes (positive samples and edge samples) on the original image to the detection feature map (50×40×2560) at a sampling rate of s=16, thus obtaining the coordinates of the corresponding regions on the feature map. The ROIPooling operation is performed. Since different anchor boxes map to different regions of varying sizes in the feature map, ROIPooling pools all sample regions to a fixed size of 7×7×2560 to ensure uniform input dimensions for subsequent networks. Each sample outputs a fixed-size feature vector of 7×7×2560, fusing multi-scale semantic features to achieve a comprehensive understanding from local details to macroscopic morphology.

[0131] The design incorporates a regression branch structure consisting of two linear layers: The first linear layer takes a 7×7×2560=125440-dimensional feature vector as input and outputs a 256-dimensional vector, activated by ReLU to introduce non-linearity and enhance feature representation; the second linear layer takes the 256-dimensional vector as input and outputs four regression parameters. These correspond to the offset of the bounding box center coordinates and the scaling factor of the width and height, respectively.

[0132] Calculate the bounding box coordinates and predict the center x-coordinate: Predicted center y-coordinate: Prediction width: High prediction: ;in,( , Let be the center coordinates of the original anchor frame, and w and h be the width and height of the original anchor frame. This forms the final bounding box. ), which restores the pixel coordinates of the original image (800×640).

[0133] Redundant boxes are removed using non-maximum suppression. For all predicted bounding boxes in the same batch, including those corresponding to positive and edge samples, they are sorted by classification probability, and the box with the highest probability is retained, while redundant boxes with an IoU > 0.3 are suppressed. For each intangible cultural heritage image, only one bounding box with the highest confidence and no redundancy is retained as the final bounding box prediction result, with the format consistent with the target element boundary. .

[0134] In one possible implementation, step S107 further includes: optimizing the parameters of the sampler and detector based on the classification results and bounding box prediction results.

[0135] Based on the classification and bounding box prediction results, the sampler and detector parameters are optimized. This is mainly achieved by backpropagating the error between the classification and bounding box prediction results to update the learnable parameters of the sampler and detector, minimizing the deviation between the predicted and true values, and improving the model's generalization ability.

[0136] Construct a joint loss function, using the cross-entropy loss function, with the following formula:

[0137] in, For classifying losses, This is a one-hot encoding of the actual tags, such as multi-branch structural elements corresponding to [1,0,...,0]. To detect the probability distribution of feature map categories, k∈{0,1,...,14}.

[0138] Location loss Using Smooth Loss, the formula is: Smooth (x) = 0.5x², |x| ≤ 1, |x| - 0.5, |x| > 1, where x is the error of the bounding box regression parameters, i.e., the predicted regression parameters. With true regression parameters The difference. Calculation of true regression parameters: based on the true bounding box. With anchor frame The true regression parameters are: , , , ,in, , , , .

[0139] Joint loss and weight allocation: The total loss is the weighted sum of the classification loss and the localization loss, as shown in the formula: We set weights α=1.0 and β=2.0 to ensure balanced optimization of the two types of losses.

[0140] Parameter gradient calculation and backpropagation employ an automatic differentiation framework, such as PyTorch's `backward()`, based on the chain rule. Backpropagation is used to calculate the gradient ∇θ for each learnable parameter, where θ is any learnable parameter. Gradients from the batch normalization layer are clipped, with the absolute value of the gradient ≤ 5 to avoid gradient explosion. Gradient descent iteratively updates the parameters with a learning rate η = 500 and a momentum coefficient α = 0.9. The parameter update formula is: θ^(t+1) = θ^(t) - η × ∇θ + α × (θ^(t) - θ^(t-1)), where θ^(t) is the parameter in the t-th round, and θ^(t) - θ^(t-1) is the update amount from the previous round. Momentum is introduced to alleviate the local optimum problem.

[0141] Iterative constraints and stopping conditions are implemented, normalization constraints are applied, and the convolutional layer weights are adjusted after each parameter update. Normalization (||θ||²=1) ensures numerical stability; stopping condition: when the number of iterations reaches 500, or after 30 consecutive iterations. Change < Stop updating parameters to avoid overfitting.

[0142] The system uses object detection to infer the nature of intangible cultural heritage (ICH) images and outputs object detection results. These results accurately identify the category name, spatial location boundaries, and corresponding confidence information of the ICH objects being processed. The ICH objects are the target elements within the ICH image.

[0143] Each output image corresponds to one detection result, containing three core pieces of information: target element category name; bounding box coordinates; and format (...). , , , (), in pixels, for easy visualization and annotation; confidence: the maximum value in the classification probability distribution, reflecting the model's confidence in the detection results.

[0144] The output format supports two structured formats. For example, JSON format, {"image_path":"xxx.jpg","category":"flower shape","bbox":[120,80,320,280],"confidence":0.92}; CSV format, image path, category name, top left x, top left y, bottom right x, bottom right y, confidence score.

[0145] Please refer to Figure 2 This illustration shows a schematic diagram of a target detection system for intangible cultural heritage images provided in an embodiment of this application. The system includes an acquisition module 21 and a processing module 22, wherein... The acquisition module 21 is used to collect images of intangible cultural heritage (ICH) and preprocess them to obtain preprocessed images. A feature extractor is used to extract high-order semantic features from the preprocessed images. Based on these high-order semantic features, direct and indirect similarities between different preprocessed images are calculated, and the direct and indirect similarities are averaged to obtain joint similarity. Based on the joint similarity, a first loss function is defined, and the loss value is minimized through gradient descent to map the high-order semantic features of the preprocessed images to a lower dimension, resulting in a dimensionality-reduced image. Clustering is performed on the dimensionality-reduced image to generate cluster labels. Dimensionality-reduced images from the cluster labels are randomly selected, compared with ICH images, and the names corresponding to the cluster labels are confirmed. A mapping table between labels and names is generated to obtain ICH-labeled images.

[0146] The processing module 22 is used to train a target detection model based on the intangible cultural heritage labeled images, and to process the intangible cultural heritage objects to be detected through the target detection model, and output the target detection results.

[0147] In one possible implementation, the acquisition module 21 is used to preprocess the intangible cultural heritage image to obtain a preprocessed image, specifically including: scaling the intangible cultural heritage image to a specified size using a proportional scaling formula; filling the edges of the scaled intangible cultural heritage image so that the length and width of the filled intangible cultural heritage image can be divided by a preset parameter to form a filled image; randomly flipping the filled image with a preset flip probability to form a flipped image; and performing RGB normalization processing on the flipped image to obtain a preprocessed image.

[0148] In one possible implementation, the feature extractor includes a convolutional layer, a first residual group, a second residual group, and a third residual group. The acquisition module 21 is used to extract high-order semantic features of the preprocessed image using the feature extractor, specifically including: extracting low-level features of the preprocessed image through the convolutional layer; extracting local features of the preprocessed image through the first residual group based on the low-level features; extracting medium-scale semantic features of the preprocessed image through the second residual group based on the local features; and extracting high-order semantic features of the preprocessed image through the third residual group based on the medium-scale semantic features.

[0149] In one possible implementation, the acquisition module 21 is used to perform clustering operations on the dimensionality-reduced image to generate cluster labels, specifically including: acquiring the target dimensionality-reduced image as the cluster center; randomly selecting the target dimensionality-reduced image from the dimensionality-reduced images; calculating the minimum distance from the unselected dimensionality-reduced images to the selected cluster centers to form clusters; selecting the next cluster center in order of increasing distance until the number of cluster centers meets the preset selection requirement; calculating the sum of squared distances based on the distance from the dimensionality-reduced image to each cluster center, and defining a second loss function based on the sum of squared distances; iteratively updating the selected cluster centers along the gradient direction of the second loss function until the cosine value between each pair of dimensionality-reduced images in each cluster is greater than a preset similarity value; outputting the target cluster centers that meet the iterative update conditions, and assigning corresponding labels to the target cluster centers to generate cluster labels.

[0150] In one possible implementation, the target detection model includes a sampler and a detector. The sampler includes a first sampling module, a second sampling module, a third sampling module, and a fourth sampling module. The processing module 22 is used to train the target detection model based on the intangible cultural heritage (ICH) labeled image, specifically including: converting the ICH labeled image into an initial feature map; sampling the initial feature map through the first sampling module, thereby doubling the number of channels and halving the spatial dimension of the initial feature map to form a first sampled image; sampling the first sampled image through the second sampling module, thereby doubling the number of channels and halving the spatial dimension of the first sampled image to form a second sampled image; sampling the second sampled image through the third sampling module, thereby doubling the number of channels and halving the spatial dimension of the second sampled image to form a third sampled image; and sampling the third sampled image through the fourth sampling module, thereby doubling the number of channels and halving the spatial dimension of the third sampled image. The spatial dimension is halved to form the fourth sampling map; the spatial feature localization of the second, third, and fourth sampling maps is enhanced; the channels of the second, third, and fourth sampling maps are uniformly converted to the same dimension to form the first, second, and third features; the third feature is fused with the second feature to obtain the fourth feature; the fourth feature is fused with the first feature to obtain the fifth feature; the third feature is downsampled to obtain the sixth feature; the sixth feature is downsampled to obtain the seventh feature; the third, fourth, fifth, sixth, and seventh features are concatenated by channel and dimension to obtain the detection feature map; the detector performs classification prediction and localization prediction on the detection feature map to obtain the classification result and bounding box prediction result; based on the classification result and bounding box prediction result, the parameters of the sampler and detector are optimized.

[0151] In one possible implementation, the processing module 22 is used to enhance the spatial feature localization of the second, third, and fourth sampled images, specifically including: reducing the dimensionality of the second, third, and fourth sampled images into single-channel feature images; performing horizontal average pooling on the pixel values ​​of each row in all single-channel feature images to obtain a first encoded feature; performing vertical average pooling on the pixel values ​​of each column in all single-channel feature images to obtain a second encoded feature; concatenating the first and second encoded features spatially to form a concatenated feature; expanding the channels of the concatenated feature and performing batch normalization and activation processing to obtain a spatial attention feature vector; segmenting the spatial attention feature vector to perform high-dimensional feature splitting to form vertical sub-features and horizontal sub-features; performing Sigmoid activation processing on the vertical and horizontal sub-features respectively to obtain vertical attention weights and horizontal attention weights; multiplying the vertical attention weights and horizontal attention weights to obtain spatial attention weights; and processing the second, third, and fourth sampled images sequentially using the spatial attention weights to complete the enhancement of spatial feature localization.

[0152] In one possible implementation, the processing module 22 is used to perform classification prediction on the detection feature map using a detector to obtain a classification result. Specifically, this includes: acquiring the label information of the detection feature map and establishing co-occurrence relationships between different label information to obtain associated features; flattening the associated features into a one-dimensional vector and mapping the one-dimensional vector to a new feature space through a linear layer to obtain a first spatial feature; expanding the first spatial feature to a four-dimensional space to obtain a second spatial feature; extracting the classification branch features of the detection feature map through a preset convolutional layer and converting the classification branch features into a weight vector; and fusing the classification branch features with the second spatial feature based on the weight vector to obtain a classification result.

[0153] In one possible implementation, the processing module 22 is used to perform localization prediction on the detection feature map using a detector, specifically including: acquiring the boundary information of the intangible cultural heritage processing object in the detection feature map; setting multiple anchor boxes near the intangible cultural heritage processing object based on the boundary information; selecting and defining candidate anchor boxes among the multiple anchor boxes; the center point of the candidate anchor box is within the ground truth box; the ground truth box is determined based on the boundary information of the intangible cultural heritage processing object; calculating the first intersection-union ratio (IUU) between the candidate anchor box and the ground truth box, and establishing an alignment matrix between the first IUU and the classification result; selecting candidate anchor boxes whose alignment matrix value is greater than a preset matrix value as positive samples.

[0154] In one possible implementation, the processing module 22 is used to obtain bounding box prediction results through classification prediction, specifically including: reducing the length and width of each anchor box by the same proportion to obtain auxiliary anchor boxes; calculating the second intersection-union ratio between each auxiliary anchor box and the ground truth box; if it is confirmed that the second intersection-union ratio is greater than 0 and the corresponding anchor box is not classified as a positive sample, then classifying the corresponding anchor box as an edge sample; extracting features from the positive samples and edge samples to obtain bounding box prediction results.

[0155] It should be noted that the system provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0156] This application also provides an electronic device. (See reference...) Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: at least one processor 301, at least one communication bus 302, a user interface 303, at least one network interface 304, and a memory 305.

[0157] The communication bus 302 is used to enable communication between these components.

[0158] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0159] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0160] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0161] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. (Refer to...) Figure 3 The memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a target detection application for intangible cultural heritage images.

[0162] exist Figure 3In the illustrated electronic device, the user interface 303 is primarily used to provide an input interface for the user and acquire user input data; while the processor 301 can be used to call the target detection application for intangible cultural heritage images stored in the memory 305. When executed by one or more processors 301, the electronic device performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0163] This application also provides a non-transitory computer-readable storage medium storing instructions. When executed by one or more processors, these instructions cause an electronic device to perform one or more of the methods described in the above embodiments.

[0164] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0165] In the various embodiments provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some service interface, device, or unit, and may be electrical or other forms.

[0166] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0167] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0168] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0169] The above description is merely an exemplary embodiment disclosed in this application and should not be construed as limiting the scope of this application. Any equivalent changes and modifications made in accordance with the teachings of this application shall still fall within the scope of this application.

[0170] This application is intended to cover any variations, uses, or adaptations disclosed herein that follow the general principles disclosed herein and include common knowledge or customary technical means in the art that are not described in this application.

Claims

1. A target detection method for intangible cultural heritage images, characterized in that, The method includes: Images of intangible cultural heritage are collected and preprocessed to obtain preprocessed images; High-order semantic features of the preprocessed image are extracted using a feature extractor. Based on the high-order semantic features of the preprocessed image, the direct similarity and indirect similarity between different preprocessed images are calculated, and the direct and indirect similarities are averaged to obtain the joint similarity. Based on the joint similarity, a first loss function is defined, and the loss value is minimized by gradient descent to map the high-order semantic features of the preprocessed image to a low dimension, thereby obtaining a dimensionality-reduced image. Perform clustering operations on the reduced-dimensional image to generate cluster labels; Randomly select the dimensionality-reduced image from the clustering labels, compare it with the intangible cultural heritage image, confirm the name corresponding to the clustering label, generate a mapping table between the label and the name, and obtain the intangible cultural heritage labeled image; A target detection model is trained based on the intangible cultural heritage labeled images, and the intangible cultural heritage objects to be detected are processed by the target detection model to output the target detection results.

2. The method according to claim 1, characterized in that, The intangible cultural heritage images are preprocessed to obtain preprocessed images, specifically including: The intangible cultural heritage image is scaled to a specified size using a proportional scaling formula; The edges of the scaled intangible cultural heritage image are filled so that the length and width of the filled intangible cultural heritage image can be divided by a preset parameter, thus forming a filled image; The filled image is randomly flipped with a preset flip probability to form a flipped image; The flipped image is then subjected to RGB normalization to obtain the preprocessed image.

3. The method according to claim 1, characterized in that, The feature extractor includes a convolutional layer, a first residual group, a second residual group, and a third residual group. The step of using the feature extractor to extract high-order semantic features from the preprocessed image specifically includes: The low-level features of the preprocessed image are extracted through the convolutional layer; Based on the underlying features, local features of the preprocessed image are extracted through the first residual group; Based on the local features, the intermediate-scale semantic features of the preprocessed image are extracted through the second residual group; Based on the medium-scale semantic features, the high-order semantic features of the preprocessed image are extracted through the third residual group.

4. The method according to claim 1, characterized in that, The step of performing clustering operations on the dimensionality-reduced image to generate cluster labels specifically includes: The target dimensionality-reduced image is obtained as the cluster center; the target dimensionality-reduced image is randomly selected from the dimensionality-reduced images. Calculate the minimum distance from the unselected dimensionality-reduced image to the center of the selected cluster to form a cluster class; Select the next cluster center in order of increasing distance until the number of cluster centers meets the preset selection requirement; The sum of squared distances is calculated based on the distances from the reduced-dimensional image to each cluster center, and a second loss function is defined based on the sum of squared distances. The selected cluster centers are iteratively updated along the gradient direction of the second loss function until the cosine value between each pair of the dimensionality-reduced images in each cluster class is greater than the preset similarity value. Output the target cluster centers that meet the iterative update conditions, and assign corresponding labels to the target cluster centers to generate the cluster labels.

5. The method according to claim 1, characterized in that, The target detection model includes a sampler and a detector. The sampler includes a first sampling module, a second sampling module, a third sampling module, and a fourth sampling module. Training the target detection model based on the intangible cultural heritage labeled image specifically includes: The intangible cultural heritage labeled images are converted into initial feature maps; The initial feature map is sampled by the first sampling module, which doubles the number of channels and halves the spatial dimension of the initial feature map to form a first sampled map. The first sampling image is sampled by the second sampling module, which doubles the number of channels and halves the spatial dimension of the first sampling image to form a second sampling image. The second sampling image is sampled by the third sampling module, which doubles the number of channels and halves the spatial dimension of the second sampling image to form the third sampling image. The third sampling image is sampled by the fourth sampling module, which doubles the number of channels and halves the spatial dimension of the third sampling image to form the fourth sampling image. Enhance the spatial feature localization of the second, third, and fourth sampling maps; The channels of the second, third, and fourth sampling images are uniformly converted to the same dimension to form the first feature, the second feature, and the third feature; The third feature is fused with the second feature to obtain the fourth feature; The fourth feature is fused with the first feature to obtain the fifth feature; The third feature is downsampled to obtain the sixth feature; The sixth feature is downsampled to obtain the seventh feature; The third feature, the fourth feature, the fifth feature, the sixth feature, and the seventh feature are concatenated by channel and dimension to obtain a detection feature map; The detector performs classification and localization prediction on the detected feature map to obtain classification results and bounding box prediction results. Based on the classification results and the bounding box prediction results, the parameters of the sampler and the detector are optimized.

6. The method according to claim 5, characterized in that, The enhancement of spatial feature localization of the second, third, and fourth sampling maps specifically includes: The second sampling image, the third sampling image, and the fourth sampling image are reduced to a single-channel feature image. Perform horizontal average pooling on each row of pixel values ​​in all the single-channel feature maps to obtain the first encoded feature; Vertical average pooling is performed on each column of pixel values ​​in all the single-channel feature maps to obtain the second encoded feature. The first encoded feature and the second encoded feature are concatenated spatially to form a concatenated feature; Expand the channels of the spliced ​​features and perform batch normalization and activation processing to obtain spatial attention feature vectors; The spatial attention feature vector is segmented into high-dimensional feature splitting to form vertical direction sub-features and horizontal direction sub-features; Sigmoid activation is applied to the vertical and horizontal sub-features respectively to obtain vertical attention weights and horizontal attention weights. Multiply the vertical attention weight and the horizontal attention weight to obtain the spatial attention weight; The spatial attention weights are used to process the second, third, and fourth sampling images sequentially to enhance the spatial feature localization.

7. The method according to claim 5, characterized in that, The detector performs classification prediction on the detected feature map to obtain a classification result, specifically including: Obtain the label information of the detected feature map, and establish co-occurrence relationships among different label information to obtain associated features; The associated features are flattened into a one-dimensional vector, and then mapped to a new feature space through a linear layer to obtain the first spatial features. The first spatial feature is expanded into four-dimensional space to obtain the second spatial feature; The classification branch features of the detection feature map are extracted by a preset convolutional layer, and the classification branch features are converted into a weight vector. Based on the weight vector, the classification branch features are fused with the second spatial features to obtain the classification result.

8. The method according to claim 5, characterized in that, The detector performs localization prediction on the detected feature map, specifically including: Obtain the boundary information of the intangible cultural heritage processing objects in the detected feature map; Based on the boundary information, multiple anchor frames are set near the intangible cultural heritage processing object; Candidate anchor frames are selected and defined from among the multiple anchor frames; the center point of each candidate anchor frame is within the actual frame; the actual frame is determined based on the boundary information of the intangible cultural heritage processing object. Calculate the first intersection-union ratio (IU) between the candidate anchor boxes and the ground truth boxes, and establish an alignment matrix between the first IU and the classification result; Candidate anchor boxes whose alignment matrix values ​​are greater than a preset matrix value are selected as positive samples.

9. The method according to claim 8, characterized in that, The bounding box prediction results are obtained through classification prediction, specifically including: The length and width of each anchor frame are reduced by the same proportion to obtain auxiliary anchor frames; Calculate the second intersection-union ratio between each of the auxiliary anchor boxes and the real box; If it is confirmed that the second intersection-union ratio is greater than 0 and the corresponding anchor frame is not classified as a positive sample, then the corresponding anchor frame is classified as an edge sample. The features of the positive samples and the edge samples are extracted to obtain the bounding box prediction results.

10. A target detection system for images of intangible cultural heritage, characterized in that, The system includes an acquisition module and a processing module, wherein, The acquisition module is used to collect intangible cultural heritage images, preprocess the intangible cultural heritage images to obtain preprocessed images; extract high-order semantic features of the preprocessed images using a feature extractor; calculate the direct similarity and indirect similarity between different preprocessed images based on the high-order semantic features of the preprocessed images, and average the direct and indirect similarities to obtain joint similarity; define a first loss function based on the joint similarity, minimize the loss value through gradient descent, map the high-order semantic features of the preprocessed images to a low dimension to obtain a dimensionality-reduced image; perform clustering operations on the dimensionality-reduced image to generate cluster labels; randomly select dimensionality-reduced images from the cluster labels, compare them with the intangible cultural heritage images, confirm the names corresponding to the cluster labels, generate a mapping table of labels and names, and obtain intangible cultural heritage labeled images; The processing module is used to train a target detection model based on the intangible cultural heritage labeled image, and to process the intangible cultural heritage object to be detected through the target detection model, and output the target detection result.

Images