Automatic encoding method and system for ethnic visual picture library

Abstract

The present invention discloses an automatic encoding method and system for an ethnic visual picture library, and belongs to the technical field of computer visual picture classification. The present invention first manually classifies and encodes the original pictures in the ethnic visual image picture library, and then completes the initial annotation of the picture library; then the annotated pictures are stored in a combination of a relational database and an object storage OSS database; then the object storage OSS database is expanded and the relational database is updated; then a feature extraction convolutional neural network is constructed: the SE module is integrated into the convolutional neural network ResNet‑18 to realize the compression and excitation of feature extraction; and the strategy of transfer learning is used to train the constructed feature extraction convolutional neural network; finally, the picture similarity is evaluated. The present invention provides support for the subsequent analysis and research of the ethnic visual picture library.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision picture classification, and in particular relates to an automatic encoding method and system for an ethnic vision picture library. Background Art

[0002] Ethnic pictures are important carriers of the cultural symbols of various ethnic groups in China. They record the cultural symbols and visual images of the culture, life, production and other activities of various ethnic groups in China. Constructing ethnic picture data sets is the basic work for processing and analyzing ethnic picture big data. With the widespread use of big data technology, the technology for obtaining ethnic picture big data has become mature, but the classification, labeling and identification of these pictures are still in the traditional file and manual management stage.

[0003] The development of new technologies such as the Internet, big data, cloud computing, and artificial intelligence has provided new directions, new means, and new methods for the study of national image and cultural symbols. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention proposes an automatic encoding method and system for an ethnic visual picture library, aiming to solve the problems of automatic encoding and management of the ethnic picture library and provide support for subsequent analysis and research of the ethnic visual picture library.

[0005] The above purpose is achieved through the following technical solutions:

[0006] The present invention first provides an automatic encoding method for an ethnic visual picture library, the method comprising the following steps:

[0007] S1. Manually classify and encode the original images in the ethnic visual image database and complete the initial annotation of the database;

[0008] S2. A relational database and an object storage OSS database are combined for the image marked in step S1, wherein the object storage OSS database stores the image data, and the relational database stores the metadata information of the image;

[0009] S3. Preprocess and standardize the images in the OSS database, generate corresponding copies of the image library, expand the OSS database, and update the relational database;

[0010] S4. Construct a feature extraction convolutional neural network: Integrate the SE module into the convolutional neural network ResNet-18 to achieve compression and excitation of feature extraction;

[0011] S5. Using a transfer learning strategy to train the feature extraction convolutional neural network constructed in step S4;

[0012] S6. Use the feature extraction convolutional neural network trained in step S5 to evaluate the image similarity.

[0013] Furthermore, the specific method of step S2 is to store the labeled pictures in a hierarchical manner in a tree-like folder manner, and use the OSS folder import method to import the pictures in the local folder as a whole into the object storage OSS database. The object storage OSS database is classified and stored in the corresponding bucket bucket according to the ethnic picture coding rules, wherein the picture files are saved as OSS file objects; the metadata of the corresponding pictures, including the file name, file type, picture size, upload time, and OSS path information, are saved in the relational database database. The relational database traverses the object storage OSS database picture metadata information, parses the relative path of the picture in the object storage OSS database level by level, automatically encodes the picture according to the coding described in step S1, and returns the file name, file type, picture size, upload time, and OSS path information of the picture in the object storage OSS database to the API in the object storage OSS database for updating the data in the relational database.

[0014] Furthermore, the preprocessing and standardization in step S3 specifically involves rotating, flipping, scaling, translating, and resizing the images in the object storage OSS database, establishing a rotation image library, a flip image library, a scaling image library, a translation image library, and a resizing image library, and updating the relational database accordingly.

[0015] Furthermore, in step S4, the convolutional neural network ResNet-18 is composed of five basic convolutional modules stacked together, wherein the basic convolutional modules include Conv1, Conv2, Conv3, Conv4, and Conv5, wherein the maximum pooling is added after Conv1 to reduce the dimension; the convolution kernel size of Conv1 is 7*7, and the convolution kernel sizes of other convolutional modules are all 3×3;

[0016] Step S4 describes integrating the SE module into the convolutional neural network ResNet-18 to achieve compression (Squeeze) and excitation (Excitation) of feature extraction. Specifically, the SE module is added between Conv3 and Conv4. First, the first two components of ResNet-18, Conv1 and Conv2, are used to extract basic features. Then, the SE module is applied to the output feature map of the Conv3 layer for fine-grained channel reweighting. The weighted feature map is input into Conv4 and Conv5. Finally, the picture category is output through the fully connected layer and the classification layer. The SE module includes two processes: compression and excitation.

[0017] Furthermore, the compression process and excitation process of the SE module are specifically as follows:

[0018] (1) Compression process

[0019] The input of the SE module is a feature map output by the Conv3 layer, and the dimension of the feature map is ,in represents the height of the feature map, represents the width of the feature map, Indicates the number of channels; first, global average pooling is used for each channel. , this process is expressed by the following formula:

[0020] = ,

[0021] in It is a channel The global pooling result is It is a channel In spatial position The eigenvalue at , the size of each channel is The two-dimensional feature maps are compressed into a scalar , the scalar Represents the channel After global average pooling, the pooling results of all channels form a length of Vector ;

[0022] (2) Motivation process

[0023] The length formed by the compression process is The vector is input into the fully connected network of the SE module to process the pooling result of each channel to generate the adaptive weight of each channel. The fully connected network of the SE module consists of two fully connected layers FC and two nonlinear activation functions, including the ReLU activation function and the Sigmoid activation function. The vector obtained by pooling Send it to two fully connected layers FC to get the adaptive weight vector :

[0024] ,

[0025] in is the calculated adaptive weight vector, which represents the importance weight of each channel after passing through the network; is the ReLU activation function; is the weight matrix of the first fully connected layer, and the dimension reduction operation is performed by setting a scaling ratio So that the output dimension is ; is the weight matrix of the second fully connected layer, restoring the vector to its original dimension ; It is a Sigmoid activation function, which is used to generate adaptive weights for each channel, with weight values ​​between [0,1]. These weights are applied to each channel of the original feature map through channel-by-channel multiplication to adjust the contribution of each channel. The formula is as follows:

[0026] ,

[0027] in, Represents the weight-adjusted The feature map of each channel, Represents the original input feature map The feature map of each channel, Indicates the first Adaptive weights for each channel.

[0028] Furthermore, the zoom ratio Set to 16.

[0029] Furthermore, in step S5, the transfer learning strategy is used to train the feature extraction convolutional neural network constructed in step S4. Specifically, the convolutional neural network ResNet-18 is pre-trained using the initial annotated image data in the OSS database, and the pre-trained weights are applied to the model training of the visual data set. The cross entropy loss function is selected in the training process, and its formula is:

[0030] ,

[0031] In the formula p It is the result of model classification, and class is the label corresponding to the data;

[0032] Using the features extracted by the convolutional neural network ResNet-18, for a given input image , after the model , get the feature vector of the corresponding image ; Set the model to inference mode. During the forward propagation process, the model's weight parameters will remain fixed and will not be automatically updated. By performing forward propagation, the model will generate a feature vector based on the features of the input image;

[0033] Then introduce global average pooling to calculate the feature vectors of all images in the same category. Take the average to get a representative feature vector , the representative feature vector It is calculated by the following mathematical formula:

[0034] ,

[0035] in, Indicates category The number of images in Indicates category Middle The feature vector of an image.

[0036] Furthermore, the image similarity evaluation in step S5 is performed using cosine similarity, which is specifically achieved by calculating the cosine value of two vectors in the inner product space. N Two vectors in dimensional space X and Y The cosine similarity between The calculation formula is:

[0037] ,

[0038] In the above formula, X and Y are the feature vectors of any two images, G is the length of the feature vector, g∈G, , Represent the modulus of eigenvectors X and Y respectively.

[0039] The present invention also provides a management system for ethnic visual picture libraries, which is used to run the automatic encoding method of the ethnic visual picture library. The back-end server of the system adopts the Python Web framework Tornado, which integrates the picture management module and the picture analysis module. The front-end interface adopts the Vue3 framework, and the front-end and back-end process interaction adopts the new WebSocket protocol of HTML5, which ensures that full-duplex real-time communication between the front-end and back-end can be achieved after a session connection.

[0040] The picture management module is used for batch uploading, picture browsing, and picture library statistics functions;

[0041] The picture analysis module is used for picture library color clustering, picture recognition, and picture library automatic encoding;

[0042] The system also includes a database, which adopts a fusion mode of a relational database and an object storage OSS database, wherein the object storage OSS database stores image data, and the relational database stores metadata information of the image.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The classification and automatic encoding method for ethnic picture libraries proposed in the present invention proposes an improved convolutional neural network feature extraction method and an ethnic picture similarity calculation method, which improves the classification accuracy of similar picture subclasses. The ethnic visual picture library automatic encoding system proposed in the present invention in combination with the method effectively simplifies the complex workflow of manual classification and annotation for building a library from an ethnic visual picture library, and can achieve automatic classification and encoding of ethnic visual pictures, providing an automated solution for the construction of large-scale ethnic cultural picture libraries. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a structural diagram of a feature extraction convolutional neural network of the present invention;

[0046] Figure 2 The fourth-level loss and accuracy curve diagram of the training of the neural network of the present invention, wherein (a) shows the training loss curve, and (b) shows the training accuracy curve;

[0047] Figure 3 It is the ethnic picture similarity distance confusion matrix of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0049] The automatic encoding method of an ethnic visual picture library in this embodiment is described by taking the collected pictures of costumes of 10 ethnic groups in southwest China as an example. Figure 3 The cosine similarity distance of the visual images of 10 ethnic groups collected in this embodiment is used to measure the similarity between two vectors. The value range is from -1 to 1, where 1 means completely identical and -1 means completely opposite. The values ​​on the diagonal are all 1, indicating that each ethnic group is completely similar to itself. For example, the similarity between the costumes of the Shui and Qiang ethnic groups is -0.52, and that of the Qiang and Zhuang ethnic groups is -0.49, indicating that they are quite different, while that of the Zhuang and Shui ethnic groups is 0.34, indicating that they are quite similar. The method of the present invention comprises the following steps:

[0050] S1. After classifying and encoding the pictures in the ethnic visual image library, each picture is labeled; in this embodiment, the ethnic visual image library is set to a 17-bit code sequence, including a 3-bit ethnic code segment and a 14-bit category code segment. The code adopts letters and decimal code sequences, which are represented by letters A and B plus decimal numbers, respectively, where:

[0051] The 3-digit ethnic code segment ranges from A01 to A56, indicating the ethnic category code to which the picture belongs, which is the first level;

[0052] The category code segment is 14 bits, indicating the category to which the picture belongs. It includes three levels. The first-level classification is set to a 2-bit code, including nine major elements: clothing, food, housing, transportation, work, soldiers, etiquette, environment, and image. The second-level classification is the first subdivision of the nine major elements, with a code length of 2 bits, from 11 to 94. The third-level classification is a secondary subdivision based on the second subdivision of the nine major elements, set to a 2-bit code, from 01 to 79. The first, second, and third level classifications are encoded in sequence, followed by the picture sequence code, indicating the picture number. The sorting method is naturally increasing, set to an 8-bit code, from 00000001 to 99999999.

[0053] S2. A combination of a relational database and an object storage OSS database is used for the images marked in step S1, wherein the object storage OSS database stores image data, and the relational database stores metadata information of the images; the marked images are stored hierarchically in a tree-like folder format, and the images in the local folder are imported into the object storage OSS database as a whole using the OSS folder import method, and the object storage OSS database is classified and stored in corresponding buckets according to ethnic image coding rules; the image files are converted into OSS BLOB binary data, and the metadata of the corresponding images, including the file name, file type, image size, upload time, and OSS path information, are correspondingly saved in the relational database, and the relational database traverses the image metadata information in the OSS database, parses the relative path of the image in the OSS database step by step, and automatically encodes the image according to the encoding in step S1, and returns the file name, file type, image size, upload time, and OSS path information of the image in the object storage OSS database to the API in the object storage OSS database for updating the data in the relational database. For image entity data, it is converted into BLOB format through the file import function of the OSS library. During the encoding process, a FileList list is created for each image file based on the relative position, and the string is split by the character " / " path to obtain an array Pathlist; the file name (excluding the extension) and file type (extension) are obtained, and a numbered file name encoding is created, which is an 8-digit number filled with zeros; the file size Size is calculated by the operating system function GetFileSize, and then another function CountSize converts it; a SQL insert statement is constructed to insert a record in the data table.

[0054] S3. Preprocess and standardize the images in the object storage OSS database, rotate, flip, scale, translate, and resize the images in the object storage OSS database, establish a rotation image library, a flip image library, a scale image library, a translation image library, and a resize conversion image library, and update the relational database accordingly. In order to adapt to the training and system display of images of different sizes, convert the images to 32×32, 64×64, and 128×128 resolution images, and perform image encoding and database update at the same time.

[0055] S4. Constructing a feature extraction convolutional neural network: Integrate the SE module into the convolutional neural network ResNet-18 to achieve compression and excitation of feature extraction; Figure 1 As shown, the convolutional neural network ResNet-18 described in this embodiment is composed of 5 basic convolutional component modules stacked, and the basic convolutional component modules include Conv1, Conv2, Conv3, Conv4, and Conv5, wherein the maximum pooling is added after Conv1 to reduce the dimension; the convolution kernel size of Conv1 is 7*7, and the convolution kernel sizes of other convolutional component modules are all 3×3;

[0056] The SE module is integrated into the convolutional neural network ResNet-18 to realize the compression (Squeeze) and excitation (Excitation) of feature extraction. Specifically, the SE module is added between Conv3 and Conv4. First, the first two components of ResNet-18, Conv1 and Conv2, are used to extract basic features. Then, the SE module is applied to the output feature map of the Conv3 layer for fine-grained channel reweighting, and the weighted feature map is input to Conv4 and Conv5. Finally, the picture category is output through the fully connected layer and the classification layer. The SE module includes two processes, compression and excitation. The compression process and excitation process of the SE module are as follows:

[0057] (1) Compression process

[0058] The input of the SE module is a feature map output by the Conv3 layer, and the dimension of the feature map is ,in represents the height of the feature map, represents the width of the feature map, Indicates the number of channels; first, global average pooling is used for each channel. , this process is expressed by the following formula:

[0059] = ,

[0060] in It is a channel The global pooling result is It is a channel In spatial position The eigenvalue at , the size of each channel is The two-dimensional feature maps are compressed into a scalar , the scalar Represents the channel After global average pooling, the pooling results of all channels form a length of Vector ;

[0061] (2) Motivation process

[0062] The length formed by the compression process is The vector is input into the fully connected network of the SE module to process the pooling result of each channel to generate the adaptive weight of each channel. The fully connected network of the SE module consists of two fully connected layers FC and two nonlinear activation functions, including the ReLU activation function and the Sigmoid activation function. The vector obtained by pooling Send it to two fully connected layers FC to get the adaptive weight vector :

[0063] ,

[0064] in is the calculated adaptive weight vector, which represents the importance weight of each channel after passing through the network; is the ReLU activation function; is the weight matrix of the first fully connected layer, and the dimension reduction operation is performed by setting a scaling ratio So that the output dimension is , in this embodiment, the scaling ratio Set to 16. is the weight matrix of the second fully connected layer, restoring the vector to its original dimension ; It is a Sigmoid activation function, which is used to generate adaptive weights for each channel, with weight values ​​between [0,1]. These weights are applied to each channel of the original feature map through channel-by-channel multiplication to adjust the contribution of each channel. The formula is as follows:

[0065] ,

[0066] in, Represents the weight-adjusted The feature map of each channel, Represents the original input feature map The feature map of each channel, Indicates the first Adaptive weights for each channel.

[0067] S5. Model training:

[0068] In this example, the batch size is set to 64, and 50 epochs of iterative training are performed. The initial learning rate is set to 0.001. Before training, the input image is preprocessed and standardized. The preprocessing operation helps to improve the model's ability to extract image features and keep the input data within a suitable range.

[0069] The present invention adopts a transfer learning strategy to perform model training, uses a large amount of image data in the ImageNet database to pre-train SResNet-18, and applies the pre-trained weights to model training at other levels of the visual dataset.

[0070] The cross entropy loss function is selected in the training process and its formula is:

[0071] ,

[0072] In the formula p It is the result of model classification, and class is the label corresponding to the data.

[0073] Using the features extracted by the convolutional neural network, for a given input image , after the model , we can get the feature vector of the corresponding image . Set the model to inference mode. During the forward propagation process, the model's weight parameters will remain fixed and will not be automatically updated. By performing forward propagation, the model will generate a feature vector based on the features of the input image.

[0074] This paper introduces global average pooling (GAP) to pool the feature vectors of all images in the same category. Take the average value to get a representative feature vector. It can be calculated by the following mathematical formula:

[0075] ,

[0076] in, Indicates category The number of images in Indicates category Middle The feature vector of an image.

[0077] S6. Image Similarity Evaluation

[0078] The cosine similarity introduced in the present invention is a quantitative method for evaluating vector similarity, which is achieved by calculating the cosine value of two vectors in the inner product space. N Two vectors in dimensional space X and Y The cosine similarity between The calculation formula is:

[0079] ,

[0080] In the above formula, X and Y are the feature vectors of any two images, G is the length of the feature vector, g∈G, , Represent the modulus of eigenvectors X and Y respectively. Figure 2 This is the accuracy and loss diagram of the fourth level of the neural network model SResNet-18 in training fine classification, from Figure 2 It can be seen that after about 50 rounds of training, the loss rate of SResNet-18 is 5.7% and the accuracy rate reaches 98.4%, indicating the advantages of the model of the present invention in image feature extraction capability and recognition accuracy.

[0081] The present invention also provides a management system for ethnic visual picture libraries. The back-end server of the system adopts the Python Web framework Tornado, integrates the trained model and realizes the automatic encoding module of the picture library. The system includes picture uploading, recognition, encoding and color clustering, etc. The front-end interface adopts the Vue3 framework, and the front-end and back-end process interaction adopts the new WebSocket protocol of HTML5, ensuring that full-duplex real-time communication between the front-end and back-end can be realized after a session connection.

[0082] The picture management module realizes the functions of batch uploading, picture browsing and picture library statistics of the picture library;

[0083] The picture analysis module realizes picture library color clustering, picture recognition, and picture library automatic encoding;

[0084] Furthermore, the system also includes a database, and the attributes of the collected ethnic visual images include: image collector, collection location, image format, image encoding, and the OSS storage location of the image. MySQL is selected as the relational database because MySQL is open source, supports multi-threading, provides APIs for multiple programming languages, and has its own optimizer that can effectively improve query speed. The OSS image object database uses MinIO, which is an open source, high-performance, cloud-native object storage system that is compatible with the Amazon S3 API, has horizontal expansion capabilities and powerful data protection functions, and is suitable for image library big data storage and management.

[0085] The description of the above embodiments is intended to facilitate the understanding and application of the present invention by those skilled in the art. It is obvious to those skilled in the art that various modifications can be made to these embodiments without creative work, and the general principles described in the present invention can be applied to other embodiments. Therefore, the protection scope of the present invention is not limited to these specific embodiments. All improvements and changes made according to the disclosure of the present invention should fall within the protection scope of the present invention.

Claims

1. An automatic encoding method for ethnic visual picture library, characterized in that: The method comprises the following steps: S1. Manually classify and encode the original images in the ethnic visual image library and complete the initial annotation of the image library; S2. A relational database and an object storage OSS database are combined for the image marked in step S1, wherein the object storage OSS database stores the image data, and the relational database stores the metadata information of the image; S3. Preprocess and standardize the images in the object storage OSS database, generate corresponding image library copies, expand the object storage OSS database and update the relational database; S4. Construct a feature extraction convolutional neural network: Integrate the SE module into the convolutional neural network ResNet-18 to achieve compression and excitation of feature extraction; S5. Using a transfer learning strategy to train the feature extraction convolutional neural network constructed in step S4; S6. Use the feature extraction convolutional neural network trained in step S5 to perform image similarity evaluation; The specific method of step S2 is to store the annotated pictures in a hierarchical manner in a tree-like folder manner, and use the OSS folder import method to import the pictures in the local folder into the object storage OSS database as a whole. The object storage OSS database is classified and stored in the corresponding bucket bucket according to the ethnic picture coding rules, wherein the picture files are saved as OSS file objects; the metadata of the corresponding pictures, including the file name, file type, picture size, upload time, and OSS path information, are saved in the relational database database, and the relational database traverses the metadata information of the object storage OSS database pictures, and parses the relative path of the pictures in the object storage OSS database step by step, and automatically encodes the pictures according to the coding described in step S1, and returns the file name, file type, picture size, upload time, and OSS path information of the pictures in the object storage OSS database to the API in the object storage OSS database for updating the data in the relational database; In step S4, the convolutional neural network ResNet-18 is composed of five basic convolutional modules stacked together, including Conv1, Conv2, Conv3, Conv4, and Conv5, wherein the maximum pooling is added after Conv1 to reduce the dimension; the convolution kernel size of Conv1 is 7*7, and the convolution kernel sizes of other convolutional modules are all 3×3; Step S4 describes integrating the SE module into the convolutional neural network ResNet-18 to achieve compression and excitation of feature extraction. Specifically, the SE module is added between Conv3 and Conv4. First, the first two components of ResNet-18, Conv1 and Conv2, are used to extract basic features. Then, the SE module is applied to the output feature map of the Conv3 layer for fine-grained channel reweighting. The weighted feature map is input into Conv4 and Conv5. Finally, the picture category is output through the fully connected layer and the classification layer. The SE module includes two processes: compression and excitation.

2. The automatic encoding method of an ethnic visual picture library according to claim 1 is characterized in that: The preprocessing and standardization described in step S3 specifically involves rotating, flipping, scaling, translating, and resizing the images in the object storage OSS database, establishing a rotation image library, a flip image library, a scaling image library, a translation image library, and a resizing image library, and updating the relational database accordingly.

3. The automatic encoding method of an ethnic visual picture library according to claim 1 is characterized in that: The compression process and excitation process of the SE module are as follows: (1) Compression process The input of the SE module is a feature map output by the Conv3 layer. The dimension of the feature map is H×W×C, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels. First, global average pooling is used for each channel. For each channel c, this process is expressed by the following formula: where z c is the global pooling result of channel c, X c (m,n) is the eigenvalue of channel c at the spatial position (m,n), and each channel’s two-dimensional feature map of size H×W is compressed into a scalar z c , the scalar z c represents the global information or importance of channel c; after global average pooling, the pooling results of all channels form a vector z=[z1,z2,z3,…z C ]; (2) Motivation process The vector of length C formed after the compression process is input into the fully connected network of the SE module to process the pooling result of each channel to generate the adaptive weight of each channel. The fully connected network of the SE module consists of two fully connected layers FC and two nonlinear activation functions, including the ReLU activation function and the Sigmoid activation function. The pooled vector z is sent to the two fully connected layers FC to obtain the adaptive weight vector s: s=σ(W2·δ(W1·z)) where s=[s1,s2,s3,…s C ] is the calculated adaptive weight vector, which represents the importance weight of each channel after passing through the network; δ(·) is the ReLU activation function; W1 is the weight matrix of the first fully connected layer, and the dimension reduction operation is performed by setting a scaling ratio r so that the output dimension is C / r; W2 is the weight matrix of the second fully connected layer, which restores the vector to the original dimension C; σ(·) is the Sigmoid activation function, which is used to generate adaptive weights for each channel, and the weight value is between [0,1]; these weights are applied to each channel of the original feature map through channel-by-channel multiplication to adjust the contribution of each channel. The formula is as follows: in, represents the feature map of the cth channel after weight adjustment, X c represents the feature map of the cth channel in the original input feature map, s c Represents the adaptive weight of the cth channel calculated by the SE module.

4. The automatic encoding method of an ethnic visual picture library according to claim 3 is characterized in that: The scaling ratio r is set to 16.

5. The automatic encoding method of an ethnic visual picture library according to claim 1 is characterized in that: In step S5, the transfer learning strategy is used to train the feature extraction convolutional neural network constructed in step S4. Specifically, the convolutional neural network ResNet-18 is pre-trained using the initial annotated image data in the OSS database, and the pre-trained weights are applied to the model training of the visual data set. The cross entropy loss function is selected in the training process, and its formula is: Where p is the result of model classification, and class is the label corresponding to the data; Using the features extracted by the convolutional neural network ResNet-18, for a given input image x, after model f θ , obtain the feature vector z=f of the corresponding image θ (x); Set the model to inference mode. During the forward propagation process, the model's weight parameters will remain fixed and will not be automatically updated. By performing forward propagation, the model will generate a feature vector based on the features of the input image. Then introduce global average pooling, take the average of the feature vectors z of all images in the same category to get a representative feature vector v i , v i It is calculated by the following mathematical formula: Among them, N i represents the number of images in category i, Represents the feature vector of the jth image in category i.

6. The automatic encoding method of an ethnic visual picture library according to claim 1 is characterized in that: The image similarity evaluation in step S5 is performed by using cosine similarity, which is specifically achieved by calculating the cosine value of two vectors in the inner product space. The calculation formula for the cosine similarity cos(θ) between two vectors X and Y in the N-dimensional space is: In the above formula, X and Y are the feature vectors of any two images, G is the length of the feature vector, g∈G, ||A|| and ||B|| represent the modulus of the feature vectors X and Y respectively.

7. A management system for ethnic visual picture libraries, the system is used to run the automatic encoding method of the ethnic visual picture library according to any one of claims 1 to 6, characterized in that: The system's backend server uses the Python Web framework Tornado, which integrates image management modules and image analysis modules. The front-end interface uses the Vue3 framework, and the front-end and back-end process interactions use the new HTML5 WebSocket protocol to ensure that full-duplex real-time communication between the front and back ends can be achieved after a session connection. The picture management module is used for batch uploading, picture browsing, and picture library statistics functions; The picture analysis module is used for picture library color clustering, picture recognition, and picture library automatic encoding; The system also includes a database, which adopts a fusion mode of a relational database and an object storage OSS database, wherein the object storage OSS database stores image data, and the relational database stores metadata information of the image.

Images