Deployment method and application terminal based on digital culture field big data model

Abstract

The invention relates to a deployment method, an application terminal and a query method based on a digital culture field big data model. The deployment method comprises a data preparation stage, a model training stage and a model deployment stage. According to the method, model training and analysis processing are performed on the data in the digital culture field in a targeted manner, and the data asset management efficiency in the digital culture field can be greatly improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a deployment method, an application terminal and a query method based on a big data model in the field of digital culture. Background Art

[0002] The current cultural enterprises and institutions are faced with the "three difficulties" of data management, display and security assurance, including:

[0003] (1) Difficulty in centralized data management: The massive amounts of data generated by various business departments (such as work plans, schedules, design drawings, pictures, audio and video files, etc.) are stored in personal computers and mobile hard disks, lacking effective centralized management methods.

[0004] (2) Data is difficult to display in a centralized manner: In scenarios such as surveys, visits, and external presentations, data needs to be temporarily organized, which is time-consuming and labor-intensive, and the display format is single, usually simply listed.

[0005] (3) Difficulty in ensuring data security: Due to the decentralized storage of data and poor security, industry development has put forward higher requirements for data security.

[0006] Based on the above problems, cultural enterprises and institutions are in urgent need of a method and intelligent terminal device that can integrate data collection, storage, governance, display, and security in one stop. Such devices need to face the query and analysis of various types of files in the digital cultural field, and all of these need to rely on an efficient digital cultural field big data model to quickly improve data asset management efficiency, ensure data security, and provide diversified and convenient data display functions. At present, there is no training, deployment, and query method specifically for big data models in the digital cultural field, nor is there a corresponding application terminal product. Summary of the invention

[0007] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the technical problems existing in the prior art.

[0008] On the one hand, the present application provides a query method based on a big data model in the field of digital culture, including:

[0009] Data preparation stage: Establishing a knowledge graph in the field of digital culture; collecting digital cultural product data of different file types provided by users, and labeling the digital cultural product data of different file types by adding data entries; after the labeling process, removing noise data, incomplete data, and data irrelevant to the labeling process from the digital cultural product data of different file types; the types of digital cultural product data of different file types include text, images, audio and video;

[0010] Model (in this application, "model" refers to "big data model in the digital culture field") training phase: for different types of digital cultural product data of different file types, adopt corresponding feature extraction methods to obtain corresponding structured digital cultural product data feature vectors; based on the structured digital cultural product data feature vectors and the digital culture field knowledge graph, set a category attention mechanism algorithm, learn multiple different types of data features in parallel through a multi-task method, and obtain a big data model in the digital culture field; select a specified loss function and optimization algorithm to optimize the parameters of the big data model in the digital culture field;

[0011] Model deployment phase: input the pre-reserved validation set and test set into the digital culture field big data model to evaluate the performance of the digital culture field big data model; adjust the parameters of the digital culture field big data model according to the performance of the digital culture field big data model; when the performance of the digital culture field big data model meets the specified performance and remains stable, embed the digital culture field big data model into the specified digital culture product application terminal.

[0012] In one embodiment, the adopting of a corresponding feature extraction method to obtain a corresponding structured digital cultural product data feature vector includes:

[0013] For text-type cultural product data, semantic word segmentation and understanding are performed through natural language processing methods to convert the text-type cultural product data into text word feature vectors or sentence feature vectors;

[0014] For image-type cultural product data, feature extraction is performed using a convolutional neural network method to generate an image feature vector representing the image-type cultural product data;

[0015] For audio-type cultural product data, perform speech recognition analysis and convert it into speech word vectors or sentence vectors through natural language processing methods;

[0016] For video-type cultural product data, the video frames are processed into image sequences, and the visual features and changes in the time dimension of each frame are extracted to form a video feature vector representing the video-type cultural product data.

[0017] In one embodiment, during the model training phase, the method further includes:

[0018] Data enhancement operations are performed on the digital cultural product data of the different file types to expand the training set of the big data model in the digital culture field.

[0019] In one embodiment, during the model training phase, the method further includes:

[0020] Through regularization methods, the big data model in the digital culture field is prevented from overfitting.

[0021] In one embodiment, performing data enhancement operations on the digital cultural product data of different file types to expand the training set of the digital cultural field big data model includes:

[0022] For text-type cultural product data, the structured digital cultural product data feature vector is expanded through semantic expansion;

[0023] For the image type cultural product data, by comparing similarity, determining whether it is suitable for rotation, cropping or color transformation, and expanding the feature vector of the structured digital cultural product data;

[0024] For audio type cultural product data, after speech recognition analysis, the structured digital cultural product data feature vector is expanded through natural language expansion;

[0025] For video type cultural product data, the image sequence converted from the video frame is rotated, cropped or color transformed to expand the structured digital cultural product data feature vector.

[0026] In one embodiment, the setting category attention mechanism algorithm includes:

[0027] Different types of cultural product data vectors are assigned different weights according to user specifications and / or query history, thereby obtaining a sorted training set for the digital culture field big data model.

[0028] In one embodiment, different types of cultural product data vectors are assigned different weights according to user specifications and / or query history, including:

[0029] Assigning different weights to different types of cultural product data vectors according to the priorities specified by the user; and / or

[0030] Based on the user's query history, the query bias of different types of files is predicted and assigned different weights.

[0031] In one embodiment, the setting category attention mechanism algorithm further includes:

[0032] Adjusting the weights through forward propagation and back propagation;

[0033] Perform normalized weighted scoring on each vector unit in the data vectors of different types of cultural products;

[0034] According to the result of the weighted scoring, a sorted training set of the big data model in the digital culture field is determined.

[0035] A second aspect of the present application provides a digital cultural product application terminal, comprising:

[0036] A big data model in the field of digital culture, deployed in the application terminal according to any one of the methods in the first aspect;

[0037] An input device, used for a user to input query information;

[0038] The display device is used to present the query results.

[0039] The third aspect of the present application provides a query method based on a big data model in the field of digital culture, characterized by comprising:

[0040] Input query conditions on the application terminal according to the second aspect, generate input suitable for the digital culture field big data model, obtain query results after processing by the digital culture field big data model, and present them to the user.

[0041] In one embodiment, presenting to the user includes:

[0042] Presenting entity names having edge relationships with the digital culture field knowledge graph on the human-computer exchange interface;

[0043] In response to a user clicking on an entity in the digital culture field knowledge graph, corresponding digital culture field data is presented on the human-computer interaction interface in the form of thumbnails.

[0044] The digital culture field big data model obtained by one or more of the above technical solutions of the present application has at least one or more of the following beneficial effects:

[0045] (1) Efficient data management: Through automated classification, labeling, and retrieval functions, data management efficiency is significantly improved, enabling rapid response to data display needs in different scenarios.

[0046] (2) Data processing of different file types: Supports comprehensive display of multiple data formats and achieves a highly interactive user experience, enriching the form and effect of data display.

[0047] (3) Intelligence and automation: Big data models in the digital cultural field can automatically manage and display data, and support remote operation and maintenance and automatic repair functions, greatly reducing operation and maintenance costs and personnel investment.

[0048] This technical solution can significantly improve the data management level and display capabilities of cultural enterprises and institutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:

[0050] Figure 1 is a schematic diagram of a hardware architecture for implementing a method of the present application according to an embodiment of the present application;

[0051] Figure 2 It is a flowchart of the main steps of a deployment method based on a big data model in the digital culture field according to an embodiment of the present application;

[0052] Figure 3 is a data processing process framework diagram according to an embodiment of the present application;

[0053] Figure 4 It is a data processing process framework diagram according to another embodiment of the present application. DETAILED DESCRIPTION

[0054] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0055] In the description of this application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, and memories, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Computer-readable storage media include any suitable media that can store program codes, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc.

[0056] The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one of A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B, or A and B. The singular forms "a", "the" and "the" may also include plural forms.

[0057] Here we first explain some terms involved in this application.

[0058] Knowledge graph: refers to a technical method that uses a graph model to describe knowledge and model the relationship between all things in the world. It consists of nodes and edges. Among them, nodes represent entities, concepts or attribute values, edges represent relationships, and entities, attributes and edges form triples.

[0059] Knowledge graph in the field of digital culture: refers to the knowledge graph extracted from different file types such as text records, digital images, audio, video, etc. in the field of digital culture. By integrating corresponding entities, relationships and attributes, professional field knowledge graphs for different types of files are obtained.

[0060] In the field of digital culture, there are many types of files that need to be processed, and the artistic, content and professional quality of the files are high, which is different from other fields. Therefore, when processing this type of data, it is necessary to build a dedicated knowledge graph. For example, the creation background of the Kunqu opera "Peony Pavilion" can be described in words, or it can be digitized by high-definition scanning of Kunqu opera shapes from different angles, or it can be recorded by recording documentaries. These digitized works belong to different forms of file types and contain a large number of artistic features and professional terms unique to the field of digital culture. If a general knowledge processing method is used for processing, it is inevitable that there will be problems of low accuracy and unsatisfactory results. Therefore, it is necessary to establish a knowledge graph unique to the field of digital culture to process this type of data.

[0061] Figure 1 A structural block diagram of a computer system provided by an exemplary embodiment of the present application is shown, and the method involved in the present application is executed by the computer system or part of the computer system.

[0062] The computer system includes: a first terminal 101, a second terminal 102, a third terminal 103, a fourth terminal 104, a central processing server 105 and a cloud. The first terminal 101 to the fourth terminal 104 can be connected to the central processing server 105 via a wireless network or a wired network.

[0063] The central processing server 105 may include at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. The server includes a processor and a memory.

[0064] The first terminal 101 to the fourth terminal 104 can be a desktop computer or a portable mobile terminal, such as a smart phone, a tablet computer, a laptop computer, etc., as long as it has display and input functions. The first terminal 101 to the fourth terminal 104 can also include a processor and a memory. In addition, those skilled in the art will understand that although Figure 1 Four terminals are shown in the figure, however, the present application is not limited thereto and may include more or fewer terminals.

[0065] It is worth noting that due to the diverse types and large number of digital cultural data files, the storage space requirements are also high. Therefore, the central processing server 105 can be connected to the cloud to store most of the digital cultural data files in the cloud. The computing function of the entire model can even be decomposed to train and calculate the model through the cloud. The central processing server 105 is mainly used for task scheduling and result presentation, thereby improving the computing and storage efficiency of the entire system.

[0066] Among them, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations such as large model modeling and training.

[0067] The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory is used to store at least one instruction, which is used to be executed by the corresponding processor to implement the method involved in the present application.

[0068] Establishing a knowledge graph in the field of digital culture

[0069] To establish a knowledge graph in the field of digital culture, it is necessary to define entities, relationships and attributes according to the requirements of digital cultural units (often subject to the norms of the digital cultural field, such as common classification patterns, etc.).

[0070] Entity definition: can be used as a node type in the knowledge graph.

[0071] Relationship definition: defines the relationship between entities.

[0072] · Attribute definition: Add attributes to the entity.

[0073] For example, taking the digital cultural unit as the Kunqu Opera group, we design a knowledge graph:

[0074] 1. Design entity

[0075] Kunqu opera repertoire: the main repertoires performed by the Kunqu opera troupe.

[0076] Kunqu plot: description of the plot related to Kunqu opera.

[0077] · Kunqu version: related to the version of Kunqu opera.

[0078] Performing artists: Artists related to Kunqu opera.

[0079] ·Performing Orchestra: Information about the group that performs music for Kunqu operas.

[0080] Documentary materials: research and historical documents about the Kunqu opera.

[0081] 2. Design relationships

[0082] Belongs to: The relationship between Kunqu opera repertoire and the historical period to which it belongs.

[0083] Participation: The relationship between performing artists and Kunqu opera repertoire.

[0084] Creation: The creative relationship between the version of the Kunqu opera.

[0085] Performance: The relationship between Kunqu opera repertoire and performance groups.

[0086] ·Record: Kunqu opera repertoire information is recorded in documentary materials.

[0087] At this point, users such as the Kunqu Opera Troupe can associate their own digital cultural data with the knowledge graph and obtain a digital cultural knowledge graph that is suitable for their own business characteristics.

[0088] In one embodiment, Figure 3 As shown, the deployment method of the digital culture field big data model of the present application includes:

[0089] S201, data preparation stage: establishing a knowledge graph in the field of digital culture; collecting digital cultural product data of different file types provided by users, and labeling the digital cultural product data of different file types by adding data entries; after the labeling, removing noise data, incomplete data, and data irrelevant to the labeling in the digital cultural product data of different file types; the types of digital cultural product data of different file types include text, images, audio and video.

[0090] Specifically, the digital cultural product data of different file types provided by the user requires the user to add data entries and perform labeling processing. These data include at least two of text data, image data, audio data and video data.

[0091] In order to train the big data model in the digital cultural field, it is necessary to annotate the collected cultural product data. Specifically, a category label should be added to each data entry. For example, a script text is annotated as "drama" and a music recording is annotated as "music". This process is usually completed by experts or through semi-automatic tools to ensure the accuracy of data annotation.

[0092] In one embodiment, for example, a user of a Kunqu Opera troupe has placed multiple types of files with certain relevance in a folder, and uploads data from the folder.

[0093] In one example, the acquired digital cultural product data includes:

[0094] Textual data: "The Peony Pavilion is a fictional legend (play) created by Tang Xianzu, a playwright of the Ming Dynasty, and published in the 45th year of the Wanli reign of the Ming Dynasty (1617)."

[0095] Picture material: High-definition stills from "The Peony Pavilion".

[0096] Audio material: "Audio of the live version of "The Peony Pavilion""

[0097] Video material: A documentary introducing "The Peony Pavilion".

[0098] For example, in the process of labeling the digital files of the above-mentioned digital cultural products, the following labels can be used for labeling:

[0099] Text tags: "Tang Xianzu", "Peony Pavilion";

[0100] Image tags: "Kunqu Opera", "Peony Pavilion", "Tang Xianzu", "Feng Menglong", "Du Liniang", "Liu Mengmei";

[0101] Audio tags: "Peony Pavilion", "Shanghai Kunqu Opera Troupe", "Jiangsu Kunqu Opera Theater", "Kunqu Music";

[0102] Video tags: "Peony Pavilion", "documentary", "live video", "youth version", "Shanghai Kunqu Opera Troupe", "Jiangsu Kunqu Opera Theater".

[0103] After labeling the data, clean the data to remove noise data, incomplete data, or data irrelevant to the classification target. This step is to ensure that the training effect of the model will not be reduced due to invalid data.

[0104] S202, model training stage: for different types of digital cultural product data of different file types, adopt corresponding feature extraction methods to obtain corresponding structured digital cultural product data feature vectors; set a category attention mechanism algorithm during the model training process; based on the structured digital cultural product data feature vectors and the digital culture field knowledge graph, learn multiple different types of data features in parallel through a multi-task method to obtain a big data model based on the digital culture field; select a specified loss function and optimization algorithm to optimize the parameters of the digital culture field big data model.

[0105] like Figure 3 As shown, in one embodiment, for different types of digital cultural product data of different file types, corresponding feature extraction methods are adopted to obtain corresponding structured digital cultural product data feature vectors, including:

[0106] S301: For text-type cultural product data, semantic word segmentation and understanding are performed through natural language processing methods to convert the text-type cultural product data into text word feature vectors or sentence feature vectors. For example, for a text file introducing the youth version of Peony Pavilion, text word feature vectors such as <Kunqu Opera, Peony Pavilion, Tang Xianzu...> can be extracted.

[0107] S302: For image-type cultural product data, feature extraction is performed using a convolutional neural network method to generate an image feature vector representing the image-type cultural product data. For example, for a still from a Kunqu opera, the character modeling, stage design, and other elements of the protagonists Du Liniang, Liu Mengmei, etc. in the still can be extracted using a convolutional neural network feature extraction method to form an image feature vector.

[0108] S303: Perform speech recognition analysis on audio-type cultural product data and convert them into speech word vectors or sentence vectors through natural language processing methods. For example, through natural language processing, the lyrics in Kunqu Opera are converted into text and speech word vectors such as <Du Liniang, Liu Mengmei...> are extracted.

[0109] S304: For video-type cultural product data, process the video frames into a sequence of images, extract the visual features of each frame and the changes in the time dimension, and form a video feature vector representing the video-type cultural product data.

[0110] Specifically, since the data content of videos is much larger than other types of files, the video processing process is relatively complicated. In order to improve efficiency, after the video frame type cultural product data is processed into a picture sequence according to the video frame, several frames can be selected as seed frames by sampling. If meaningful features can be extracted from these video frames, these seed frames are retained and feature extraction is performed to obtain a video feature vector. Then, in addition to the above-mentioned seed frames, several frames are randomly selected as seed frames, feature extraction is performed, and expanded into the aforementioned video feature vector. When the features in the video feature vector tend to be stable, the entire video feature extraction process is stopped. In this process, if meaningful features cannot be extracted, the frame is discarded.

[0111] Of course, we can also use the preset threshold to determine whether the newly generated data features are consistent with the data in the existing knowledge graph. If they are consistent, we can try to integrate them to enrich and expand the content of the knowledge graph. For example, if we find that "Feng Menglong" has adapted "The Peony Pavilion", we can include "Feng Menglong" and its related entities and relationships in the digital cultural knowledge graph.

[0112] Continuing with the previous example, see Figure 3 As shown, it can be seen that the model entity recognition:

[0113] The Kunqu opera “The Peony Pavilion”, the character “Tang Xianzu” and “Ming Dynasty” were identified from the textual data.

[0114] Identify the Kunqu opera "The Peony Pavilion" from the picture materials.

[0115] The Kunqu opera "The Peony Pavilion" and the institution "Jiangsu Kunqu Opera Theater" were identified from the audio materials.

[0116] Identify the Kunqu opera "The Peony Pavilion" and its related creative background from video materials.

[0117] Relation Extraction

[0118] According to the text data, the relations "The Peony Pavilion belongs to the Ming Dynasty" and "Tang Xianzu created The Peony Pavilion" are extracted.

[0119] Based on the picture information, draw the makeup accessories for "The Peony Pavilion".

[0120] According to the audio data, the relationship "Jiangsu Kunqu Opera Theater and Shanghai Kunqu Opera Troupe have both performed "The Peony Pavilion"" is extracted.

[0121] According to the video data, the relationship "The Peony Pavilion is recorded in the documentary" is extracted.

[0122] Figure 4 is a data processing process framework diagram according to another embodiment of the present application. It can be seen that Figure 4 It is an embodiment of obtaining a big data model based on the digital culture field by learning multiple different types of data features in parallel in a multi-task manner based on the structured digital cultural product data feature vector and the digital culture field knowledge graph.

[0123] like Figure 4 As shown in the figure, the dotted box is the deployment process of the big data model in the digital culture field. Figure 3 The structured digital cultural product data feature vector 401 and the digital culture field knowledge graph 402 obtained in the steps shown are used to train the big data model in the digital culture field. The big data model in the digital culture field can be improved based on some existing big data models, such as improving the ChatGLM big model. It should be noted that in this application, a category attention mechanism algorithm is specially introduced in a large model such as ChatGLM, which is represented by 404 in the figure.

[0124] It is worth noting that the structured digital cultural product data feature vector 401 and the triple set of entities, attributes and relationships obtained from the digital culture field knowledge graph 402 are used as input. After passing through the input layer 403, during the entire feature extraction process, the extracted features and the entities and edges in the digital culture field knowledge graph can meet a sufficient similarity threshold without any concept offset.

[0125] In the comparison layer 404 after the input layer 403, a process of comparing the structured digital cultural product data feature vector 401 with the triple set will be carried out to remove some features that are irrelevant to the digital cultural field. Specifically, the triple set can be traversed to screen the elements in the structured digital cultural product data feature vector 401, remove irrelevant elements, and retain elements related to the digital cultural field. For example, in the process of extracting text data of the introduction of Kunqu works, some feature texts such as "email" and "shampoo" may be extracted, but this type of text is actually not very related to the digital cultural field, and the feature extraction process will discard these features.

[0126] In the category attention mechanism algorithm 403, the weight parameters of different types of data can be adjusted to achieve the preference setting for specific data, thereby achieving customized retrieval of different types of files. This process can be achieved through forward propagation and back propagation mechanisms in neural networks.

[0127] In one embodiment, during the model training phase, the method further includes:

[0128] Through regularization methods, the big data model in the digital culture field is prevented from overfitting.

[0129] Specifically, overfitting refers to the large gap between the model training error and the test error, that is, the model is over-trained on the training set, resulting in poor generalization ability. The main reason for overfitting is that there are too few samples in the training set or there is noise. However, too few training sets or the presence of noise cannot be controlled, so what we can do is to use regularization technology to prevent the model from learning the noise in the training samples, thereby reducing the possibility of overfitting and increasing the robustness of the model. The basic idea of ​​the regularization method is to limit the size of the weights so that the model cannot fit arbitrary noise data, thereby achieving the purpose of preventing overfitting.

[0130] In one embodiment of the present application, during the model training phase, the method further includes:

[0131] Data enhancement operations are performed on the digital cultural product data of the different file types to expand the training set of the big data model in the digital culture field.

[0132] like Figure 3 As shown, in one embodiment of the present application, the data enhancement operation includes:

[0133] S311: For text-type cultural product data, expand the structured digital cultural product data feature vector through semantic expansion;

[0134] S312: for the image type cultural product data, determine through similarity comparison that it is suitable for rotation, cropping or color transformation, and expand the feature vector of the structured digital cultural product data;

[0135] S313: For audio type cultural product data, after speech recognition analysis, the structured digital cultural product data feature vector is expanded through natural language expansion;

[0136] S314: For video-type cultural product data, the image sequence converted from the video frame is rotated, cropped or color transformed to expand the structured digital cultural product data feature vector.

[0137] Through data enhancement technology, more diverse training samples are generated to improve the generalization ability of the model. For example, synonym replacement is performed on the text introducing "The Peony Pavilion"; operations such as rotation, cropping, and interception are performed on the stills of "The Peony Pavilion"; text expansion is performed on the text analysis results of the audio interview records of the creation team of "The Peony Pavilion"; and operations such as rotation, cropping, and interception are performed on the images converted from the video frames in the documentary "The Peony Pavilion". All of these can obtain rich training samples.

[0138] Since there are large differences between cultural product categories (such as drama and music), an attention mechanism is added to the model to help the model focus on features related to specific categories when classifying.

[0139] In another embodiment, a category attention mechanism algorithm includes:

[0140] Different types of cultural product data vectors are assigned different weights according to user specifications and / or query history, thereby obtaining a sorted training set for the digital culture field big data model.

[0141] In one embodiment, different types of cultural product data vectors are assigned different weights according to user specifications and / or query history, including:

[0142] Assigning different weights to different types of cultural product data vectors according to the priorities specified by the user; and / or

[0143] Based on the user's query history, the query bias of different types of files is predicted and assigned different weights.

[0144] Specifically, in the attention mechanism algorithm 404, the priority can be adjusted in a user-specified manner so that the big data model in the digital culture field focuses on the training and query of a certain type of data. For example, during the model training process, the user may want to find the video version of the Kunqu Opera "Peony Pavilion" performed by the "Shanghai Kunqu Opera Troupe". In this case, the weight of the "Shanghai Kunqu Opera Troupe" video file can be increased, so that the model training process will focus more on the features related to it.

[0145] Of course, artificial intelligence methods can also be used to more intelligently analyze the user's query history, discover the user's interests and preferences, and adjust the weights of different features to achieve more accurate queries.

[0146] In another embodiment, the category attention mechanism algorithm further includes:

[0147] Adjusting the weights through forward propagation and back propagation;

[0148] Perform normalized weighted scoring on each vector unit in the data vectors of different types of cultural products;

[0149] According to the result of the weighted scoring, a sorted training set of the big data model in the digital culture field is determined.

[0150] For example, in the big data model of digital culture, forward propagation is the process of obtaining output from input. After calculating the loss function of the output, back propagation propagates the information of the loss function backward along the network layer to calculate the gradient, so as to achieve the purpose of optimizing parameters. Back propagation is based on the gradient descent optimization method, which updates the network weights by calculating the gradient of the loss function relative to the network parameters, with the purpose of minimizing the loss function and thus improving the prediction accuracy of the model.

[0151] S203, model deployment phase: after the model training phase is completed, the pre-reserved validation set and test set are input into the digital culture field big data model to evaluate the performance of the digital culture field big data model; according to the performance of the digital culture field big data model, the parameters of the digital culture field big data model are adjusted; when the performance of the digital culture field big data model meets the specified performance and remains stable, the digital culture field big data model is embedded in the specified cultural product application terminal.

[0152] In a specific application scenario, after the big data model in the digital culture field is trained, the model performance can be evaluated. The evaluation criteria can include commonly used evaluation criteria in the data query field such as recall and precision. If the performance tends to be stable and can meet the specified indicators, it can be deployed. If the ideal indicators cannot be achieved, the parameters of the model need to be adjusted, such as adjusting the structural parameters of the neural network, the loss function, etc., to achieve model optimization.

[0153] After entering the deployment stage, taking cultural enterprises and institutions as an example, Figure 1 As shown, the digital cultural data of such units may be distributed in the hands of one or more users holding the first terminal 101 to the fourth terminal 104. The central processing server 105 is a server of the model developer, which has strong computing power, for example, it has an AI (artificial intelligence) processor, which is used to collect data, build training models, and tune the models to finally obtain a deployable model. Thus, the model is used to classify and manage different types of data files of various digital cultural units.

[0154] Optionally, after training, the model can still be deployed in the central server 105, and different types of data of the user unit can be uploaded to the central processing server 105 for classification. The user can query and manage through the relevant management software on his terminal device (one or more of the first terminal 101 to the fourth terminal 104).

[0155] Alternatively, the model can be deployed in the user unit's own server after training, thereby improving the autonomous controllability of the data.

[0156] Alternatively, if the user unit can also conduct secondary development based on the existing model, build and train a model that meets the requirements of its own unit and deploy it on its own server.

[0157] On the other hand, after classifying the data of the user unit, the present application also provides a question-and-answer system for external users to query and understand the cultural assets of the user unit, and the user can query through his own terminal device (one or more of the first terminal 101 to the fourth terminal 104). Alternatively, the user unit can develop a cultural asset display all-in-one machine with a server and display and input functions, such as placing it in an exhibition hall. In this case, there is no need to equip a dedicated terminal device.

[0158] A second aspect of the present application provides a digital cultural product application terminal, comprising:

[0159] A big data model in the field of digital culture, deployed in the application terminal according to any one of the methods in the first aspect;

[0160] An input device, used for a user to input query information;

[0161] The display device is used to present the query results.

[0162] Specifically, after the model has been deployed, users can start queries based on the deployed model. The query results entered by the user need to be converted into inputs that can match the big data model in the digital cultural field through methods such as semantic recognition. After obtaining the user's input, the trained model can start searching, and after obtaining the search results, it will return the results to the user in the form of a graphical user interface.

[0163] For example, the input device may be an input device such as a keyboard, a mouse, etc., and the output device may be an output device such as a display screen, a projector, etc. Of course, the input device and the output device may also be integrated, such as a touch screen that is commonly used nowadays, which can be used as both an input device and an output device.

[0164] The third aspect of the present application provides a query method based on a big data model in the field of digital culture, characterized by comprising:

[0165] Input query conditions on the application terminal according to the second aspect, generate input suitable for the digital culture field big data model, obtain query results after processing by the digital culture field big data model, and present them to the user.

[0166] Specifically, the application terminal in the second aspect can be in the form of an integrated cabinet commonly seen in current digital cultural units. The backend of such an integrated cabinet can be equipped with one or more servers, and the front end can provide user input and display functions, integrating data collection and model training, deployment, and application in one, which can more conveniently adapt to the various data processing and result display needs of digital cultural units.

[0167] In one embodiment, presenting to the user includes:

[0168] Presenting entity names having edge relationships with the digital culture field knowledge graph on the human-computer exchange interface;

[0169] In response to a user clicking on an entity in the digital culture field knowledge graph, corresponding digital culture field data is presented on the human-computer interaction interface in the form of thumbnails.

[0170] For example, when a user clicks on the entity "Dream of Red Mansions" in the digital culture field map, a specified video frame in the video file, or an image file, or a text summary can be selected and presented to the user as a thumbnail to enhance the user experience.

[0171] It is understandable that in this application, the model can also be tuned based on the user's query feedback. For example, if the user finds that the retrieval results are offset, the weight parameters in the category attention mechanism algorithm can be adjusted to make the attention of the entire model more focused. For another example, the user can adjust the loss function and optimization algorithm through feedback to improve the accuracy of the model.

[0172] It is understandable to those skilled in the art that the present application implements all or part of the processes in the classification method, query method or question-answering method of the above-mentioned embodiments, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0173] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.

Claims

1. A deployment method based on a big data model in the digital culture field, characterized in that: include: Establish a knowledge graph in the field of digital culture; The digital cultural product data of different file types provided by the user are collected, and the digital cultural product data of different file types are subjected to labeling processing by adding data entries; after the labeling processing, noise data, incomplete data, and data irrelevant to the labeling processing are removed from the digital cultural product data of different file types; the types of the digital cultural product data of different file types include text, image, audio and video; According to the different types of digital cultural product data of the different file types, corresponding feature extraction methods are adopted to obtain corresponding structured digital cultural product data feature vectors; based on the structured digital cultural product data feature vectors and the digital culture field knowledge graph, a category attention mechanism algorithm is set to learn multiple different types of data features in parallel through a multi-task method to obtain a digital culture field big data model; a specified loss function and optimization algorithm are selected to optimize the parameters of the digital culture field big data model; Input the pre-reserved validation set and test set into the digital culture domain big data model to evaluate the performance of the digital culture domain big data model; adjust the parameters of the digital culture domain big data model according to the performance of the digital culture domain big data model; and deploy the digital culture domain big data model in the digital culture product application terminal when the performance of the digital culture domain big data model meets the specified performance and remains stable.

2. The method according to claim 1, characterized in that The corresponding feature extraction method is adopted to obtain the corresponding structured digital cultural product data feature vector, including: For text-type cultural product data, semantic word segmentation and understanding are performed through natural language processing methods to convert the text-type cultural product data into text word feature vectors or sentence feature vectors; For image-type cultural product data, feature extraction is performed using a convolutional neural network method to generate an image feature vector representing the image-type cultural product data; For audio-type cultural product data, perform speech recognition analysis and convert it into speech word vectors or sentence vectors through natural language processing methods; For video-type cultural product data, the video frames are processed into image sequences, and the visual features and changes in the time dimension of each frame are extracted to form a video feature vector representing the video-type cultural product data.

3. The method according to claim 1, characterized in that: The method further comprises: Data enhancement operations are performed on the digital cultural product data of the different file types to expand the training set of the big data model in the digital culture field.

4. The method according to claim 1, characterized in that: The method further comprises: Through regularization methods, the big data model in the digital culture field is prevented from overfitting.

5. The method according to claim 3, characterized in that: The performing of data enhancement operation on the digital cultural product data of different file types to expand the training set of the digital cultural field big data model includes: For text-type cultural product data, the structured digital cultural product data feature vector is expanded through semantic expansion; For the image type cultural product data, by comparing similarity, determining whether it is suitable for rotation, cropping or color transformation, and expanding the feature vector of the structured digital cultural product data; For audio type cultural product data, after speech recognition analysis, the structured digital cultural product data feature vector is expanded through natural language expansion; For video type cultural product data, the image sequence converted from the video frame is rotated, cropped or color transformed to expand the structured digital cultural product data feature vector.

6. The method according to claim 1, characterized in that The algorithm for setting the category attention mechanism includes: Different types of cultural product data vectors are assigned different weights according to user specifications and / or query history, thereby obtaining a sorted training set for the digital culture field big data model.

7. The method according to claim 6, characterized in that The algorithm for setting the category attention mechanism also includes: Adjusting the weights through forward propagation and back propagation; Perform normalized weighted scoring on each vector unit in the data vectors of different types of digital cultural products; According to the result of the weighted scoring, a sorted training set of the big data model in the digital culture field is determined.

8. A digital cultural product application terminal, characterized in that: include A big data model in the field of digital culture, deployed in the application terminal according to the method described in any one of claims 1 to 7; An input device, used for a user to input query information; The display device is used to present the query results.

9. A query method based on a big data model in the field of digital culture, characterized in that: include: Input query conditions on the application terminal according to claim 8, generate input suitable for the digital culture field big data model, obtain query results after processing by the digital culture field big data model, and present them to the user.

10. The query method according to claim 9, characterized in that: The presenting to the user comprises: Presenting entity names having edge relationships with the digital culture field knowledge graph on the human-computer exchange interface; In response to a user clicking on an entity in the digital culture field knowledge graph, corresponding digital culture field data is presented on the human-computer interaction interface in the form of thumbnails.

Images