Ethnicity visual audio data ai agent processing system, method and electronic device

Abstract

The application relates to the technical field of broadcast television and network audio-visual supervision, and provides a national language audio-visual data AI intelligent body processing system, method and electronic equipment. The system comprises a large language model, which is used for identifying a user intention according to a user input instruction and to-be-processed national language audio-visual data; a console, which comprises a work flow scheduling unit and is used for scheduling a light-weight model and / or a plug-in corresponding to a task of identifying illegal content in response to the user intention being identified as identifying illegal content, identifying suspected illegal content in the national language audio-visual data through the light-weight model and / or the plug-in; and a multi-modal large model, which is used for performing deep semantic identification and understanding on complex semantic content in the suspected illegal content, obtaining a discrimination result of whether the national language audio-visual data contains illegal content, obtaining a corresponding analysis report, and outputting the discrimination result and the analysis report. The technical scheme of the application can accurately identify illegal content in national language audio-visual data.

Description

Technical Field

[0001] This invention relates to the field of broadcast television and online audiovisual supervision technology, and in particular to an AI intelligent agent processing system, method and electronic device for ethnic language audiovisual data. Background Technology

[0002] Broadcast and television and online audiovisual regulation refers to the control exercised by regulatory authorities over audiovisual content (such as online dramas, short videos, and live streams) disseminated through traditional broadcast and television (such as television stations and radio stations) and the internet. One of the goals of this regulation is the identification of illegal content, such as sensitive audio, sensitive subtitles, sensitive symbols, sensitive faces, and sensitive scenes (blood, violence, pornography). Currently, traditional technologies and methods for broadcast and television and online audiovisual regulation are mainly focused on languages ​​such as Chinese and English. However, they cannot accurately identify illegal content in minority languages ​​such as Tibetan, especially languages ​​with multiple dialects. For example, Tibetan has multiple dialects such as Kham, Ü-Tsang, and Amdo. Identifying illegal content with complex semantics is particularly difficult. Current identification schemes designed for languages ​​such as Chinese and English cannot identify illegal content in Tibetan or other minority languages. Summary of the Invention

[0003] This application provides an AI intelligent agent processing system, method, and electronic device for ethnic minority language audiovisual data, which can effectively solve the technical problem of difficulty in identifying illegal content in ethnic minority languages.

[0004] This application provides an AI intelligent agent processing system for ethnic language audiovisual data. The system includes: a large language model for identifying user intent based on user input and the ethnic language audiovisual data to be processed; a console including a workflow scheduling unit; the workflow scheduling unit, in response to the identified user intent of identifying illegal content, schedules a lightweight model and / or plugin corresponding to the task of identifying illegal content, and identifies suspected illegal content in the ethnic language audiovisual data through the lightweight model and / or plugin; and a multimodal large model for performing deep semantic recognition and understanding of the complex semantic content in the suspected illegal content, obtaining a judgment result on whether the ethnic language audiovisual data contains illegal content, acquiring a corresponding analysis report, and outputting the judgment result and analysis report; wherein the large language model, the multimodal large model, and the lightweight model are all models trained using ethnic language corpora.

[0005] According to an embodiment of this application, an AI intelligent agent processing system for ethnic language audiovisual data includes a workflow scheduling unit. This unit, in response to the user's intent to obtain ethnic language audiovisual data within a target range, schedules a lightweight model and / or plugin corresponding to the intelligent compilation task. Through the lightweight model and / or plugin, it obtains ethnic language audiovisual data that conforms to the target range. The system also includes a multimodal large model comprising an ethnic language intelligent cataloging module, a multimodal ethnic language content retrieval module, and a business data intelligent compilation module. The ethnic language intelligent cataloging module determines cataloging information for ethnic language audiovisual data within the target range based on the user's intent. The multimodal ethnic language content retrieval module retrieves at least one piece of ethnic language audiovisual data that matches the type and belongs to the target range based on the cataloging information. The business data intelligent compilation module intelligently compiles and generates corresponding ethnic language audiovisual content from at least one piece of ethnic language audiovisual data.

[0006] According to an embodiment of this application, an AI intelligent agent processing system for ethnic language audiovisual data is provided. The system further includes a lightweight model subsystem. The lightweight model subsystem includes one or more of the following: an ethnic language identification model, an ethnic language speech recognition model, an ethnic language OCR recognition model, an ethnic language text translation model, and a sensitive content detection model. The ethnic language identification model is used to identify the language of the ethnic language audiovisual data to be processed based on convolutional neural networks and recurrent neural networks. The ethnic language speech recognition model is trained using speech recognition corpora of ethnic languages ​​and their corresponding dialects, and is used to perform dialect-supported speech recognition on the ethnic language audiovisual data to be processed using end-to-end speech recognition technology. The ethnic language OCR recognition model is used to perform character recognition on the ethnic language audiovisual data to be processed based on deep learning optical character recognition technology. The Sino-Tibetan text translation model is used to perform ethnic language to Chinese text translation conversion on the ethnic language audiovisual data to be processed based on deep learning text translation technology. The illegal content detection model is used to detect illegal content on the ethnic language audiovisual data to be processed based on a deep neural network model that integrates multimodal information.

[0007] According to an embodiment of this application, an AI intelligent agent processing system for ethnic language audiovisual data is provided. The system further includes an ethnic language corpus annotation and training platform. The ethnic language corpus annotation and training platform includes a model training and evaluation module. The model training and evaluation module is used to train a target model based on multiple dialect training sets constructed from corpora of different dialects of an ethnic language, so as to obtain a target model that supports processing different dialects under the same ethnic language. The target model includes at least one of a large language model, a multimodal large model, and a lightweight model.

[0008] According to an embodiment of this application, an AI intelligent agent processing system for ethnic language audiovisual data is provided. The ethnic language corpus annotation and training platform further includes at least one of an intelligent annotation module, a multimodal corpus production module, and a quality standard and control module. The intelligent annotation module employs an iterative processing mode of data filtering and automatic machine recognition. Data filtering includes automatic recognition of ethnic language speech and automatic translation of ethnic language text to obtain ethnic language materials. Automatic machine recognition includes pre-annotating the obtained ethnic language materials to obtain annotated training corpus data. The multimodal corpus production module is used to construct various types of corpus data. Source database; multiple types of corpus resources including at least one of ethnic language speech recognition corpora, language identification corpora, and text translation corpora; quality standards and control module, used for digital management of the entire process of data collection, annotation, and review, and to control the corpus annotation error rate; model training and evaluation module, which is also used to automatically divide the annotated training corpus data into training set, development set, and test set according to certain criteria. The training set is used for model training; the development set is used to verify the model effect during the training phase and to make targeted adjustments to the trained model based on the results of the development set; the test set is used to verify the effect of the trained model.

[0009] According to an embodiment of this application, an AI intelligent agent processing system for ethnic language audiovisual data is provided. The system also includes a knowledge base, which includes a violation knowledge base and a violation historical case base for ethnic language audiovisual programs or content. The violation knowledge base includes at least one category of violation knowledge bases, such as people, images, videos, text, and audio. The knowledge base is implemented using vector database technology and supports similarity retrieval and semantic matching retrieval.

[0010] According to an embodiment of this application, an AI intelligent agent processing system for ethnic language audiovisual data is provided, with plugins including at least one of the following: a web crawler plugin for actively searching and collecting suspected illegal ethnic language audiovisual data on the Internet; a content review plugin for automatically reviewing and marking ethnic language audiovisual data; a cataloging generation plugin for generating standardized radio and television program cataloging information; and a report generation plugin for automatically generating analysis reports based on the judgment results.

[0011] This application also provides an AI intelligent agent processing method for ethnic language audiovisual data. The method includes: using a large language model to identify user intent based on user input instructions and the ethnic language audiovisual data to be processed; in response to the identified user intent being to identify illegal content, scheduling a lightweight model and / or plugin corresponding to the task of identifying illegal content, and using the lightweight model and / or plugin to identify suspected illegal content in the ethnic language audiovisual data; using a multimodal large model to perform deep semantic recognition and understanding on the complex semantic content in the suspected illegal content, obtaining a judgment result on whether the ethnic language audiovisual data contains illegal content, obtaining a corresponding analysis report, and outputting the judgment result and analysis report; wherein, the large language model, the multimodal large model, and the lightweight model are all models trained using ethnic language corpora.

[0012] According to an embodiment of this application, an AI intelligent agent method for processing ethnic language audiovisual data is provided. This method may further include: responding to the recognition that a user's intent is to obtain ethnic language audiovisual data within a target range, scheduling a lightweight model and / or plugin corresponding to the intelligent compilation task, and obtaining ethnic language audiovisual data within the target range through the lightweight model and / or plugin; determining the cataloging information of the ethnic language audiovisual data within the target range based on the user's intent using a multimodal large model, and retrieving at least one piece of ethnic language audiovisual data that matches the type and belongs to the target range based on the cataloging information; and intelligently compiling and reporting the at least one piece of ethnic language audiovisual data to intelligently generate corresponding ethnic language audiovisual content.

[0013] This application also provides an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the ethnic language audiovisual data AI intelligent agent processing system as described above.

[0014] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an AI intelligent agent processing system for ethnic language audiovisual data as described above.

[0015] This application also provides a computer program product, including a computer program that, when executed by a processor, implements an AI intelligent agent processing system for ethnic language audiovisual data as described above.

[0016] The AI ​​intelligent agent processing system, method, and electronic device for ethnic language audiovisual data provided in this application embodiment are based on a basic architecture combining a large language model (intent recognition), a multimodal large model (complex semantic understanding), and a lightweight model (a dedicated model for identifying a specific type of illegal content). First, based on the user's input instructions and the ethnic language audiovisual data to be processed, the system identifies the user's intent. Then, the console schedules the corresponding lightweight model and / or plugins to identify suspected illegal content in the ethnic language audiovisual data. The lightweight model can be a dedicated model set up for different categories of content, which can improve the detection accuracy of illegal content for a specific type of content, achieving "specialized and precise" extraction of suspected illegal content. The suspected illegal content is then fed back to the multimodal large language model for deep semantic understanding. The multimodal large model trained with ethnic language corpus has the ability to deeply understand the complex semantics in ethnic languages, accurately identifying metaphors and difficult-to-identify illegal content that may be contained within. Thus, this solution can identify illegal content in audiovisual data of Tibetan and other minority languages. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is an example diagram of the system architecture of the AI ​​intelligent agent processing system for ethnic language audiovisual data provided in the embodiments of this application; Figure 2 This is another example diagram of the system architecture of the AI ​​intelligent agent processing system for ethnic language audiovisual data provided in the embodiments of this application; Figure 3 This is a flowchart illustrating the method for processing ethnic language audiovisual data using an AI intelligent agent, as provided in an embodiment of this application. Figure 4 This is a flowchart illustrating one embodiment of the AI ​​intelligent agent processing method for ethnic language audiovisual data provided in this application. Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0021] Currently, the technologies and methods for monitoring and regulating broadcast television and online audiovisual content mainly focus on languages ​​such as Chinese and English. Among these technologies, specialized models based on artificial intelligence automatically analyze program content for potential violations. These violations typically include the identification of sensitive audio content, sensitive subtitle content, sensitive landmarks, sensitive facial recognition, and sensitive scenes (such as gore, violence, and pornography).

[0022] Although the relevant technologies have been applied to some extent, they all have significant shortcomings and cannot meet the current practical needs for efficient, accurate, and large-scale discovery of broadcast television and online audiovisual content in specific minority languages ​​(such as Tibetan). Specifically, the technical problems are as follows: (1) Inability to accurately identify illegal (or sensitive) content in ethnic minority language broadcast television and online audiovisual programs: Ethnic minority languages ​​(hereinafter referred to as ethnic languages) usually contain different dialect variations. For example, Tibetan includes three dialects: Kham, Ü-Tsang, and Amdo. Traditional speech recognition and subtitle recognition technologies are difficult to accurately handle these dialect differences. In particular, for complex semantic content such as metaphors and sarcasm in ethnic minority languages, the relevant technologies lack in-depth language understanding and contextual reasoning capabilities, and cannot achieve accurate semantic analysis and sentiment judgment.

[0023] (2) Insufficient ability to segment and catalog radio and television programs in ethnic languages: The segmentation and cataloging of radio and television programs, as well as news items, requires in-depth semantic content analysis and identification. Traditional methods are mainly based on simple rules and keyword matching, lacking a deep semantic understanding of ethnic language content, resulting in low segmentation accuracy and cataloging quality that cannot meet the needs of practical applications.

[0024] (3) Lack of adaptive learning capability for ethnic minority languages: The relevant technologies do not have continuous learning capabilities, while ethnic minority languages, especially different dialects and program types (such as Tibetan TV dramas), have problems such as background noise interference and non-standard accents, requiring the system to have corpus annotation and adaptive learning capabilities. Traditional methods require a large amount of labeled data to retrain the model, which cannot quickly adapt to new language variants and program formats, resulting in a significant drop in recognition accuracy in practical applications.

[0025] (4) Insufficient multimodal fusion analysis capability for ethnic minority languages: Although the monitoring and supervision system in the relevant technologies has a certain multimodal analysis capability, in the context of ethnic minority languages, the ability to fuse and analyze various information such as voice, subtitles, and images is weak, making it difficult to achieve cross-modal collaborative understanding and comprehensive analysis, which limits the overall improvement of the supervision effect.

[0026] In view of this, embodiments of this application propose an AI intelligent agent processing system, method, and electronic device for ethnic language audiovisual data, in order to solve at least one of the above-mentioned technical problems.

[0027] The ethnic language audiovisual data AI intelligent agent processing system, method, and electronic device proposed in this application construct a monitoring and supervision intelligent agent that integrates ethnic language corpus annotation, optimization and training of ethnic language-specific AI recognition models, language identification, speech recognition, OCR subtitle recognition, ethnic language-Chinese text translation, and the complex semantic cognition capabilities of AI large language models. Based on this intelligent agent, content identification is performed on the audiovisual data objects that users need to identify, accurately determining whether there is any illegal content. Furthermore, in some embodiments, intelligent program compilation is also supported, that is, users can input instructions, and the intelligent agent will automatically retrieve ethnic language audiovisual content that meets the user's needs, intelligently compile it, and return it to the customer. This intelligent agent adopts a multi-integrated technical architecture that includes at least "dual AI large models + AI-specific small models + business systems." Through a dual-core driving mode of "deep inference large language models" and "multimodal large models," combined with ethnic language-specific AI models, it realizes the digital mapping of human "perception," "decision-making," "memory," and "action" behavior patterns, ultimately constructing an intelligent monitoring and supervision capability for ethnic language audiovisual program content.

[0028] It should be noted that the ethnic language mentioned in the embodiments of this application can be one or more of the following ethnic languages: Tibeto-Burman languages ​​include: Tibetan (with three major dialects: Ü-Tsang, Kham, and Amdo), Monpa, Lhoba, Gyarong, Qiang, Pumi, Dulong, Nu, Yi, Lisu, Naxi, Hani, Lahu, Bai, Jino, Jingpo, Zaiwa, Achang, Tujia, Queyu, Baima, Zaba, Namuyi, Ersu, Guiqiong, Shixing, Muya, Ergong, and La'urong, among others (multiple languages ​​spoken by Tibetans). Miao-Yao language family: Miao, Bunu, Mian, She, Baheng, Jiongnai, and Younuo; The Zhuang-Dong language family (also known as the Dong-Tai language family) includes: Zhuang, Buyi, Dai, Dong, Shui, Mulam, Maonan, Li, Gelao, Lajia, and Mo; Altaic language family: Turkic group: Uyghur, Kazakh, Kyrgyz, Uzbek, Tatar, Salar, Western Yugur, Tuwa; Mongolian language family: Mongolian, Daur, Dongxiang, Eastern Yugur, Tu, Bao'an, Tangwang; Manchu-Tungusic language family: Manchu, Xibe, Hezhe, Evenki, Oroqen; Austronesian languages: The languages ​​of the Gaoshan people include more than ten languages ​​such as Amis, Paiwan, Bunun, Atayal, Saisiyat, Rukai, Puyuma, Thao, Tsou, Shaarua, Kanakanafu, Kavalan, and Yami; Huihui language. Austroasiatic languages: It belongs to the Mon-Khmer language family, Wa language; De'ang language, Blang language, and Khmer language; Indo-European languages: Russian, Tajik; And Korean and Kyung, whose affiliation is undetermined.

[0029] The solutions proposed in this application can be applied to any of the above-mentioned ethnic languages. The following description mainly uses Tibetan as an example. It should not be understood that the solutions proposed in this application can only be applied to Tibetan.

[0030] like Figure 1 As shown in the embodiments of this application, the AI ​​intelligent agent processing system for ethnic language audiovisual data includes at least a large artificial intelligence model (equivalent to the brain) for ethnic languages ​​and a control console (equivalent to limbs and tentacles). Optionally, it may also include a lightweight model (sensory organ) specifically for ethnic languages, or related plugins may be deployed.

[0031] The AI ​​large-scale model employs a dual-model design, including a large language model for understanding user intent and a multimodal model capable of recognizing complex semantics. The console, when the large language model or multimodal model performs inference tasks, calls various plugins or lightweight models according to a pre-designed workflow scheduling strategy, based on different task categories, to assist the large model in completing inference tasks. The lightweight models are specialized models for acquiring suspected illegal content. Tasks such as ethnic language recognition, speech recognition, OCR subtitle recognition, translation, and illegal content detection are performed through dedicated lightweight models to improve the ability to perform each specialized task. The multimodal large model accurately identifies the suspected illegal content captured by the lightweight model and provides a judgment result on whether it contains illegal content. In other words, the lightweight model is used to acquire suspected illegal content, while the multimodal large model is used to accurately determine whether illegal content is contained.

[0032] Specifically, the large language model is used to identify user intent based on user input instructions and the ethnic language audiovisual data to be processed.

[0033] It should be noted that the AI ​​agent proposed in the embodiments of this application (e.g., Figure 1 The AI ​​agent shown can be integrated into the business system for monitoring and supervising ethnic language broadcasting and television and online audiovisual media. This business system includes a user interface, in which a human-computer interaction interface can be set. Through this human-computer interaction interface, the user can call the AI ​​agent provided in this application embodiment to achieve accurate identification of illegal content and intelligent program editing, etc.

[0034] For example, such as Figure 1 As shown, users can input commands and data into the human-computer interaction interface. The data can be audio-visual data to be processed, including multimodal data such as audio, video, images, and text. Commands can be of various types, such as commands to query illegal content, intelligent program editing, or other functional commands. For example, if the audio-visual data to be processed is a local TV station's news program, the user can input the video file (attachment) of the news program into the human-computer interaction interface and input the corresponding command "Please identify whether the attachment contains illegal or sensitive content." The large language model will perform intent recognition based on the user's input, and if it finds that the user's intent is to identify illegal content, it will trigger the workflow for the illegal content task in the console.

[0035] Alternatively, users can enter the name of the object they want to query through the human-computer interaction interface, without having to upload multimodal data such as videos. For example, if a user enters "Please identify whether the XXX Tibetan program broadcast by XX Tibetan TV station contains illegal content" in the human-computer interaction interface, the big language model will perform intent recognition, determine that the task category is illegal content recognition, and that it needs to retrieve the specific content of the XXX Tibetan program broadcast by XX Tibetan TV station. Therefore, the big language model will trigger the console to schedule the retrieval task.

[0036] The console may include a workflow scheduling unit. In response to the user's intent to identify illegal content, the workflow scheduling unit schedules lightweight models and / or plugins corresponding to the illegal content identification task, and uses these lightweight models and / or plugins to identify suspected illegal content in ethnic language audiovisual data.

[0037] Optionally, in some embodiments, the AI ​​intelligent agent processing system for ethnic language audiovisual data proposed in this application is equipped with a lightweight model subsystem (i.e., Figure 1 The lightweight model subsystem (shown as a special model for ethnic minority languages) may include one or more of the following: ethnic minority language identification model, ethnic minority language speech recognition model, ethnic minority language OCR recognition model, ethnic minority language text translation model, and sensitive content detection model.

[0038] Among them, the ethnic language identification model is used to identify the language of the ethnic language audiovisual data to be processed based on convolutional neural networks and recurrent neural networks; the ethnic language speech recognition model is trained using speech recognition corpora of ethnic languages ​​and their corresponding dialects, and is used to perform dialect-supported speech recognition on the ethnic language audiovisual data to be processed using end-to-end speech recognition technology; the ethnic language OCR recognition model is used to perform character recognition on the ethnic language audiovisual data to be processed based on deep learning optical character recognition technology; the Sino-Tibetan text translation model is used to perform text translation conversion from ethnic language to Chinese on the ethnic language audiovisual data to be processed based on deep learning text translation technology; and the illegal content detection model is used to detect illegal content on the ethnic language audiovisual data to be processed based on a deep neural network model that integrates multimodal information.

[0039] For example, if the console triggers a task categorized as "violation content identification" and the user uploads a corresponding attachment, the workflow for this task is as follows: The lightweight model uses a minority language identification model to identify the language category of the attachment; after language identification, a minority language speech recognition model is used to identify the speech in the attachment, and a minority language OCR subtitle model is used to perform OCR recognition on the content of the attachment. Through speech and subtitle recognition, the text content in the attachment can be identified. If the attachment contains images or difficult-to-recognize characters, a minority language image-text retrieval model can be used to retrieve the meaning of the image or character. The identified text content and image meaning are then input into a minority language / Chinese text translation model to translate the identified text content and image meaning into corresponding Chinese text. This Chinese text is then input into a violation content detection model to preliminarily identify whether the Chinese text contains violation content and outputs target modal data that may contain violation content. For example, the target modal data may be text. The data of suspected violation content in the target modality is then fed back to a multimodal large model for accurate identification.

[0040] Alternatively, the console can invoke a plugin or lightweight model with search functionality to search for the specific content of the XXX Tibetan language program broadcast by XX Tibetan Language TV Station. After the search, each lightweight model is sequentially scheduled to perform language recognition, speech recognition, subtitle recognition, image text retrieval, text translation, and illegal content detection, and output suspected illegal content.

[0041] The multimodal large model is used to perform deep semantic recognition and understanding of complex semantic content in suspected illegal content, obtain the judgment result of whether the ethnic language audiovisual data contains illegal content, and obtain the corresponding analysis report, outputting the judgment result and analysis report.

[0042] For example, a multimodal ethnic language content intelligent review module is set up in the multimodal large model to perform deep semantic recognition on suspected illegal content reported by the lightweight model, especially to perform accurate reasoning on complex semantics, and output the judgment result of whether the attachments or XXX Tibetan language programs broadcast by XX Tibetan language TV station that the user needs to query contain illegal content.

[0043] Among them, the large language model, the multimodal large model, and the lightweight model are all models trained using ethnic language corpora.

[0044] Obtaining the analysis report can be achieved by using a wrapper to call the corresponding format conversion plugin and / or analysis report generation plugin based on the data output by the large model, to obtain the analysis report in the target format, such as Word document format or Excel chart format. Plugins with corresponding functions can be called to wrap and convert the data output by the multimodal large model to obtain the analysis report in Word document format.

[0045] It should be noted that the above exemplary description is only based on the intelligent agent provided in the embodiments of this application. It not only supports the task of handling illegal content, but also supports other tasks such as intelligent program editing in some embodiments.

[0046] For example, a user inputs a voice message through the human-computer interaction interface of a business system: "When the large language model performs intent recognition, it first calls the ethnic language identification model to identify the language, which is Tibetan. Then it calls the ethnic language speech recognition model to recognize the user's input speech and obtain the corresponding Tibetan text based on the speech." Then, the ethnic language / Chinese translation model is invoked to translate the Tibetan text. The translation yields the corresponding Chinese text "Please help me find news about XXX in the past week". This Chinese text is then input into a large language model for intent recognition, and the task category is determined to be an intelligent editing task (or intelligent program editing task).

[0047] Optionally, the multimodal large model can also include a cataloging module for ethnic minority language programs, a multimodal intelligent review module for ethnic minority language content, and an intelligent reporting module for business data. Among them, the ethnic minority language program cataloging module is used to intelligently segment and catalog various programs from various regulated entities (broadcasting stations, online platforms, etc.), and to classify and catalog them according to the content, program category, broadcast time, etc.

[0048] After determining the task category as intelligent editing and reporting, the console calls the ethnic language program cataloging module in the multimodal big data model to select target programs that meet the user's requirements from multiple programs. For example, it determines the cataloging information of news programs about XXX in the past week. Then, based on the cataloging information, it calls the multimodal ethnic language content intelligent retrieval module to search for news programs in the past week. The retrieved program content may be messy and disordered fragments. Then, by calling the business data intelligent editing and reporting module, it intelligently edits and reports the retrieved news program content, automatically arranging multiple retrieved news program materials. After arrangement, the encapsulator generates program content that can be directly viewed by the user.

[0049] Program cataloging refers to the process of describing, analyzing, and classifying the content characteristics, external form, and management attributes of radio and television programs or online audiovisual content according to certain standards and norms, and forming structured data.

[0050] Based on the above exemplary description, it can be seen that in some embodiments, the workflow scheduling unit in the console is further configured to, in response to recognizing that the user's intent is to obtain ethnic language audiovisual data within a target range, schedule a lightweight model and / or plugin corresponding to the intelligent reporting task, and obtain ethnic language audiovisual data that conforms to the target range through the lightweight model and / or plugin. For example, the target range includes a time range and / or a category range. For example, in the above example, "within the past week" is a time range, and "news" belongs to the category range.

[0051] The multimodal big model can further include an intelligent cataloging module for ethnic languages, a multimodal ethnic language content retrieval module, and an intelligent reporting module for business data. The intelligent cataloging module for ethnic languages ​​is used to determine the cataloging information of ethnic language audiovisual data within a target range based on user intent. The multimodal ethnic language content retrieval module is used to retrieve at least one piece of ethnic language audiovisual data that matches the type and falls within the target range based on the cataloging information. The intelligent reporting module for business data is used to intelligently report at least one piece of ethnic language audiovisual data, intelligently generating corresponding ethnic language audiovisual content, such as various programs that can be broadcast on radio and television stations or online platforms.

[0052] Cataloging information can include descriptive information (what the program is), technical and administrative information (how the program is managed and used), and structural information (how the program is broken down).

[0053] For example, descriptive information includes at least one of the following: program name (e.g., "XX News Broadcast", "Tibet XX"), subtitle, series title, etc., content description, content summary, keywords / theme terms (e.g., "rural revitalization", "Tibetan language", "Kham dialect"), classification information (what type of program it is, such as news, TV series, documentary, variety show, animation, advertisement, etc.), language information (the language used in the program (e.g., Tibetan dialect, Mandarin Chinese, Mongolian)).

[0054] For example, technical and management information used for the storage, retrieval, and broadcast control of media asset systems may include at least one of the following: Identification information: Unique program number (e.g., UMID), barcode, file number; Date information: Production date, broadcast date, copyright date, and data entry date; Format information; Duration; Video parameters: Resolution (4K / HD / SD), aspect ratio; Audio parameters: Number of channels, sampling rate; Copyright information: Copyright owner, copyright validity period, and scope of authorization.

[0055] Figure 1 The AI ​​intelligent agent system shown (Ethnic Minority Language Audiovisual Data AI Intelligent Agent Processing System) is an intelligent analysis system that combines corpus annotation for multiple Tibetan dialects, speech recognition model optimization and training, language identification, speech recognition, OCR subtitle recognition, Sino-Tibetan text translation, and the complex semantic cognitive capabilities of a large language model. This system achieves accurate understanding and analysis of ethnic minority language broadcast and television content through deep learning and multimodal fusion technology. It leverages the powerful generalization and reasoning capabilities of a large language model, combined with the accuracy of a dedicated lightweight model on specific tasks, forming a complementary analytical framework.

[0056] also, Figure 1 The system shown is an embodied intelligent agent architecture for monitoring and supervising ethnic minority language audiovisual programs. Addressing the lack of embodiment in large-scale model technology, this application's embodiments design and develop a content supervision intelligent agent for ethnic minority language audiovisual programs. It constructs a dual-core driven mode of a "deep reasoning large language model" and a "multimodal large model," combined with dedicated intelligent perception models for ethnic minority languages ​​(such as Tibetan language recognition, speech recognition, OCR recognition, and Sino-Tibetan text translation models). By simulating human perception and cognitive processes, it organically combines perception, decision-making, memory, and action, enabling the intelligent agent to interact with the actual physical environment and solve complex problems in real-world ethnic minority language audiovisual program supervision scenarios.

[0057] This application also proposes an adaptive learning and capability evolution mechanism. This scheme aims to construct a continuous optimization and evolution framework for recognition models of low-resource ethnic minority languages ​​(with Tibetan as a typical example). The core of this mechanism is to build an ethnic minority language corpus annotation and training platform that integrates intelligent corpus annotation, model optimization training, and evaluation. By incorporating advanced algorithms such as reinforcement learning and multi-task collaborative training, it significantly improves the performance, robustness, and scene adaptability of specialized models for Tibetan language recognition, speech recognition, and Sino-Tibetan text translation.

[0058] Here is another specific example.

[0059] like Figure 2 As shown, in one specific embodiment, the ethnic language audiovisual data AI intelligent agent processing system may include an artificial intelligence large model for ethnic languages ​​(equivalent to the brain), a control console (equivalent to limbs and tentacles), a lightweight model specifically for ethnic languages ​​(sensory organs), an ethnic language corpus annotation and training platform (learning ability), and a vector knowledge base for regulatory domains (memory ability).

[0060] In this specific embodiment, the multimodal AI large model ("brain") (including a large language model and a multimodal large model): through reinforcement learning of regulatory domain expertise, the system possesses the cognitive ability to recognize general knowledge in multimodal data (image, text, audio, and video), while also possessing specialized cognitive abilities in the regulatory domain. The multimodal AI large model acquires deep language understanding capabilities through pre-training on a large amount of ethnic language corpus. In specific implementation, a large-scale corpus containing Tibetan dialects such as Kham, Ü-Tsang, and Amdo is first constructed, pre-trained using a self-supervised learning method, and then fine-tuned using labeled data from the regulatory domain.

[0061] In this specific embodiment, the ethnic language-specific AI mini-model ("sensory organ") uses dedicated mini-models such as speech recognition and subtitle recognition as the "sensory organs" of the intelligent agent. Optionally, it may also include a face recognition model, which is used to specifically identify whether there are illegal facial images in the audiovisual content to be processed. The above-mentioned lightweight models can assist the large model in better and more accurately perceiving the content in audiovisual programs.

[0062] For example, taking Tibetan as an example, the language identification model includes language identification models corresponding to different dialects under minority languages ​​to support the identification of different dialects. For example, the multilingual dialect identification model of Tibetan dialects (Kham, Ü-Tsang, Amdo) is based on convolutional neural networks and recurrent neural networks to achieve accurate differentiation of different Tibetan dialects.

[0063] For example, a convolutional neural network can be a CNN (Convolutional Neural Network) or a ResNet (Residual Network); a recurrent neural network can be an RNN (Recurrent Neural Network), an LSTM (Long Short-Term Memory Network), or a GRU (Gated Recurrent Unit).

[0064] Similarly, speech recognition models can also support the separate recognition of different dialects under minority languages. For example, a speech recognition model can be a speech recognition model for multiple Tibetan dialects (Kham, Ü-Tsang, Amdo): using end-to-end speech recognition technology, utilizing the mastered Tibetan dialect speech recognition data (broadcast and television scenarios, about 1500 hours of data), and performing reinforcement learning on the characteristics of each Tibetan dialect in broadcast and television scenarios.

[0065] Taking Tibetan as an example, the OCR recognition model can be a Tibetan OCR recognition model: an optical character recognition technology based on deep learning, specifically designed for the characteristics of Tibetan characters.

[0066] Ethnic language / Chinese translation model, which can be a Sino-Tibetan text translation model: a text translation technology based on deep learning that supports text translation from Tibetan to Chinese.

[0067] Sensitive content detection models can be deep neural network models that integrate multimodal information.

[0068] In this specific embodiment, a regulatory domain vector knowledge base ("memory area") is also deployed. The knowledge base includes various types of violation knowledge bases for ethnic language audiovisual programs, such as people, images, videos, text, and audio, as well as a historical violation case database. The knowledge base is implemented using vector database technology, supporting efficient similarity retrieval and semantic matching.

[0069] In this specific embodiment, a console is also deployed. The console serves as the central control platform for the business system. Through a workflow invocation module, it assists the large model in controlling and invoking various application system plugins to simulate human behavior and complete various tasks such as actual regulatory or intelligent reporting. The console includes an interface for invoking the aforementioned lightweight model and can further be configured to invoke the following plugins: Web crawler plugin: Used to actively search for and collect suspected illegal audiovisual content on the Internet; Content moderation plugin: Enables automated content moderation and flagging; Cataloging generation plugin: Automatically generates standardized radio and television program cataloging information; Report generation plugin: Automatically generates regulatory reports based on analysis results.

[0070] The load balancing scheduling module in the console is used to manage and balance the computing resources required by different types of tasks. For example, user-initiated tasks are high-priority tasks, such as the aforementioned illegal content identification task and intelligent reporting task. Routine tasks are low-priority tasks. When there is a conflict between the computing resources required by the two types of tasks, the high-priority task is executed first.

[0071] The compute node management module is used to manage the multiple compute node resources held by the system.

[0072] The status monitoring and management module is used to monitor the status of various objects, such as tasks, and to monitor the execution status of different tasks.

[0073] A plug-in computing environment is used to manage various plug-ins, and to determine whether the system's current remaining computing resources can allow the plug-in to be called based on the computing power required by each plug-in.

[0074] The consoles of business systems such as configuration management, external application systems, data acquisition systems, and data interaction components are used for daily configuration information maintenance and operation, as well as for interaction with external systems. These components will not be elaborated here.

[0075] In this specific embodiment, a language corpus annotation and training platform ("learning ability") is also deployed.

[0076] This platform, serving as the core learning and evolution hub for intelligent agents, addresses pain points such as the difficulty in acquiring ethnic minority language corpora, the high requirements for professional annotation, and the long model training cycle. It constructs a sustainable learning system integrating corpus collection, annotation, quality control, model training, and evaluation. Its main technical implementations are as follows: Intelligent Annotation System: This platform adopts an iterative processing model of "data filtering, automatic machine recognition, and manual review," establishing a comprehensive workflow management system. The platform first performs machine pre-annotation of input Tibetan language program materials through automatic Tibetan speech recognition and automatic Chinese-Tibetan text translation. Then, manual review and modification are performed, thereby automating and intelligently annotating Tibetan language corpora and significantly reducing the workload of manual annotation.

[0077] Multimodal Corpus Production: This platform enables the construction of various types of corpus resources, including Tibetan speech recognition corpora, language identification corpora, and Sino-Tibetan text translation corpora. These corpora will be used for subsequent optimization and training of Tibetan AI recognition models, having a profound impact on the recognition, transcription, large-scale model semantic understanding and analysis, and content censorship of Tibetan programs in the future.

[0078] Quality Standards and Control: The platform adopts a modular design to achieve digital management of the entire process of data collection, annotation, and review. It provides quality control mechanisms such as review and spot checks to strictly control the error rate of corpus annotation, significantly improving the quality and efficiency of corpus construction.

[0079] Model Training and Evaluation: The platform automatically divides the labeled training corpus data into training, development, and test sets according to certain criteria. The training set is used for model training; the development set is used to verify the model's effectiveness during the training phase and to make targeted adjustments to the trained model based on the results of the development set; the test set is used to verify the effectiveness of the trained model. The platform can simultaneously support the optimization training and evaluation of language recognition, speech recognition, and text translation models. Before a model is deployed online, each model can select a matching test set to verify the completed training. Only after meeting the expected requirements can the model be deployed online, thus ensuring that the output AI models for various ethnic languages ​​meet the requirements of actual regulatory applications in terms of accuracy, efficiency, and adaptability.

[0080] Based on the above exemplary description, it can be obtained that, in Figure 1Based on the system example, in this application embodiment, the plugins for console invocation include at least one of the following plugins: a web crawler plugin for actively searching and collecting suspected illegal ethnic language audiovisual data on the Internet; a content review plugin for automatically reviewing and marking ethnic language audiovisual data; a cataloging generation plugin for generating standardized radio and television program cataloging information; and a report generation plugin for automatically generating analysis reports based on the discrimination results.

[0081] Based on the above exemplary description, it can be obtained that, in Figure 1 Building upon existing system examples, the system proposed in this application also includes a minority language corpus annotation and training platform. This platform includes a model training and evaluation module. This module trains the target model using a multi-dialect training set constructed from corpora of different dialects of a single minority language, to obtain a target model that supports processing different dialects of the same minority language. The target model can be at least one of a large language model, a multimodal large model, and a lightweight model.

[0082] Based on the above exemplary description, it can be obtained that, in Figure 1 Based on the existing system example, the ethnic language corpus annotation and training platform also includes at least one of the following: an intelligent annotation module, a multimodal corpus production module, and a quality standard and control module. The intelligent annotation module employs an iterative processing mode of data filtering and automatic machine recognition. Data filtering includes automatic recognition of ethnic language speech and automatic translation of ethnic language text to obtain ethnic language materials; automatic machine recognition includes pre-annotating the obtained ethnic language materials to obtain annotated training corpus data. The multimodal corpus production module is used to construct various types of corpus resource libraries, including at least one of ethnic language speech recognition corpus, language identification corpus, and text translation corpus. The quality standard and control module is used for digital management of the entire process of data collection, annotation, and review, controlling the corpus annotation error rate. The model training and evaluation module automatically divides the annotated training corpus data into training, development, and test sets according to certain criteria. The training set is used for model training; the development set is used to verify the model's effectiveness during the training phase and to make targeted adjustments to the trained model based on the results of the development set; the test set is used to verify the effectiveness of the trained model.

[0083] Based on the above exemplary description, it can be obtained that, in Figure 1 Based on the system examples, the system proposed in this application embodiment also includes a knowledge base, which includes a violation knowledge base and a violation history case base for ethnic language audiovisual programs or content; wherein, the violation knowledge base includes at least one category of violation knowledge bases among people, images, videos, text, and audio.

[0084] The knowledge base is implemented using vector database technology and supports similarity retrieval and semantic matching retrieval.

[0085] The method for processing ethnic language audiovisual data using AI intelligent agents, as described in the embodiments of this application, can be referred to in correspondence with the system for processing ethnic language audiovisual data using AI intelligent agents described above, and can achieve the same or corresponding technical effects.

[0086] This application provides a method for processing ethnic language audiovisual data using AI intelligent agents. This method can be based on... Figure 1 or Figure 2 The system implementation is shown.

[0087] Specifically, such as Figure 3 As shown in the embodiments of this application, a method for processing ethnic language audiovisual data using AI intelligent agents may include the following process: Step 301: Using a large language model, identify the user's intent based on the user's input instructions and the ethnic language audiovisual data to be processed.

[0088] Step 302: In response to the user's intent to identify illegal content, schedule the lightweight model and / or plugin corresponding to the task of identifying illegal content, and identify suspected illegal content in the ethnic language audiovisual data through the lightweight model and / or plugin.

[0089] Step 303: Using a multimodal large model, perform deep semantic recognition and understanding on the complex semantic content in the suspected illegal content, obtain the judgment result of whether the ethnic language audiovisual data contains illegal content, obtain the corresponding analysis report, and output the judgment result and analysis report.

[0090] Optionally, in some embodiments, according to Figure 3 The method embodiments shown are as follows: Figure 4 As shown, this method may further include the following steps: Step 401: In response to the recognition that the user's intention is to obtain ethnic language audiovisual data within the target range, the lightweight model and / or plugin corresponding to the intelligent compilation task is scheduled, and the ethnic language audiovisual data that meets the target range is obtained through the lightweight model and / or plugin.

[0091] Step 402: Determine the cataloging information of ethnic language audiovisual data within the target range according to the user's intent, and retrieve at least one piece of ethnic language audiovisual data that matches the type and belongs to the target range based on the cataloging information; intelligently compile and report at least one piece of ethnic language audiovisual data, and intelligently generate corresponding ethnic language audiovisual content.

[0092] In summary, the AI ​​intelligent agent processing system and method for ethnic language audiovisual data proposed in this application can achieve the following technical effects: Technical benefit 1: Breaking through the technical bottleneck of identifying illegal content in ethnic minority languages, achieving precise content supervision.

[0093] The solution proposed in this application effectively addresses the shortcomings of traditional technologies in recognizing ethnic language content by constructing a multimodal analysis system specifically for ethnic languages. Particularly in the case of multiple dialects such as Tibetan, the system can accurately identify and understand the speech and text content of different dialects, and possesses a deep understanding of complex semantic content such as metaphors and irony. Compared with existing technologies, this solution significantly improves the accuracy of ethnic language content recognition in broadcast television scenarios.

[0094] Technical effect 2: It has the ability to continuously evolve, significantly improving the model performance and adaptation efficiency in low-resource scenarios of ethnic languages.

[0095] This solution leverages the continuous learning mechanism built into the ethnic language corpus annotation and training platform, enabling the agent to possess strong adaptive and evolutionary capabilities when facing data-scarce ethnic languages ​​and their dialect variations. Compared to traditional static models that require a large amount of labeled data and full retraining to adapt to new scenarios, this system can effectively iteratively optimize the model using unlabeled data and self-reinforcement learning techniques, significantly shortening the corpus annotation and model optimization training cycle and substantially reducing model training costs.

[0096] Technical benefit 3: Enables fully automated monitoring of the entire process, significantly improving work efficiency.

[0097] This solution streamlines the entire process from content perception and analysis to handling, achieving an intelligent closed loop for regulatory operations. The system can automatically complete tasks such as program content review, intelligent cataloging, identification of illegal content, and report generation, significantly reducing manual intervention, improving regulatory efficiency, and ensuring consistency and accuracy in supervision.

[0098] Technical benefit 4: Multimodal information fusion analysis enhances the comprehensiveness of supervision.

[0099] By deeply integrating multimodal information such as voice, text, and images, the system can analyze and understand content from multiple dimensions, greatly improving the comprehensiveness and accuracy of supervision. Compared with single-modal analysis, multimodal fusion significantly improves the completeness of content understanding, and is particularly suitable for the audio and video content analysis needs common in ethnic minority language scenarios.

[0100] Technological benefit five: Combining knowledge-driven and data-driven approaches enhances the scientific nature of decision-making.

[0101] This solution combines regulatory expertise with the general cognitive capabilities of large language models, ensuring both the professionalism of regulation and full utilization of the advantages of big data analytics. This combination enables the system not only to identify known violation patterns but also to help discover new violation trends and patterns, providing more scientific and comprehensive support for regulatory decision-making.

[0102] Figure 5 An example is a schematic diagram of the physical structure of an electronic device. For example... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call computer programs stored in the memory 530 to execute the steps of the ethnic language audiovisual data AI intelligent agent processing system, such as including: Using a large language model, the system identifies user intent based on user input and the ethnic language audiovisual data to be processed. In response to the identified user intent being to identify illegal content, it schedules a lightweight model and / or plugin corresponding to the illegal content identification task. This lightweight model and / or plugin then identifies suspected illegal content in the ethnic language audiovisual data. Finally, using a multimodal large model, it performs deep semantic recognition and understanding of the complex semantic content within the suspected illegal content, obtaining a judgment result on whether the ethnic language audiovisual data contains illegal content, and generating a corresponding analysis report. The system then outputs both the judgment result and the analysis report.

[0103] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the ethnic language audiovisual data AI intelligent agent processing system provided in the above embodiments, such as including: Using a large language model, based on user input and the ethnic language audiovisual data to be processed, the system identifies user intent. In response to the identified intent being to identify inappropriate content, it schedules a lightweight model and / or plugins corresponding to the inappropriate content identification task. These lightweight models and / or plugins then identify suspected inappropriate content in the ethnic language audiovisual data. Finally, using a multimodal large model, it performs deep semantic recognition and understanding of the complex semantic content within the suspected inappropriate content, obtaining a judgment result on whether the ethnic language audiovisual data contains inappropriate content, and generating and outputting the judgment result and analysis report.

[0105] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing a processor to perform the steps of the methods provided in the above embodiments, such as including: Using a large language model, the system identifies user intent based on user input and the ethnic language audiovisual data to be processed. In response to the identified user intent being to identify illegal content, it schedules a lightweight model and / or plugin corresponding to the illegal content identification task. This lightweight model and / or plugin then identifies suspected illegal content in the ethnic language audiovisual data. Finally, using a multimodal large model, it performs deep semantic recognition and understanding of the complex semantic content within the suspected illegal content, obtaining a judgment result on whether the ethnic language audiovisual data contains illegal content, and generating a corresponding analysis report. The system then outputs both the judgment result and the analysis report.

[0106] Processor-readable storage media can be any available medium or data storage device that the processor can access, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A system for processing ethnic minority language audiovisual data using AI intelligent agents, characterized in that, The system includes: A large language model is used to identify user intent based on user input instructions and the ethnic language audiovisual data to be processed; The console includes a workflow scheduling unit; the workflow scheduling unit is used to respond to the user's intent to identify illegal content, schedule a lightweight model and / or plugin corresponding to the task of identifying illegal content, and identify suspected illegal content in the ethnic language audiovisual data through the lightweight model and / or plugin; A multimodal large model is used to perform deep semantic recognition and understanding of the complex semantic content in the suspected illegal content, obtain the judgment result of whether the ethnic language audiovisual data contains illegal content, obtain the corresponding analysis report, and output the judgment result and analysis report; The large language model, the multimodal large model, and the lightweight model are all models trained using ethnic language corpora.

2. The system according to claim 1, characterized in that, The workflow scheduling unit is also used to respond to the user's intention to obtain ethnic language audio-visual data within a target range, schedule the lightweight model and / or plugin corresponding to the intelligent compilation task, and obtain ethnic language audio-visual data that conforms to the target range through the lightweight model and / or plugin; The multimodal big model includes an intelligent cataloging module for ethnic minority languages, a multimodal ethnic minority language content retrieval module, and an intelligent reporting module for business data; The ethnic language intelligent cataloging module is used to determine the cataloging information of ethnic language audiovisual data within the target range according to the user's intention; The multimodal ethnic language content retrieval module is used to retrieve at least one piece of ethnic language audiovisual data that conforms to the type and belongs to the target range based on the cataloging information. The business data intelligent reporting module is used to intelligently report the at least one piece of ethnic language audiovisual data and intelligently generate corresponding ethnic language audiovisual content.

3. The system according to claim 1, characterized in that, The system also includes a lightweight model subsystem; the lightweight model subsystem includes one or more of the following: ethnic language identification model, ethnic language speech recognition model, ethnic language OCR recognition model, ethnic language text translation model, and sensitive content detection model; The ethnic language identification model is used to identify the language of the ethnic language audiovisual data to be processed based on convolutional neural networks and recurrent neural networks. The ethnic language speech recognition model is trained using speech recognition corpora of ethnic languages ​​and their corresponding dialects, and is used to perform dialect-supported speech recognition on the ethnic language audiovisual data to be processed using end-to-end speech recognition technology. The ethnic language OCR recognition model is used to perform character recognition on the ethnic language audiovisual data to be processed using deep learning-based optical character recognition technology. The Sino-Tibetan text translation model is used for deep learning-based text translation technology to perform text translation conversion from ethnic language to Chinese on the ethnic language audiovisual data to be processed. The illegal content detection model is used to detect illegal content in the ethnic language audiovisual data to be processed based on a deep neural network model that integrates multimodal information.

4. The system according to claim 1, characterized in that, The system also includes a platform for labeling and training ethnic minority language corpora; the platform includes a model training and evaluation module. The model training and evaluation module is used to train the target model based on multiple dialect training sets constructed from corpora of different dialects of a national language, so as to obtain a target model that supports processing different dialects under the same national language. The target model includes at least one of the large language model, the multimodal large model, and the lightweight model.

5. The system according to claim 1, characterized in that, The ethnic language corpus annotation and training platform also includes at least one of the following: an intelligent annotation module, a multimodal corpus production module, and a quality standard and control module; The intelligent annotation module is used to adopt an iterative processing mode of data filtering and automatic machine recognition. Data filtering includes automatic recognition of ethnic language speech and automatic translation of ethnic language text to obtain ethnic language materials; automatic machine recognition includes machine pre-annotation of the obtained ethnic language materials to obtain annotated training corpus data. The multimodal corpus production module is used to construct various types of corpus resource libraries; the various types of corpus resource libraries include at least one of the following: ethnic minority language speech recognition corpus, language identification corpus, and text translation corpus; The quality standards and control module is used to digitally manage the entire process of data collection, annotation, and review, and to control the error rate of corpus annotation. The model training and evaluation module is also used to automatically divide the labeled training corpus data into a training set, a development set, and a test set according to certain criteria. The training set is used for training the model; the development set is used to verify the model's performance during the training phase and to make targeted adjustments to the trained model based on the results of the development set; and the test set is used to verify the performance of the trained model.

6. The method according to claim 1, characterized in that, The system also includes a knowledge base, which includes a knowledge base of violations and a database of historical cases of violations for ethnic language audiovisual programs or content. The violation knowledge base includes at least one category of violation knowledge bases, namely, people, images, videos, text, and audio. The knowledge base is implemented using vector database technology and supports similarity retrieval and semantic matching retrieval.

7. The method according to any one of claims 1-5, characterized in that, The plugin includes at least one of the following plugins: A web crawler plugin used to actively search for and collect suspected illegal ethnic language audiovisual data on the Internet; Content moderation plugin for automated review and tagging of ethnic language audiovisual data; Cataloging generation plugin, used to generate standardized cataloging information for broadcast television programs; A report generation plugin is used to automatically generate an analysis report based on the discrimination results.

8. A method for processing ethnic minority language audiovisual data using AI intelligent agents, characterized in that, The method includes: Using a large language model, the user's intent is identified based on the user's input instructions and the ethnic language audiovisual data to be processed; In response to the user's intent to identify illegal content, a lightweight model and / or plugin corresponding to the task of identifying illegal content is scheduled, and the suspected illegal content in the ethnic language audiovisual data is identified through the lightweight model and / or plugin. Using a multimodal large model, deep semantic recognition and understanding are performed on the complex semantic content in the suspected illegal content to obtain the judgment result of whether the ethnic language audiovisual data contains illegal content, and the corresponding analysis report is obtained and output. The large language model, the multimodal large model, and the lightweight model are all models trained using ethnic language corpora.

9. The method according to claim 8, characterized in that, The method further includes: In response to the recognition that the user's intention is to obtain ethnic language audiovisual data within a target range, a lightweight model and / or plugin corresponding to the intelligent compilation task is scheduled, and ethnic language audiovisual data that conforms to the target range is obtained through the lightweight model and / or plugin. Using a multimodal large model, the cataloging information of ethnic language audiovisual data within the target range is determined based on the user's intent. Based on the cataloging information, at least one piece of ethnic language audiovisual data that matches the type and belongs to the target range is retrieved. The at least one piece of ethnic language audiovisual data is intelligently compiled and reported to generate corresponding ethnic language audiovisual content.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory storing a computer program, wherein the processor executes the computer program to implement the AI ​​intelligent agent processing method for ethnic language audiovisual data as described in claim 8 or 9.

Images