Real-time monitoring system oriented to live sales of network commodities
Through ASR, OCR and LLM technologies, combined with multi-level identification strategies, a real-time monitoring system is built, which solves the efficiency, real-time and reliability of live e-commerce monitoring, and realizes automatic monitoring and recording of violations in live e-commerce. , ensuring user information security and market order.
Patent Information
- Application Number
- CN202410979367.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-22
AI Technical Summary
The existing technology cannot meet the efficiency, real-time, accuracy and reliability needs of live broadcast monitoring, especially when facing chaos in the live e-commerce industry and user information security threats.
Using automatic speech recognition (ASR), image text recognition (OCR) and large language model (LLM) technologies, combined with multi-level recognition strategies, a real-time monitoring system for live sales of online products is built, and live broadcast data is obtained through crawlers and screen recording modules. 识别语音和图像内容,检测违规线索,包括敏感词、变体词、禁售限售商品和虚假信息,实现自动化监测和记录。
A comprehensive monitoring of illegal and irregular behaviors in live e-commerce has been achieved, the accuracy and efficiency of monitoring has been improved, the pressure of manual monitoring has been reduced, the timeliness and pertinence of supervision has been improved, and the guarantee of healthy development has been brought to the live broadcast industry.
Smart Images

Figure CN118967154A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of live broadcast monitoring and processing, and in particular to a real-time monitoring system for live broadcast sales of online commodities. Background Art
[0002] Live streaming e-commerce has become a popular shopping method for consumers due to its high interactivity and entertainment value, and has penetrated into people's production and life. The success of the live streaming model lies in its ability to provide instant feedback, enhance the shopping experience, and attract and retain viewers in an entertaining way.
[0003] Since the live broadcast e-commerce industry started not long ago but has developed rapidly, the relevant market supervision system has lagged behind its development speed, resulting in a series of chaos, including but not limited to false and exaggerated propaganda, price fraud, counterfeit and shoddy goods, abuse of extreme vocabulary, inappropriate words and deeds of anchors, and sales of banned products. Live broadcast e-commerce platforms have a large user base and high-speed information dissemination capabilities, which provides channels for the rapid spread of false information and malicious links, thereby threatening users' network information security. The chaos of live broadcasts not only infringes on the legitimate rights and interests of consumers, but also undermines the fair competition order of the market and affects the healthy development of the live broadcast e-commerce industry.
[0004] In view of the above-mentioned problems and phenomena, there is an urgent social need to effectively supervise and manage the chaos of live streaming. However, due to the on-site and real-time characteristics of online live streaming sales, and the fact that some live streaming deliberately evades monitoring measures and violates regulations, the monitoring systems in related technologies cannot meet the current requirements for high efficiency, real-time, accuracy and reliability of live streaming monitoring. Summary of the invention
[0005] The present invention provides a real-time monitoring system for live broadcast sales of online commodities, which solves the problem that the monitoring system in the related art cannot meet the current requirements for high efficiency, real-time performance, accuracy and reliability of live broadcast monitoring.
[0006] In order to achieve the above objectives, this application adopts the following technical solutions:
[0007] A real-time monitoring system for online commodity live sales is provided, comprising:
[0008] Data acquisition module, data analysis module and demonstration system module;
[0009] The data acquisition module is used to acquire live video data;
[0010] The data analysis module includes an ASR module, an OCR module, a violation clue discovery module and a commodity category classification module;
[0011] The ASR module is used to identify the voice content of the live video data and convert it into live text; the OCR module is used to identify the screen content of the live video data and convert it into live text; the violation clue discovery module is used to identify whether the live text content contains violation clues; the types of violation clues include sensitive word types, variant word types, prohibited and restricted sale commodity types, and commodity false information types, and the recognition model processing is configured according to the type of violation clues; the commodity category classification module is used to mark the commodity category label for each live video segment according to the live text based on the large language model;
[0012] The demonstration system module is used for associated visual display, including data tables of violation clues and database information.
[0013] In a first possible implementation, the data acquisition module includes a crawler module for directly acquiring the live video data and a screen recording module for indirectly acquiring the live video data;
[0014] The crawler module cyclically reads the live link pool in the database, and for each live link read, starts a new thread to download the live video content in parallel;
[0015] The screen recording module captures pixel data and audio data of the screen image based on FFmpeg, encodes and merges the pixel data and the audio data in real time, and encapsulates them into a multimedia container format to form a multimedia file;
[0016] The downloaded live video content and the downloaded media files are stored in a video buffer to be analyzed, and corresponding records are added to a video table to be analyzed in the database.
[0017] In a second possible implementation, the violation clue discovery module includes:
[0018] A sensitive word matching module, based on a general sensitive word recognition model, matches the live text with words in a sensitive word library, and if a sensitive word exists in the live text, marks the corresponding live video data;
[0019] A variant word recognition module, used for performing multi-level variant word recognition according to variant word types;
[0020] The variant words include structural variant words, phonetic variant words, and semantic variant words;
[0021] The multi-level variant word recognition includes: identifying the structural variant words through a preset regular expression; analyzing the language features of the live text based on a statistical language model, correcting spelling errors and identifying the phonetic and morphological variant words; performing semantic understanding and context analysis on the live text based on a large model to identify the semantic variant words; based on the identified variant words, obtaining the original words of the variant words, and matching the original words with a pre-constructed sensitive word library to determine whether the original words exist in the sensitive word library; if so, marking the corresponding live video data;
[0022] The prohibited and restricted-sale commodity identification module identifies whether there is any sales behavior of prohibited and restricted-sale commodities in the live broadcast content based on the prohibited and restricted-sale commodity identification model. If so, the corresponding live broadcast video data is marked;
[0023] The commodity false information identification module performs product matching analysis based on the commodity description in the live text and the commodity description recorded in the commodity database based on the commodity false information identification model. If there is a contradiction, the corresponding live video data is marked.
[0024] Based on the second possible implementation, in a third possible implementation, the multi-level variant word recognition is configured with a time priority classification filtering strategy, namely:
[0025] In the real-time stream of live video content, the time priority of variant word recognition based on regular matching is greater than the time priority of variant word recognition based on a statistical language model, and the time priority of variant word recognition based on a statistical language model is greater than the time priority of variant word recognition based on a large language model.
[0026] Based on the third possible implementation, in a fourth possible implementation, after the multi-level variant word recognition is started, the variant word recognition based on regular matching performs feedback;
[0027] After each set first period of time, the variant word recognition based on the statistical language model recognizes and feeds back the data of the previous first period of time;
[0028] After each set second period of time, the variant word recognition based on the statistical language model recognizes and feeds back the data of the previous second period of time;
[0029] The duration of the first time period is shorter than the duration of the second time period.
[0030] Based on any of the above possible implementations, in a fifth possible implementation, after performing the multi-level variant word identification, the original word of the identified variant word is compared with the sensitive word library. If the original word exists in the sensitive word library, the variant word is stored in the variant word library.
[0031] In a sixth possible implementation, the commodity classification module classifies commodities into health products, medicines, cosmetics, and medical devices according to their types;
[0032] The commodity category classification module obtains the response result from the large model through real-time network services;
[0033] The commodity category classification module uses global variables to temporarily store the response results; before the next classification request, the global variables are cleared.
[0034] Based on the sixth possible implementation, in a seventh possible implementation, the demonstration system module includes:
[0035] The system homepage is used to display statistical information, including at least one of the following: the number of monitoring platforms, the number of live broadcast rooms, the number of monitored anchors, the number of violation types, the number of clue videos, the number of variant words, the word cloud of high-frequency trigger keywords, and the word cloud of high-frequency trigger variant words;
[0036] Entity live broadcast pool management, used to view live broadcast room information in the live broadcast pool, and provide interaction for adding new links to the live broadcast pool;
[0037] Automatic live broadcast inspection, providing interaction for configuring automated monitoring of random live broadcasts;
[0038] Customized live broadcast monitoring, providing interactions for configuring automated monitoring of selected live broadcasts;
[0039] Violation clues display, which associates and displays the live broadcast room information where the violation clues are detected by the violation clues discovery module and the corresponding violation clues;
[0040] The database display includes a table of evidence videos associated with violation clues, a table of banned and restricted sales items, a table of sensitive words, a table of pharmaceuticals, medical devices and cosmetics, a table of health foods, and a table of annotated matching relationships.
[0041] Based on the seventh possible implementation, in an eighth possible implementation, the annotated matching relationship table includes a sensitive word matching table, a special variant word matching table, and a prohibited and restricted sale matching table that match the product information.
[0042] Based on the eighth possible implementation, in a ninth possible implementation, the sensitive word matching table includes four attributes, namely, 'id', 'general sensitive word_id', 'live_id' and 'eligible'; wherein, 'id' is the primary key of the sensitive word matching table, 'general sensitive word id' and 'live_id' are foreign keys of the sensitive word matching table; 'eligible' has two values, '0' represents that the video is unqualified, and '1' represents that the video is qualified;
[0043] The special variant word matching table includes four attributes, namely, 'id', 'special variant word_id', 'live_id' and 'eligible'; wherein, 'id' is the main key of the special variant word matching table, 'special variant word_id' and 'live_id' are foreign keys of the special variant word table; 'eligible' has two values, '0' represents that the video is unqualified, and '1' represents that the video is qualified;
[0044] The banned and restricted sale matching table includes four attributes, namely 'id', 'banned and restricted sale_id', 'live_id' and 'eligible'; among them, 'id' is the main construction of the banned and restricted sale matching table, 'banned and restricted sale_id' and 'live_id' are foreign keys of the special variant vocabulary; 'eligible' has two values, '0' represents that the video is unqualified, and '1' represents that the video is qualified.
[0045] This application combines technologies such as automatic speech recognition, image and text recognition, and large language models, which can comprehensively monitor voice, image, and text information in live e-commerce, covering various illegal and irregular behaviors that may exist in live broadcasts. The system adopts a multi-level recognition strategy, from simple regular expression matching to more complex statistical language models and large language model matching, and calls more complex matching mechanisms on demand, which not only ensures the accuracy of recognition, but also avoids unnecessary waste of computing resources and improves the overall efficiency of the system. Product information matching and product category classification improve the pertinence and professionalism of monitoring. The system can monitor live broadcast content in real time, automatically discover and record illegal clues, reduce the pressure of manual monitoring, and improve the timeliness of supervision. The system display section provides an intuitive interface and has a complete database management function, which provides a solid foundation for subsequent data analysis and decision-making, simplifies the supervision process, and improves supervision efficiency; this system takes into account efficiency, real-time, accuracy, and reliability, and has brought positive changes to the live broadcast industry. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 A schematic diagram of the structure of a real-time monitoring system for live online sales of goods provided in an embodiment of the present application;
[0047] Figure 2 A flowchart of a real-time monitoring system for live online sales of goods provided in an embodiment of the present application;
[0048] Figure 3 A schematic diagram of the structure of a violation clue discovery module provided in an embodiment of the present application;
[0049] Figure 4 A schematic diagram of a prompt design for a commodity category classification module provided in an embodiment of the present application. DETAILED DESCRIPTION
[0050] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the technical solutions in the embodiments of the present application are clearly described. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present application.
[0051] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0052] The description of the method flow in the specification of the present application and the steps of the flowchart in the drawings of the present specification do not have to be strictly executed according to the step numbers, and the method steps can be executed in a different order. Moreover, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.
[0053] The following is a detailed description of a real-time monitoring system for live online commodity sales provided by an embodiment of the present application in conjunction with the accompanying drawings and preferred embodiments.
[0054] The embodiment of the present application proposes a real-time monitoring system for live streaming sales of online commodities. Through key technologies such as automatic speech recognition (ASR), image and text recognition (OCR), and large language model (LLM), it realizes automatic monitoring of illegal and irregular behaviors of live streaming e-commerce, thereby protecting the legitimate rights and interests of consumers, maintaining market order, and effectively reducing the manpower costs and timeliness of supervision of regulatory authorities.
[0055] Among them, automatic speech recognition (ASR) is a technology that converts voice signals into text. In the present invention, it is used to extract voice information in live video and convert it into text so as to monitor the content in the text for violations.
[0056] Specifically, the input speech signal is first preprocessed, including noise reduction, echo removal, and speech feature extraction, to improve the quality of the speech signal and make it easier for subsequent processing; key features such as frequency and pitch are extracted from the preprocessed speech signal to distinguish different sounds; the extracted speech features are matched with the corresponding sound units (such as phonemes) using an acoustic model to determine the basic phonemes in the speech signal; these basic phonemes are combined into words and sentences using a language model, and the language model can predict and correct the recognition results based on the grammar and lexical probability of the language; finally, the recognized speech signal is converted into text output for content monitoring.
[0057] Existing ASR models such as Google Cloud Speech-to-Text have problems such as charging by usage and high long-term cost. IBM Watson Speech-to-Text may encounter performance bottlenecks when processing high concurrency and large-scale voice data. OpenAI's Whisper model has a problem with the low accuracy of the lightweight version model.
[0058] Optical character recognition (OCR) is a technology that converts text information in an image into editable text, and is used in the present invention to extract text information in a video frame by frame.
[0059] Specifically, the input image is first preprocessed, including removing noise, adjusting contrast and brightness, etc., to improve the image quality and prepare for subsequent text recognition; then the text area is detected in the image. This step is similar to finding all the text blocks in the image and determining the parts that need to be recognized; after identifying the text area, these areas are further segmented into individual characters, and the characters are recognized one by one, rather than recognizing the entire text together, by matching each segmented character with a pre-trained character model to determine the specific content of the character; finally, the recognized text is proofread and corrected to ensure that the final output text is as accurate as possible and to monitor the content.
[0060] Existing OCR models such as Tesseract OCR have limited recognition accuracy, slow recognition speed when processing large batches of images, and low recognition accuracy in complex backgrounds or low-quality images. ABBYY FineReader has problems such as high cost of use and high hardware resource requirements. KerasOCR does not support Chinese recognition.
[0061] The Large Language Model (LLM) is a deep learning model trained with a large amount of text data and is used for natural language processing (NLP) tasks. In the present invention, it is applied to the live broadcast monitoring part, and the powerful understanding ability of the large language model is used to assist in tasks such as live broadcast product classification and content monitoring.
[0062] Existing large language models such as ChatGPT require a lot of computing resources, are difficult to run on ordinary devices, and cannot be deployed locally. BERT has problems such as poor performance in processing long texts. Wenxinyiyan has problems such as not being able to accurately perform the tasks given by the system and generating low-quality text content.
[0063] In view of the application problems of the above-mentioned ASR model, OCR model and large language model in real-time monitoring of live broadcast, the present invention applies key technologies such as ASR, OCR, LLM to real-time monitoring of live broadcast for the first time, and further focuses on comprehensive analysis of multimodal information such as voice, text, video, etc., to achieve a lightweight model with high accuracy, high efficiency and easy deployment, realize fully automatic live broadcast detection, and meet the current requirements for real-time, accuracy and reliability of live broadcast monitoring.
[0064] See also Figure 1-2 , the embodiment of the present application provides a real-time monitoring system for online commodity live sales, such as Figure 1-2 As shown, the real-time monitoring system of the embodiment of the present application includes: a data acquisition module 100, a data analysis module 200, and a demonstration system module 300. Specifically:
[0065] The data acquisition module 100 is used to acquire live video data, including a crawler module 110 for directly acquiring live video data and a screen recording module 120 for indirectly acquiring live video data.
[0066] Crawler module 110, for live broadcast platforms that support crawlers, the crawler module is developed based on the open source DouyinliveRecorder to achieve cyclic duty and multi-live concurrent video recording. When in use, after the crawler module 110 starts running, it reads the configuration file config.ini, cyclically reads the live link pool in the database, opens a thread for each live link, and downloads the live video content in parallel. The downloaded live video content is stored in the video buffer to be analyzed, and the corresponding record is added to the video table to be analyzed in the database.
[0067] The screen recording module 120 uses the FFmpeg tool to indirectly obtain live video data for live platforms such as Taobao Live that do not provide a standard API and cannot be directly obtained through crawlers. FFmpeg is an open source multimedia processing tool that can be used to process data in various formats such as audio, video, and images, and can also be used to record screen videos. The specific recording process principle is as follows:
[0068] Screen capture, FFmpeg realizes the recording function by capturing the image on the screen, which involves interacting with the operating system to obtain the pixel data on the screen. Different operating systems may have different methods to realize screen capture. In the application embodiment, for the Windows side, the Screen CapturerRecorder software is used to capture the screen video information; for the Linux side, the x11grab is used to capture the screen video information.
[0069] Image encoding: The captured screen image usually exists in the original pixel data format, such as RGB format. When recording video, these original pixel data need to be encoded into a video format, such as H.264, H.265, etc. The encoding process will compress the image data to reduce the file size and improve the transmission efficiency.
[0070] Audio capture and encoding,In live video recording, in addition to video, audio also needs to be recorded. FFmpeg can capture the system's audio input (such as a microphone) and encode it to merge audio and video data into a multimedia file.
[0071] Multimedia container encapsulation, video and audio data will be encapsulated into a container format so that the player can recognize and parse it. Container formats include MP4, AVI, MKV, etc. FFmpeg can encapsulate the encoded video and audio data into these container formats.
[0072] Real-time processing and saving,The recording process is carried out in real time, that is, the captured image and audio data will be encoded and packaged instantly and saved to files.
[0073] The data analysis module 200 includes an ASR module 210 , an OCR module 220 , a violation clue discovery module 230 and a commodity category classification module 240 .
[0074] ASR module 210 is used to identify the voice content of live video data and convert it into live text; the input data is a video file in flv format, and the ASR module 210 is used to analyze the voice content in the video and extract the voice content in the video screen, and finally output the voice content as a text file in txt format. In the early stage of this application, Kaldi and whisper model ASR were used for development. As the project progressed, the accuracy requirements for ASR detection gradually increased, and then the more powerful nextgen-kaldi model was used. The flexible nextgen-kaldi model in the field of speech recognition provides a higher accuracy rate for ASR module 210 recognition, and it occupies low computing resources and is easy to deploy locally. In addition, the open source nature of Kaldi also means that existing models can be directly used for modification and development, so that it can efficiently extract the voice text content in the video.
[0075] The OCR module 220 is used to identify the screen content of the live video data and convert it into live text; PaddlePaddle is combined with the OpenVINO OCR model for efficient text recognition, aiming to accurately identify and extract text from visual media such as videos. PaddleOCR is selected for its high efficiency and accuracy in processing text content in videos. Regardless of whether the text appears in a simple or complex background, the system can quickly and accurately extract text from various videos, giving full play to the model's efficient processing capabilities and adaptability to various fonts and layouts. At the same time, the model occupies low computing resources and is easy to deploy.
[0076] The violation clue discovery module 230 is used to identify whether the live broadcast text content obtained after processing by the ASR module 210 and the OCR module 220 contains violation clue content. It is the most important module in the data analysis module 200. Due to the complexity of the types of violation clues and the large amount of content, it is difficult to handle them using a single recognition model. Therefore, this application uses a variety of clue discovery models to comprehensively discover and identify clues for different types of situations. In response to possible violations that may occur in the live broadcast of goods, the types of violation clues can be divided into the following types: general sensitive word type, variant word type, banned or restricted sale product type, and product false information type. Specifically:
[0077] Common sensitive word type: In live broadcasts of products, merchants often create misleading advertising slogans and promotional statements in order to induce consumers to make purchases. For example, according to Article 9, Item 3 of the Advertising Law of the People's Republic of China, it is stipulated that the terms "national level", "highest level", "best" and so on shall not be used. When such words that are clearly prohibited by relevant laws and regulations and online live broadcast platform specifications appear in live broadcast videos, they belong to the common sensitive word type.
[0078] Variant word types: Variant words are the distinctive feature of Internet language as a non-standard language. People often replace relatively serious, standardized, and sensitive words with relatively non-standard and insensitive words to avoid censorship, express emotions, satire, entertainment, etc. The new words used to replace the original words are called variant words. According to Article 4 of the "Advertising Law of the People's Republic of China", advertisements shall not contain false or misleading content, and shall not deceive or mislead consumers. Advertisers shall be responsible for the authenticity of the content of advertisements. Some variant words, for the purpose of circumventing censorship, have the effect of inducing consumers by misleading or false propaganda. This part of variant words belongs to the type of variant words that are the focus of this invention.
[0079] Types of prohibited and restricted goods: When prohibited and restricted goods appear in live broadcasts as specified by relevant laws and regulations, they are prohibited and restricted goods. For example, according to Article 13 of the E-Commerce Law and Article 61 of the Drug Administration Law (2019), vaccines, blood products, narcotic drugs, psychotropic drugs, toxic drugs for medical use, radioactive drugs, and pharmaceutical precursor chemicals, etc., which are subject to special state management, shall not be sold online.
[0080] Types of false product information: In live broadcasts of products, there are sometimes exaggerated promotions, that is, the promotional statements in the advertisements do not match the actual information of the product. According to Article 4 of the Advertising Law of the People's Republic of China, advertisements shall not contain false or misleading content, and shall not deceive or mislead consumers. When advertising language that does not match the actual situation of the product appears in the live broadcast, it belongs to the type of false product information.
[0081] See also Figure 3 For the above-mentioned types of violation clues, the violation clue discovery module 230 of the present application constructs multiple clue discovery models to process them in a targeted manner, including:
[0082] The sensitive word matching module 231 is based on the general sensitive word recognition model. The module first reads the sensitive word list from the database, and matches the content identified by OCR and ASR with the words in the sensitive word library one by one. If a sensitive word exists in the text, the system can mark the possible risks of the live content, thereby automatically monitoring and managing the live content, ensuring that the live content does not involve illegal and illegal information, exaggerated propaganda, inappropriate remarks and other bad information, and effectively preventing violations of live behavior. The matching of sensitive words not only improves the efficiency of supervision, but also helps the platform to respond quickly to potential problems, thereby protecting the audience from the impact of bad content, and avoiding the platform from suffering legal risks and brand reputation damage due to content problems. For example, in a live broadcast, the anchor or user may accidentally or intentionally publish content that does not comply with the "Advertising Law of the People's Republic of China", such as using exaggerated advertising slogans such as "best" and "national level", or involving obscene, pornographic and other illegal content. At this time, the sensitive word matching module 231 automatically detects these sensitive words through the text and voice recognized by the ASR module 210 and the OCR module 220. Once such sensitive words are identified, the system will save the detected sensitive words and the corresponding live video and the text recognized by the ASR module 210 and the OCR module 220.
[0083] Variant word identification module 232, the purpose of this module is to effectively identify and process sensitive words and their variants used intentionally or unintentionally in live content to ensure the content compliance of the live platform. The dynamic and interactive nature of the live environment means that content is generated and disseminated very quickly, and participants may use non-standard words or deliberately modify sensitive words to circumvent the automatic detection system. Therefore, this module not only needs to deal with the detection of standard sensitive words, but also needs to be able to intelligently identify those words that have been slightly modified but have the same intention. This module performs multi-level variant word recognition based on the text recognized by the ASR module 210 and the OCR module 220, including: variant word recognition based on regular matching, variant word recognition based on statistical language models, and variant word recognition based on large language models.
[0084] This application divides variant words into structural variant words, phonetic variant words, and semantic variant words based on the statistical analysis results of variant words commonly used by live-streaming anchors on different platforms.
[0085] Structural variant words include variant words that change the physical structure of the original word while maintaining its visual or auditory similarity; they mainly circumvent censorship by changing the internal structure of the word, such as changing the original word by inserting, deleting, or replacing characters. They are visually or auditorily similar to the original word, but the structure has changed. This strategy maintains the visual or auditory similarity to the original word, so that people can understand its true intention, while circumventing the automatic recognition of the machine. For example, in the live broadcast of non-pharmaceuticals or medical devices, since Article 17 of the new "Advertising Law" stipulates: "Except for medical, pharmaceutical, and medical device advertisements, any other advertisements involving disease treatment functions are prohibited, and medical terms or terms that easily confuse the promoted products with drugs and medical devices shall not be used", the anchor will replace the banned word "clinical" with "Clinical Bed" and "enhance immunity" with "increase and strengthen resistance", thereby evading the platform's review.
[0086] Words with phonetic and morphological variants include words that replace words with similar pronunciations or similar written forms; words that rely on Chinese homophones or similar-looking characters to avoid being recognized by automatic review systems by replacing them with words with similar pronunciations or similar forms. They are similar to the original words in pronunciation or written form, but the confusion is achieved through replacement. This type of variant word mainly exists in the background, title and other information of the live broadcast room. For example, in the live broadcast room of health products, when the anchor promotes the "anti-sugar" function of the product, he may use the similar-sounding "K sugar" as a variant in the background of the live broadcast room to circumvent review.
[0087] Semantic variant words include variant words with different structures but similar semantics or that can refer to other words; by changing the semantic context of a word or replacing the original word with a word with the same or similar meaning, the purpose is to evade detection while maintaining information transmission. This variant word may be completely different from the original word in structure, but can achieve the same expression purpose through semantic association. For example, some anchors will replace the banned word "doctor" with "white coat" and "pill" with "white tablets" to prevent the live broadcast room from being banned by the detection system.
[0088] The multi-level variant word recognition described in this application is to process the extracted texts of the above three categories of variant words in parallel through three targeted variant word recognition modules, specifically including:
[0089] Step S201 : identifying structural variant words through a preset regular expression based on text data.
[0090] Exemplarily, through the data statistics of the structural variant words of each live streaming platform, the vast majority of the streamers adopt the form of inserting characters and follow specific rules. Here, for the four most frequently occurring variant ways, regular expressions are set manually to extract the variant words from the original text. The following are the targeted regular expression designs for the four structural variants with the inserted words being "mou", "shenme", "xiao", and "he" respectively:
[0091] r‘([\u4e00-\u9fa5]+)mou([\u4e00-\u9fa5]+)’ is in the form of ‘*mou*’,
[0092] r‘([\u4e00-\u9fa5]+)shenme([\u4e00-\u9fa5]+)’ is in the form of ‘*shenme*’,
[0093] r‘([\u4e00-\u9fa5]+)xiao([\u4e00-\u9fa5]+)’ is in the form of ‘*xiao*’,
[0094] r‘([\u4e00-\u9fa5]+)he([\u4e00-\u9fa5]+)’ is in the form of ‘*he*’.
[0095] When the text detected by the OCR and ASR modules contains the text with these four inserted words, the characters before and after the inserted words are combined and matched with the designed sensitive word library. If the combined word exists in the sensitive word library, the detected word is stored in the variant word library as a variant of the sensitive word.
[0096] The following is an example of variant word recognition based on regular expressions in this application:
[0097] "The original input is: This product is the most shenme popular;
[0098] Sensitive word detected: the most popular;
[0099] Variant word saved: the most shenme popular;
[0100] The original input is: This has a therapeutic effect on our cardiovascular and cerebrovascular diseases;
[0101] Sensitive word detected: cardiovascular;
[0102] Variant word saved: cardiovascular and cerebrovascular;
[0103] The original input is: The effect of this has been verified in linmou clinic;
[0104] Sensitive word detected: clinic;
[0105] Variant word saved: linmou clinic".
[0106] Based on this, the variant word recognition method based on regular expressions can quickly and accurately detect the structural variant words in the text according to the pre-set regular expressions.
[0107] Step S202: Analyze the language features of the text data based on a statistical language model, correct spelling mistakes, and identify sound-shape variant words.
[0108] Considering the recognition requirements of sound-shape variant words, this application uses a Chinese N-gram language model trained by the KenLM statistical language model tool to implement the recognition. The Chinese NGram language model trained based on KenLM combines the rule method and the confusion set, and can not only correct Chinese spelling mistakes, but also identify homophones and near-homophone variants, effectively dealing with the sound-shape changes of variant words.
[0109] Statistical language models are good at realizing the recognition of variant words based on homophones and near-homophones. When combined with the variant word model based on regular expressions, they can handle more situations. The following are several examples of variant word recognition based on statistical language models:
[0110] “{'source': '这种药物的林床笑果非常好', 'target': '这种药物的临床效果非常好', 'errors': [('林床', '临床', 5), ('笑果', '效果', 7)]}
[0111] {'source': '咱们医院里的伊生都在推荐啊', 'target': '咱们医院里的医生都在推荐啊', 'errors': [('伊生', '医生', 6)]}
[0112] {'source': '咱们的欣闹学管疾病等等都可以治疗啊', 'target': '咱们的心脑血管疾病等等都可以治疗啊', 'errors': [('欣闹', '心脑', 3), ('学管', '血管', 5)]}”.
[0113] It can be seen from the third example that the statistical language model can replace the sound-shape variant word "欣闹" in the text with the prohibited word "心脑". It can be seen from the first two examples that in addition to detecting variant words, the statistical language model can also detect misspelled words in OCR and ASR texts. Therefore, combining the statistical language model with the variant word recognition based on regular expressions can detect more variant words hidden in the text.
[0114] Step S203: Perform semantic understanding and context analysis on the text data based on a large model to identify semantic variant words.
[0115] Although the first two methods can solve most variant words, it is difficult to detect and identify emerging semantic variant words. Moreover, as time goes by, more and more semantic variant words will appear, so it is necessary to use a large language model that can understand the deep semantics and continuously learn and update over time.
[0116] Large language models are particularly good at understanding the deeper meaning and context of language, and are able to take into account the contextual information around words. This means that even if the variant word is structurally very different from the original word, as long as they are semantically related, the large model will be able to recognize this connection. This is critical for identifying and understanding semantic variant words, which often avoid detection by replacing words with the same or similar meaning. And for application scenarios that need to deal with semantic variant words in multiple languages or dialects, large language models often have cross-lingual understanding capabilities, which allows them to work effectively in a wider range of cultural and geographical contexts. Most importantly, even if new semantic variant words continue to appear, large language models that are continuously learning and updating can adapt to such changes because they can learn new language patterns and vocabulary usage from large amounts of text.
[0117] For example, the Spark Big Model can understand complex language structures and deep semantic information, and can handle situations where most intentions are cleverly hidden in semantic variant words. More importantly, compared with other big models, the Spark Big Model has a huge advantage in the accumulation of Chinese corpus, and can detect Chinese semantic variant words more accurately and efficiently. This application uses the Spark Big Model as an important component of the variant word recognition module based on the big model.
[0118] By calling the Spark Big Model API interface and designing a reasonable prompt method, we can achieve semantic understanding of text data and identify words with deep semantic associations. For example, the prompt design can be found in Figure 4 .
[0119] The following is an example of how the Spark model processes semantic variant words:
[0120] "Me: It's very high, and I even have to go to the white coat to eat white pills, then you have to eat link number six.
[0121] spark:
[0122] The variant words in this passage are paired with their corresponding variant words as follows:
[0123] Variant word: from the gown, original word: doctor
[0124] Variant word: take white tablets, original word: take medicine".
[0125] It can be seen that the Spark large model can detect semantic-level variant words such as "white doctor's coat" and "white tablets", and give the correct original words according to the semantics. Thanks to the powerful semantic understanding ability of the Spark large model and the accumulation of a large amount of Chinese language materials, the Spark large model has proved its effectiveness in dealing with semantic variant words.
[0126] It should be noted that this application has proved the feasibility of variant word recognition from three modules: variant word recognition based on regular expressions, variant word recognition based on statistical language models, and variant word recognition based on large models. Among them, the variant word recognition based on regular expressions consumes the least computing power, but requires manual setting of the rule library and has a high dependence on experience; the variant word recognition based on statistical language models is executed locally and is good at recognizing variant words of the homophone and near-homophone types; the variant word recognition based on large models is implemented by calling the remote API interface and can achieve semantic-level understanding.
[0127] In order to coordinate the three recognition methods and form an overall variant word recognition process. When constructing a multi-level variant word recognition system, this application comprehensively uses the above three different technical strategies and adopts a time priority hierarchical filtering strategy, that is: the time priority of variant word recognition based on rules > the time priority of variant word recognition based on statistical language models > the time priority of variant word recognition based on large models.
[0128] After the multi-level variant word recognition is started, the variant word recognition based on regular matching gives feedback; after each set first time period, the variant word recognition based on the statistical language model recognizes and gives feedback on the data of the previous first time period; after each set second time period, the variant word recognition based on the statistical language model recognizes and gives feedback on the data of the previous second time period; where the duration of the first time period is less than the duration of the second time period.
[0129] Specifically, in the real-time stream of live video content, the variant word recognition based on regular expressions quickly recognizes and filters out those obvious structural variant words according to the preset regular expressions; moreover, it can continuously update the variant word library and improve the update frequency to a certain extent; the low latency and high efficiency of the variant word recognition based on regular expressions make it an ideal choice for preliminary filtering, providing the first-level real-time variant word recognition for the multi-level variant word recognition.
[0130] As live content accumulates, after a set period of time, the statistical language model identifies variant words, and tests variants of homophones and near-phonetic words. For example, the system will start the variant word recognition process based on the statistical language model after 1 minute, and conduct in-depth analysis specifically on variants of homophones and near-phonetic words. The statistical language model accurately identifies variant words that have changed in sound and shape through pre-trained language models and confusion matrices, making up for the lack of accuracy of regular expression-based recognition methods that are limited by the comprehensiveness and update frequency of the rule base; it provides variant word recognition that takes into account accuracy at the intermediate delay level for multi-level variant word recognition.
[0131] Furthermore, after a set period of time, the recognition based on the large model processes deep semantic variant words. For example, after 5 minutes of cumulative analysis, the system will start variant word recognition based on the large model, which is the most delicate link in the entire recognition process. By calling the large model through the remote API interface, the system can understand the deep semantics of the text and identify those complex variant words whose structure and pronunciation have been cleverly changed. This process provides strong support for identifying semantic variant words, especially for sensitive words that are neither structural variants nor pronunciation variants, providing the highest accuracy variant word recognition at the long-term cumulative level for multi-level variant word recognition.
[0132] The time priority hierarchical filtering strategy of multi-level identification in the present application ensures that variant word identification methods of different degrees of precision are adopted at different time nodes. Each method plays a unique role in the variant word identification process, aiming to take into account both real-time and accuracy. Such a design not only takes into account the need for real-time response, but also ensures in-depth analysis of long-term accumulated text, achieving a balance between efficiency and accuracy, and providing a content review solution that is both efficient and reliable for live streaming platforms.
[0133] In some possible implementations, after performing multi-level variant word identification, the following step S204 is further included:
[0134] The original word of the identified variant word is compared with the sensitive word library. If the original word exists in the sensitive word library, the variant word is stored in the variant word library.
[0135] Next, based on the identified variant words, the original word of the variant word is obtained, and the original word is matched with the pre-constructed sensitive word library to determine whether the original word exists in the sensitive word library. When a variant word is detected, the original word of the variant word is matched with the pre-constructed sensitive word library to determine whether the original word exists in the sensitive word library.
[0136] The hierarchical analysis ensures the efficiency and accuracy of the system, allowing more complex matching mechanisms to be called on demand while avoiding unnecessary calculations and resource waste. Based on this, the live broadcast platform can manage content more effectively, ensuring that all live broadcast activities are carried out in a healthy and legal environment, while also improving users' trust and satisfaction with the platform.
[0137] The prohibited or restricted goods identification module 233, based on the prohibited or restricted goods identification model, identifies whether there is any sales of prohibited or restricted goods in the live broadcast content, and marks possible violations. In live broadcasts, there may sometimes be sales of prohibited or restricted goods, such as drugs, counterfeit goods, etc. This module ensures that users and consumers do not come into contact with or purchase goods that may pose a threat to their safety or health, or goods that violate specific laws or policies. The prohibited or restricted goods matching module is of great significance for maintaining the legitimacy of the live broadcast platform, protecting consumer rights, and optimizing regulatory operations. By achieving this goal through technical means, the platform can better control the quality of content and provide a safer and more reliable live broadcast environment.
[0138] The product false information identification module 234 performs product matching analysis based on the product false information identification model. This module automatically detects whether the live broadcast text contains descriptions of specific products, and further analyzes whether there are contradictions between these descriptions and the product descriptions recorded in the product database, thereby safeguarding consumer rights, preventing false or misleading propaganda, and ensuring the accuracy and compliance of advertising content.
[0139] For example, suppose in a live broadcast, the host is promoting a new health product, claiming that it has multiple health benefits. This description may attract uninformed viewers to buy it, but if these benefits are not confirmed in the official description of the health product, this constitutes false advertising.
[0140] The product false information identification module 234 can monitor the live broadcast content in real time and automatically detect whether the host's words are consistent with the information in the product database. Once a contradiction or misleading description is found, the system will save the video evidence.
[0141] Furthermore, the commodity false information identification module 234 of the present application performs matching analysis on four categories of commodities: health products, medicines, medical devices, and cosmetics. Specifically, first obtain the information of various commodities (health products, cosmetics, medicines, medical devices) from the database, and then merge the commodity names and related description information (such as health functions, product categories, product properties, etc.) into a list of two tuples for subsequent matching and contradiction detection with the live text. Among them, the classification of commodities is realized by the commodity category classification module 240, as described below. The main purpose of this module is to automatically detect the contradiction between the live text and the commodity description in the database, ensure end-to-end processing flow, from acquiring and processing data to verifying data through network services, and finally to the management of user feedback, which can create a safer, more transparent and more responsible trading environment for consumers and themselves. This not only meets regulatory requirements, but also helps to build long-term customer relationships and a good market reputation.
[0142] Product category classification module 240: The proposed product types in this application include the following four types: health products, medicines, cosmetics, and medical devices. Similar to variant word recognition based on a large language model, the live text content is used as the input of the large model, and each live video segment is marked as one of the above four product category labels by designing appropriate prompts. See prompt design for details. Figure 4 .
[0143] The core part of the commodity category classification module 240 integrates multiple steps of text processing, large model interface interaction and result processing, and realizes automatic classification of live text content through real-time network services. Global variables are used to store the response results obtained from the large model. Before calling again, the global variables will be cleared to ensure that the data of the last call will not be retained. Based on this, real-time text processing is performed to adapt to the live monitoring application scenario that requires fast response.
[0144] Demonstration system module 300 is a system display section, including multiple pages such as system homepage, entity live pool management, automatic live inspection, customized live monitoring, violation clue display, database display, etc., which effectively integrates and presents live monitoring data, provides in-depth analysis capabilities, helps managers make more informed decisions, and ensures the compliance and security of the live environment. Specifically:
[0145] The system homepage is mainly responsible for displaying statistical information, including: the number of monitored platforms, the number of live broadcast rooms, the number of monitored anchors, the number of violation types, the number of clue videos, the number of variant words, the word cloud of high-frequency trigger keywords, the word cloud of high-frequency trigger variant words, etc. This page provides users with a clear overview to help them quickly understand the system's operating status and monitoring results.
[0146] Entity live pool management, this page can be used to view the live room information in the live pool, or click "Add record" to add a new link to the live pool, and the automatic live inspection function will extract from the live pool. This page ensures that the system monitors the live room list updated in real time, making it easy to add and manage monitoring targets.
[0147] Automatic live inspection is used to configure the automatic monitoring of random live broadcasts; any three live broadcasts in the live broadcast pool can be added to this page for detection by searching, and the detection status can be set and historical inspection information can be recorded. This page automatically monitors the live broadcast room, reduces manual intervention, and improves monitoring efficiency.
[0148] Customized live monitoring is used to configure automatic monitoring of selected live broadcasts. The information displayed on the automatic live inspection interface is almost the same. You can add specified links to this page for detection by manually adding links. This page provides flexibility and allows targeted monitoring of specific live broadcast rooms.
[0149] Violation clues display: This page displays the live broadcast room information and corresponding violation clues where the violation clues are detected by the violation clue discovery module 230. Based on this, the relevant violation information can be further reviewed to provide the system with more accurate and reliable monitoring results, facilitating timely processing and action.
[0150] Database display, where the database mainly includes: evidence video table, prohibited and restricted sales table, sensitive word table, variant word table, drug, medical device and cosmetic table, health food table, and matching relationship table with annotations. This page ensures the transparency and traceability of data and provides support for monitoring decisions. For example:
[0151] Evidence video table: contains 5 attributes, namely: 'id', 'video address', 'video OCR file address', 'video ASR file address', 'video annotation'. Among them, the primary key is 'id'.
[0152] The sales restriction table contains 6 attributes, namely 'id', 'lexicon type', 'category', 'responsible department', 'keywords', and 'basis (legal provisions)'. The primary key is 'id'.
[0153] Sensitive word list: includes general sensitive word list, special variant word list and banned word list except for drugs and medical devices.
[0154] The general sensitive word table contains three attributes, namely 'id', 'sensitive word' and 'legal provision'. Among them, 'id' is the primary key of the general sensitive word table.
[0155] The special variant word table includes four attributes, namely, 'id', 'original word', 'variant word', and 'eligible'. Among them, 'id' is the primary key, 'variant word' is identified by the variant word identification module 232, 'original word' is transformed by the variant word identification module 232, and 'eligible' has two values, '0' means that the variant word is unqualified, and '1' means that the variant word is qualified.
[0156] The banned word list, except for drugs and medical devices, contains three attributes, namely 'id', 'original word' and 'legal provision'. 'id' is the primary key of this table.
[0157] Table of medicines, medical devices and cosmetics: This application is divided into several sub-tables according to different product categories and the reference materials of the Food and Drug Administration. They are the table of domestic cosmetics, the table of domestic medicines, the table of domestic medical device registration, the table of domestic medical device registration, the table of imported cosmetics, the table of imported medicines, the table of imported medical device registration and the table of imported medical device registration. Among them:
[0158] The domestic cosmetics table contains 8 attributes, namely 'id', 'product name in Chinese', 'product category', 'registrant in Chinese', 'registrant's country / region', 'domestic responsible person name', 'registration certificate valid until' and 'status'. Among them, the primary key is 'id'.
[0159] The domestic drug table contains 14 attributes, namely 'id', 'product name', 'English name', 'trade name', 'dosage form', 'specification', 'marketing authorization holder', 'production unit', 'approval date', 'production address', 'product category', 'original approval number', 'drug base code' and 'drug base code remarks'. Among them, the primary key is 'id'.
[0160] The domestic medical device registration form contains 13 attributes, namely 'Id', 'Product Name / Product Category Name', 'Model Specification / Packaging Specification', 'Product Description / Main Components', 'Intended Use', 'Product Storage and Validity Period', 'Registration Number', 'Registrant Name', 'Registrant Registration Address', 'Manufacturing Address', 'Remarks', 'Registration Unit', and 'Registration Date'. Among them, the primary key is 'Id'.
[0161] The domestic medical device registration form contains 16 attributes, namely, 'Id', 'Product Name', 'Management Category', 'Model Specification', 'Structure and Composition / Main Components', 'Scope of Application / Intended Use', 'Product Storage Conditions and Validity Period', 'Registration Certificate Number', 'Registrant Name', 'Registrant Residence', 'Manufacturing Address', 'Remarks', 'Approval Department', 'Approval Date', 'Effective Date' and 'Valid Until'. Among them, the primary key is 'Id'.
[0162] The imported cosmetics table contains 9 attributes, namely 'id', 'product name in Chinese', 'product category', 'registrant in Chinese', 'registrant's country / region', 'domestic responsible person name', 'approval date', 'registration certificate valid until' and 'status'. Among them, the primary key is 'id'.
[0163] The imported drug table contains 36 attributes, namely 'id', 'original registration number', 'registration certificate number remarks', 'subpackaging approval number', 'marketing authorization holder's Chinese name', 'marketing authorization holder's English name', 'marketing authorization holder's address (Chinese)', 'marketing authorization holder's address (English)', 'company name (Chinese)', 'company name (English)', 'address (Chinese)', 'address (English)', 'country / region (Chinese)', 'country / region (English)', 'product name (Chinese)', 'product name (English)', 'trade name (Chinese)', 'product name (English)', 'trade name (Chinese)', 'product name (English)', 'trade name (Chinese)', 'product name (English)', 'trade name (Chinese)', 'trade ...Chinese)', 'trade name (Chinese)', 'trade name (Chinese)', 'trade name (English)', 'trade name (English)', 'trade name (Chinese)', 'trade name (English)', 'trade name (Chinese)', 'trade name (English)', 'trade name (English)', ' )', 'Trade Name (English)', 'Dosage Form (Chinese)', 'Specification (Chinese)', 'Packaging Specification (Chinese)', 'Manufacturer (Chinese)', 'Manufacturer (English)', 'Manufacturer Address (Chinese)', 'Manufacturer Address (English)', 'Manufacturer Country / Region (Chinese)', 'Manufacturer Country / Region (English)', 'Product Category', 'Issuance Date', 'Expiration Date', 'Subpackaging Enterprise Name', 'Subpackaging Enterprise Address', 'Subpackaging Document Number Approval Date', 'Subpackaging Document Number Validity Date', 'Drug Basic Code', and 'Drug Basic Code Remarks'. The primary key is 'id'.
[0164] The imported medical device registration form contains 15 attributes, namely, 'Id', 'Product Name / Product Category Name', 'Model Specification / Packaging Specification', 'Product Description / Main Components', 'Intended Use', 'Product Storage and Validity Period', 'Registration Number', 'Registrant Name', 'Registrant Registration Address', 'Manufacturing Address', 'Agent', 'Agent Registration Address', 'Remarks', 'Registration Unit' and 'Registration Date'. Among them, the primary key is 'Id'.
[0165] The imported medical device registration form contains 18 attributes, namely, 'Id', 'Product Name', 'Management Category', 'Model Specification', 'Structure and Composition / Main Components', 'Scope of Application / Intended Use', 'Product Storage Conditions and Validity Period', 'Registration Certificate Number', 'Registrant Name', 'Registrant Address', 'Manufacturing Address', 'Agent Name', 'Agent Address', 'Remarks', 'Approval Department', 'Approval Date', 'Effective Date' and 'Valid Until'. Among them, the primary key is 'Id'.
[0166] Health food table: contains 6 attributes, namely 'id', 'product name', 'registration company', 'registration company address', 'registration record' and 'health function'. Among them, the primary key is 'id'.
[0167] Matching relationship table with annotations: It is divided into several matching relationship tables according to different matching methods, including sensitive word matching table, special variant word matching table and prohibited and restricted sale matching table.
[0168] The sensitive word matching table includes four attributes, namely 'id', 'general sensitive word_id', 'live_id' and 'eligible'. Among them, 'id' is the primary key of the sensitive word matching table, 'general sensitive word id' and 'live_id' are the foreign keys of the sensitive word matching table. 'Eligible' has two values, '0' means that the video is unqualified, and '1' means that the video is qualified.
[0169] The special variant word matching table includes four attributes, namely 'id', 'special variant word_id', 'live_id' and 'is it qualified'. Among them, 'id' is the main key of the special variant word matching table, 'special variant word_id' and 'live_id' are the foreign keys of the special variant word table. 'Is it qualified' has two values, '0' means that the video is not qualified, and '1' means that the video is qualified.
[0170] The prohibited and restricted sale matching table includes four attributes, namely 'id', 'prohibited and restricted sale_id', 'live_id' and 'is it qualified'. Among them, 'id' is the main key of the prohibited and restricted sale matching table, 'prohibited and restricted sale_id' and 'live_id' are foreign keys of the special variant word table. 'Is it qualified' has two values, '0' means that this video is unqualified, and '1' means that this video is qualified.
[0171] The above primary key is one or a group of fields, whose values are unique in the table and are used to uniquely identify each row in the table. Therefore, in the above database display, each table has an id field as the primary key. A foreign key refers to a column in a table whose value refers to the primary key of another table and is used to establish a relationship between the two tables. For example, live_id in the sensitive word matching table is a foreign key, which refers to the ID of a live broadcast and is used to associate the relationship between sensitive words and specific live broadcasts.
[0172] Demonstration system module 300 provides the system with intuitive visual display, flexible live broadcast management and monitoring. Its detailed violation clue display presents the details of the violation, including the live broadcast room information and specific violation clues, which facilitates further review and processing, and improves the identification and processing speed of violations. The detailed database information display page provides a wealth of data tables, which are interrelated, supporting the system's comprehensive monitoring of live broadcast content and compliance judgment, and enhancing the professionalism and authority of the system; the efficient database structure ensures the consistency and relevance of the data, enabling the system to efficiently retrieve and process a large amount of live broadcast data, while simplifying the complexity of data management. The system can provide multi-dimensional analysis results, support more comprehensive compliance assessments, and help regulators and platforms formulate more effective strategies to protect consumer rights and maintain market order.
[0173] In summary, this application combines key technologies such as automatic speech recognition, image and text recognition, and large language models, which can comprehensively monitor voice, image and text information in live e-commerce, covering various illegal and irregular behaviors that may exist in live broadcasts. The system adopts a multi-level recognition strategy, from simple regular expression matching to more complex statistical language models and large language model matching, and calls more complex matching mechanisms on demand, which not only ensures the accuracy of recognition, but also avoids unnecessary waste of computing resources and improves the overall efficiency of the system. Product information matching and product category classification improve the pertinence and professionalism of monitoring. The system can monitor live broadcast content in real time, automatically discover and record illegal clues, reduce the pressure of manual monitoring, and improve the timeliness of supervision. The system display section provides an intuitive interface and has complete database management functions, which provides a solid foundation for subsequent data analysis and decision-making, simplifies the supervision process, and improves supervision efficiency; it has brought positive changes to the live broadcast industry.
[0174] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0175] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A real-time monitoring system for live online sales of goods. It is characterized in that include: Data acquisition module, data analysis module and demonstration system module; The data acquisition module is used to acquire live video data; The data analysis module includes an ASR module, an OCR module, a violation clue discovery module and a commodity category classification module; The ASR module is used to identify the voice content of the live video data and convert it into live text; the OCR module is used to identify the screen content of the live video data and convert it into live text; the violation clue discovery module is used to identify whether the live text content contains violation clues; the types of violation clues include sensitive word types, variant word types, prohibited and restricted sale commodity types, and commodity false information types, and the recognition model processing is configured according to the type of violation clues; the commodity category classification module is used to mark the commodity category label for each live video segment according to the live text based on the large language model; The demonstration system module is used for associated visual display, including data tables of violation clues and database information.
2. According to claim 1, the real-time monitoring system for online commodity live sales, It is characterized in that The data acquisition module includes a crawler module for directly acquiring the live video data and a screen recording module for indirectly acquiring the live video data; The crawler module cyclically reads the live link pool in the database, and for each live link read, starts a new thread to download the live video content in parallel; The screen recording module captures pixel data and audio data of the screen image based on FFmpeg, encodes and merges the pixel data and the audio data in real time, and encapsulates them into a multimedia container format to form a multimedia file; The downloaded live video content and the downloaded media files are stored in a video buffer to be analyzed, and corresponding records are added to a video table to be analyzed in the database.
3. According to claim 1, the real-time monitoring system for online commodity live sales, It is characterized in that The violation clue discovery module includes: A sensitive word matching module, based on a general sensitive word recognition model, matches the live text with words in a sensitive word library, and if a sensitive word exists in the live text, marks the corresponding live video data; A variant word recognition module, used for performing multi-level variant word recognition according to variant word types; The variant words include structural variant words, phonetic variant words, and semantic variant words; The multi-level variant word recognition includes: identifying the structural variant words through a preset regular expression; analyzing the language features of the live text based on a statistical language model, correcting spelling errors and identifying the phonetic and morphological variant words; performing semantic understanding and context analysis on the live text based on a large model to identify the semantic variant words; based on the identified variant words, obtaining the original words of the variant words, and matching the original words with a pre-constructed sensitive word library to determine whether the original words exist in the sensitive word library; if so, marking the corresponding live video data; The module for identifying prohibited or restricted goods for sale is used to identify whether there is any sales of prohibited or restricted goods in the live broadcast content based on the prohibited or restricted goods identification model. If so, the corresponding live broadcast video data is marked; The commodity false information identification module performs product matching analysis based on the commodity description in the live text and the commodity description recorded in the commodity database based on the commodity false information identification model. If there is a contradiction, the corresponding live video data is marked.
4. According to claim 3, the real-time monitoring system for online commodity live sales, It is characterized in that The multi-level variant word recognition is configured with a time priority classification filtering strategy, namely: In the real-time stream of live video content, the time priority of variant word recognition based on regular matching is greater than the time priority of variant word recognition based on a statistical language model, and the time priority of variant word recognition based on a statistical language model is greater than the time priority of variant word recognition based on a large language model.
5. According to claim 4, the real-time monitoring system for online commodity live sales, It is characterized in that After the multi-level variant word recognition is started, the variant word recognition based on regular matching is fed back; After each set first period of time, the variant word recognition based on the statistical language model recognizes and feeds back the data of the previous first period of time; After each set second period of time, the variant word recognition based on the statistical language model recognizes and feeds back the data of the previous second period of time; The duration of the first time period is shorter than the duration of the second time period.
6. A real-time monitoring system for online live sales of goods according to any one of claims 3 to 5, It is characterized in that After the multi-level variant word recognition is performed, the original word of the identified variant word is compared with the sensitive word library. If the original word exists in the sensitive word library, the variant word is stored in the variant word library.
7. The real-time monitoring system for live broadcast sales of online commodities according to claim 1, It is characterized in that The commodity classification module classifies commodities into health products, medicines, cosmetics, and medical devices according to their types; The commodity category classification module obtains the response result from the large model through real-time network services; The commodity category classification module uses global variables to temporarily store the response results; before the next classification request, the global variables are cleared.
8. According to claim 7, the real-time monitoring system for live broadcast sales of online commodities, It is characterized in that Demonstration system module, including: The system homepage is used to display statistical information, including at least one of the following: the number of monitoring platforms, the number of live broadcast rooms, the number of monitored anchors, the number of violation types, the number of clue videos, the number of variant words, the word cloud of high-frequency trigger keywords, and the word cloud of high-frequency trigger variant words; Entity live broadcast pool management, used to view live broadcast room information in the live broadcast pool, and provide interaction for adding new links to the live broadcast pool; Automatic live broadcast inspection, providing interaction for configuring automated monitoring of random live broadcasts; Customized live broadcast monitoring, providing interactions for configuring automated monitoring of selected live broadcasts; Violation clues display, which associates and displays the live broadcast room information where the violation clues are detected by the violation clues discovery module and the corresponding violation clues; The database display includes a table of evidence videos associated with violation clues, a table of banned and restricted sales items, a table of sensitive words, a table of pharmaceuticals, medical devices and cosmetics, a table of health foods, and a table of annotated matching relationships.
9. The real-time monitoring system for live broadcast sales of online commodities according to claim 8, It is characterized in that The marked matching relationship table includes a sensitive word matching table, a special variant word matching table and a prohibited and restricted sale matching table that match the product information.
10. The real-time monitoring system for online commodity live sales according to claim 9, It is characterized in that The sensitive word matching table includes four attributes, namely, 'id', 'general sensitive word_id', 'live_id' and 'eligible'; wherein, 'id' is the primary key of the sensitive word matching table, 'general sensitive word id' and 'live_id' are the foreign keys of the sensitive word matching table; 'eligible' has two values, '0' represents that the video is unqualified, and '1' represents that the video is qualified; The special variant word matching table includes four attributes, namely, 'id', 'special variant word_id', 'live_id' and 'eligible'; wherein, 'id' is the main key of the special variant word matching table, 'special variant word_id' and 'live_id' are foreign keys of the special variant word table; 'eligible' has two values, '0' represents that the video is unqualified, and '1' represents that the video is qualified; The banned and restricted sale matching table includes four attributes, namely 'id', 'banned and restricted sale_id', 'live_id' and 'eligible'; among them, 'id' is the main construction of the banned and restricted sale matching table, 'banned and restricted sale_id' and 'live_id' are foreign keys of the special variant vocabulary; 'eligible' has two values, '0' represents that the video is unqualified, and '1' represents that the video is qualified.
Citation Information
Patent Citations
A detection method and a detection system for network sensitive content
CN108984530A
Multi-level natural language anti-junk text method and system
CN109977416A
Live broadcast room violation detection method and device
CN112995696A
Clue monitoring method and system based on diversified data fusion
CN113268649A
Instant message transmission control method and device, computer equipment and storage medium
CN113794624A
Cited By
Real-time video sensitive content identification method and system
CN119296008A
Advertisement video badness-oriented fine-grained detection method based on knowledge graph and large language model
CN120451872A
Intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization
CN120640020A
Intelligent inspection method and system for network live broadcast goods carrying
CN120769117A
