Live broadcast stream voice-to-text auditing system, method, device and medium

Abstract

The invention relates to a live broadcast stream voice-to-text auditing system and method, equipment and a medium, and belongs to the technical field of live broadcast auditing. According to the system, through collaborative design of an initialization configuration module, a task management module, a service provider matching module, an intermittent transliteration module, a slice monitoring and exception switching module, a recheck module and an auditing module, efficient management of multiple live broadcast stream tasks can be realized, resources of multiple transliteration service providers are integrated, exception switching is realized, and transliteration continuity is guaranteed; meanwhile, the overall transliteration cost can be reduced through intermittent transliteration under the conditions of low risk, no voice and no face, and the recorded and broadcast file corresponding to the transliteration-free time slice is extracted again through setting a recheck proportion to perform transliteration recheck, so that the transliteration cost can be saved, the content safety can be effectively guaranteed, missing detection and error detection are avoided, and the accuracy of transliteration of the recorded and broadcast file is improved. And finally, machine or manual auditing is carried out on the transliteration content, so that the recognition precision of the illegal content and the auditing reliability of the live broadcast stream are effectively ensured.

Description

Technical Field

[0001] This application relates to the field of live streaming moderation technology, and in particular to a system, method, device and medium for moderation of live streaming speech-to-text conversion. Background Technology

[0002] With the rapid popularization of internet and mobile internet technologies, live streaming, with its real-time interactive features combining visuals and audio, has become a widely popular form of information dissemination and social interaction. Against this backdrop, the security review of live streaming content has become a crucial link in maintaining a healthy online environment, protecting users' legitimate rights and interests, and promoting the standardized development of the industry. Due to the real-time and massive nature of live streaming content, the traditional method of manually listening to audio word by word for violation identification not only faces extremely high labor costs but is also prone to missed reviews due to human error, making it difficult to meet the needs of large-scale review.

[0003] To address these issues, existing content moderation systems generally employ a specific technical solution: first, the live stream is sliced; then, the sliced ​​audio data is converted into text; and finally, the text content is used to identify violations, thus completing the content security review. Compared to traditional manual review, this solution significantly improves both efficiency and accuracy, becoming the mainstream approach for live audio moderation.

[0004] However, existing technologies still have many shortcomings in practical applications that urgently need to be addressed: First, conventional technologies can only allocate a single live stream to a single speech-to-text service provider for transcribing, lacking dynamic switching capabilities. Furthermore, they cannot dynamically start or stop slicing and transcribing tasks based on the real-time status of the live stream (such as the presence of human voices or faces). This results in slicing and transcribing operations still being completed according to the process in some live stream scenarios without human voices or faces, generating unnecessary service fees. The probability of audio violations in such scenarios is extremely low, leading to significant cost waste. Second, the reliance on a single service provider has obvious limitations. There are several risks. First, when service providers experience technical malfunctions, existing solutions can only passively rely on third-party alerts and handling, which can easily lead to the loss of transliterated data after the live stream is sliced, resulting in missed reviews of illegal content. Second, different speech-to-text service providers have different transliteration models in terms of the accuracy of illegal content recognition, the coverage of supported violation types, and service costs. Existing technology cannot combine the high-frequency violation types and cost budgets in actual business to conduct targeted review of the sliced ​​audio data, making it difficult to give full play to the advantages and disadvantages of each service provider's model. This leads to unreasonable allocation of review resources and affects the overall review effect and cost-effectiveness. Summary of the Invention

[0005] Therefore, it is necessary to provide a system, method, device, and medium for reviewing live stream speech-to-text conversion to address the aforementioned technical issues.

[0006] A live stream speech-to-text verification system, the system comprising: The initialization configuration module is used to initialize the live broadcast review business access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine when the service starts. The task management module is used to receive live stream tasks submitted by the live streaming application, verify the task information, write the successfully verified live stream tasks into the Redis queue, and monitor and manage the tasks according to the task processing coroutine. The service provider matching module is used to match service providers according to the multiple transliteration strategies configured in the business, and submit each live stream task in the queue to the matched service provider in sequence for transliteration; The intermittent transliteration module is used to schedule various service providers to perform intermittent transliteration of live stream tasks through intelligent low-risk filtering, voiceless filtering, and faceless filtering. The slice monitoring and anomaly switching module is used to periodically scan all live stream tasks of the current transliteration, receive slices returned by each service provider and obtain the latest slice time, judge transliteration anomalies based on the latest slice time, and re-match the service provider of the current live stream task when an anomaly is judged. The re-inspection module is used to record the tasks of each live stream after intermittent transliteration is enabled. Service For the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and according to the preset review ratio, extract the recording file corresponding to the untranslated time segments, and send it separately to the matching service provider for translation review. The review module is used to review the real-time output transliterated content by machine or human, and to review each live stream.

[0007] Furthermore, the configuration for live streaming review service access includes the configuration of the live streaming application name, transliteration strategy, return API address, Kafka address, topic name, and token field. The initialization configuration module is also used to initialize the client object according to the configuration information corresponding to the slice return method, and to store the live streaming review service access configuration information and the client object pointer in a global variable through the live streaming application name.

[0008] Furthermore, the task management module includes: a queue length monitoring module, a task submission status monitoring module, a service provider weight adjustment module, and a data storage module; The queue length monitoring module is used to monitor the queue length in real time and report it to the task processing coroutine. When the queue length exceeds the set threshold, a new task processing coroutine is automatically added. The task submission status monitoring module is used to monitor the submission status when a live stream task is submitted to a matching service provider. If the submission fails, the live stream task will be resubmitted to the currently matched service provider, and the number of failures of the currently matched service provider will be recorded. If the same live stream task fails to be submitted consecutively and the number of failures exceeds a preset threshold, the service provider will be switched and a new service provider will be matched for task submission. The service provider weight adjustment module is used to reduce the service provider's task matching weight when the number of service provider failures exceeds a preset limit; The data storage module is used to write the live stream task and the corresponding matching service provider data into the database after the live stream task is successfully submitted.

[0009] Furthermore, multiple transliteration strategies include cost-first strategy, efficiency-first strategy, and hybrid transliteration strategy; The cost-first strategy is used to sort the service providers from low to high based on the transliteration cost and real-time task matching weight after matching multiple available service providers that meet the requirements of live stream task detection, and to prioritize the service providers ranked higher for transliteration. The efficiency-first strategy is used to sort service providers from high to low based on their transliteration efficiency data and real-time task matching weight after matching multiple available service providers that meet the requirements of live stream task detection. The service providers are selected first for transliteration. The hybrid transliteration strategy is used to manually configure multiple available service providers that meet the requirements of live stream task detection, and to switch between multiple service providers during transliteration. The switching logic is as follows: configure the transliteration ratio of each service provider. When a service provider transliterates for a certain period of time and reaches the configured transliteration ratio, switch to the next service provider for transliteration. When switching, first submit the live stream task to the next service provider for transliteration. When the next service provider has a normal slice returned, stop the transliteration of the previous service provider.

[0010] Furthermore, the intermittent transliteration module includes: an intelligent low-risk filtering module, an unmanned voice filtering module, and an unmanned face filtering module; The intelligent low-risk filtering module is used to process the risk assessment status returned by the service provider by monitoring the slices. If the assessment model is not hit, the number of misses is accumulated; if the assessment model is hit, the number of hits is accumulated. Based on the number of slices and the slice time interval, the translation duration of the live stream task is calculated. When the ratio of the number of hits to the number of misses is less than the configured low-risk condition threshold, and the number of slices reaches the preset upper limit, the current live stream task is judged to be low-risk, the service provider is notified to stop the translation, and the live stream task is placed in the pending start list. The filtering ratio configuration is obtained, and it is monitored whether the translation stop duration is greater than the product of the translation duration and the preset filtering ratio. If it is greater, the service provider is notified to restart the translation, and the service provider continues to monitor the slices returned by the service provider until the translation is completed. The silent speech filtering module determines whether a speech segment contains human voice based on the text length of the segment returned by the service provider. If the text length of the current segment is 0, it is determined to be silent, and the number of silent speech segments for the current live stream task is incremented. If the text length of the next segment is not 0, it is determined to contain human voice, and the number of silent speech segments is reset to 0. When the number of silent speech segments accumulates to a certain threshold, the current live stream task is determined to be low-risk, the service provider is notified to stop transcribing, and the live stream task is placed in the pending start list. The module monitors whether the transcribing stop duration reaches the set filtering ratio for silent speech duration. If it does, the service provider restarts transcribing, continues to judge human voice, and continues to increment the number of silent speech segments when there is no human voice. The silent speech duration is the product of the accumulated number of silent speech segments and the segment time interval. The face-free filtering module receives the face recognition result identifier of the live stream obtained by the business end through image recognition. When the obtained face recognition result identifier is no face, it immediately detects the cumulative number of current voice slices. If the cumulative number is not 0, it can be determined that there is no face and no voice, and the service provider is notified to stop the transliteration. At the same time, the live stream task is placed in the list to be started. When the transliteration stop time exceeds the preset threshold or a face recognition result with a face is received, the service provider restarts the transliteration and continues to monitor the face signal.

[0011] Furthermore, the slice monitoring and anomaly switching module includes: a slice monitoring unit and an anomaly switching unit; The slice monitoring unit is used to periodically scan all live stream tasks of the current transliteration, receive slices returned by each service provider and obtain the latest slice time, and judge transliteration anomalies based on the distance between the latest slice time and the current time. If the distance exceeds the preset threshold, it is judged that there is a transliteration anomaly. The anomaly switching unit is used to re-match the service provider for the current live stream task when a transliteration anomaly is detected. After a new service provider is matched, a transliteration start request is sent and the transliteration of the current service provider is stopped. At the same time, the number of anomalies of the current service provider is accumulated, and when the number of anomalies accumulates to a certain threshold, the task matching weight of the current service provider is reduced. The current service provider is continuously monitored, and if it does not return to normal for a long time, the task matching of the current service provider is stopped and the existing tasks are transferred to the new service provider.

[0012] Furthermore, the process for transliteration re-verification in the re-verification module includes: Retrieve all untranslated time segments of the live stream task; Obtain the re-inspection ratio and calculate the number of interval segments as the reciprocal of the re-inspection ratio; The first segment among all untranslated segments of the live stream task is taken as the starting segment for re-examination, and the remaining re-examination segments are extracted successively according to the number of interval segments until the end time period of all untranslated time segments is reached. The complete recording file of the live stream task is parsed by time period, and the recording file of the corresponding review segment is extracted. Submit the recorded files of all re-examined segments to the matching service provider for transliteration; The transliterated content is manually reviewed, the number of violations is obtained based on the results of the manual review, and the omission rate is generated based on the number of violations and the total number of time segments without transliteration. The re-inspection ratio and the parameters for intelligent low-risk filtering during intermittent transliteration were further adjusted based on the missed review rate.

[0013] A method for reviewing speech-to-text conversion in live streams, the method being implemented based on the aforementioned speech-to-text conversion review system for live streams, includes the following steps: When the service starts, the initialization configuration module performs the initialization of the live broadcast review business access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine. The task management module receives live stream tasks submitted by the live streaming application, verifies the task information, writes successfully verified live stream tasks into a Redis queue, and monitors and manages the tasks based on the task processing goroutine. The service provider matching module matches service providers based on the multi-transliteration strategy configured in the business, and submits each live stream task in the queue to the matched service provider in sequence for transliteration. Based on the intermittent transliteration module, intelligent low-risk filtering, no-voice filtering, and no-face filtering are performed, and various service providers are scheduled to perform intermittent transliteration on the live stream tasks. The slice monitoring and anomaly switching module periodically scans all live stream tasks for the current transliteration, receives slices returned by each service provider and obtains the latest slice time, performs transliteration anomaly judgment based on the latest slice time, and re-matches the service provider for the current live stream task when an anomaly is judged. According to the re-inspection module records, each live stream after intermittent transliteration was enabled... Service For the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and according to the preset review ratio, extract the recording file corresponding to the untranslated time segments, and send it separately to the matching service provider for translation review. Finally, the real-time transliterated content is reviewed by machine or human review through the review module to achieve review of each live stream.

[0014] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps: When the service starts, the initialization configuration module performs the initialization of the live broadcast review business access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine. The task management module receives live stream tasks submitted by the live streaming application, verifies the task information, writes successfully verified live stream tasks into a Redis queue, and monitors and manages the tasks based on the task processing goroutine. The service provider matching module matches service providers based on the multi-transliteration strategy configured in the business, and submits each live stream task in the queue to the matched service provider in sequence for transliteration. Based on the intermittent transliteration module, intelligent low-risk filtering, no-voice filtering, and no-face filtering are performed, and various service providers are scheduled to perform intermittent transliteration on the live stream tasks. The slice monitoring and anomaly switching module periodically scans all live stream tasks for the current transliteration, receives slices returned by each service provider and obtains the latest slice time, performs transliteration anomaly judgment based on the latest slice time, and re-matches the service provider for the current live stream task when an anomaly is judged. According to the re-inspection module records, each live stream after intermittent transliteration was enabled... Service For the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and according to the preset review ratio, extract the recording file corresponding to the untranslated time segments, and send it separately to the matching service provider for translation review. Finally, the real-time transliterated content is reviewed by machine or human review through the review module to achieve review of each live stream.

[0015] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor: When the service starts, the initialization configuration module performs the initialization of the live broadcast review business access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine. The task management module receives live stream tasks submitted by the live streaming application, verifies the task information, writes successfully verified live stream tasks into a Redis queue, and monitors and manages the tasks based on the task processing goroutine. The service provider matching module matches service providers based on the multi-transliteration strategy configured in the business, and submits each live stream task in the queue to the matched service provider in sequence for transliteration. Based on the intermittent transliteration module, intelligent low-risk filtering, no-voice filtering, and no-face filtering are performed, and various service providers are scheduled to perform intermittent transliteration on the live stream tasks. The slice monitoring and anomaly switching module periodically scans all live stream tasks for the current transliteration, receives slices returned by each service provider and obtains the latest slice time, performs transliteration anomaly judgment based on the latest slice time, and re-matches the service provider for the current live stream task when an anomaly is judged. According to the re-inspection module records, each live stream after intermittent transliteration was enabled... Service For the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and according to the preset review ratio, extract the recording file corresponding to the untranslated time segments, and send it separately to the matching service provider for translation review. Finally, the real-time transliterated content is reviewed by machine or human review through the review module to achieve review of each live stream.

[0016] The aforementioned live stream speech-to-text review system, method, device, and medium, after initial configuration, can efficiently manage each live stream task through the task management module; it can integrate resources from multiple transliteration service providers through the service provider matching module and the segment monitoring and anomaly switching module, enabling the selection of the best service provider for transliteration based on actual business needs, and quickly switching to a backup service provider when one service provider malfunctions, ensuring the continuity of transliteration services; based on the intermittent transliteration module combined with intelligent low-risk filtering, no-voice filtering, and no-face filtering, it can schedule various service providers to perform intermittent transliteration of live stream tasks when there is low risk, no voice, and no face, significantly reducing the overall transliteration cost through this "partial transliteration, partial pause" approach; and based on the re-review ratio set by the re-review module, it can re-examine the recorded files corresponding to the time segments without transliteration, effectively ensuring content security and avoiding missed or incorrect checks while saving transliteration costs; finally, the review module performs machine or manual review of the transliterated content, effectively ensuring the accuracy of identifying illegal content and the reliability of live stream review. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of a live stream speech-to-text review system in one embodiment; Figure 2 This is a flowchart illustrating a method for reviewing live stream speech-to-text conversion in one embodiment; Figure 3 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] In one embodiment, such as Figure 1 As shown, a live stream speech-to-text review system is provided, including: The initialization configuration module 101 is used to initialize the live broadcast audit service access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine when the service starts.

[0020] The task management module 102 is used to receive live stream tasks submitted by the live streaming application, verify the task information, write the successfully verified live stream tasks into the Redis queue (message queue), and perform task monitoring and management according to the task processing coroutine.

[0021] The service provider matching module 103 is used to match service providers according to the multi-transliteration strategy configured in the business, and submit each live stream task in the queue to the matched service provider for transliteration in sequence.

[0022] The intermittent transliteration module 104 is used to schedule various service providers to perform intermittent transliteration of live stream tasks through intelligent low-risk filtering, no-voice filtering, and no-face filtering.

[0023] The slice monitoring and anomaly switching module 105 is used to periodically scan all live stream tasks of the current transliteration, receive slices returned by each service provider and obtain the latest slice time, judge transliteration anomalies based on the latest slice time, and re-match the service provider of the current live stream task when an anomaly is judged.

[0024] The re-inspection module 106 is used to record the tasks of each live stream after intermittent transliteration is enabled. Service For the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and extract the recording file corresponding to the untranslated time segments according to the preset review ratio, and send it separately to the matching service provider for translation review.

[0025] The review module 107 is used to review the transliterated content output in real time by machine or by human, so as to review each live stream.

[0026] In the aforementioned live stream speech-to-text review system, after initial configuration, the task management module can efficiently manage each live stream task; the service provider matching module and the segment monitoring and anomaly switching module can integrate resources from multiple transliteration service providers, enabling the selection of the best service provider for transliteration based on actual business needs, and quickly switching to a backup service provider when one service provider malfunctions, ensuring the continuity of the transliteration service; the intermittent transliteration module, combined with intelligent low-risk filtering, no-voice filtering, and no-face filtering, can schedule various service providers to perform intermittent transliteration of live stream tasks when there is low risk, no voice, or no face, significantly reducing the overall transliteration cost through this "partial transliteration, partial pause" approach; and the re-review module, based on the re-review ratio set, re-extracts the recorded files corresponding to the untransliterated time segments for transliteration re-review, effectively ensuring content security and avoiding missed or incorrect checks while saving transliteration costs; finally, the review module performs machine or manual review of the transliterated content, effectively ensuring the accuracy of identifying illegal content and the reliability of live stream review. In summary, the live stream voice-to-text review solution provided by this system achieves the business goals of "low cost, high stability, and high efficiency," and is suitable for the business needs of live stream review at different scales.

[0027] Furthermore, the configuration for live streaming review service access includes the configuration of fields such as live streaming application name (app_name), transliteration strategy, return API (interface) address, Kafka address, topic name, and token (verification token). The initialization configuration module is also used to initialize the client object according to the configuration information corresponding to the slice return method, and to store the live streaming review service access configuration information and the client object pointer in global variables through the live streaming application name.

[0028] Through the aforementioned end-to-end initialization and configuration storage, a stable and directly callable basic configuration support is provided for subsequent business processes such as live stream translation and violation review. For example, when the API interface receives a submitted live stream task, it can match information from memory based on the app_name and token in the submitted task information to verify the task information. If the verification fails, it returns immediately; if successful, the task is written to a Redis queue. This verification ensures the security, legitimacy, and controllability of API access while also considering the efficiency of request processing.

[0029] Furthermore, the task management module includes: a queue length monitoring module, a task submission status monitoring module, a service provider weight adjustment module, and a data storage module.

[0030] The queue length monitoring module is used to monitor the queue length in real time and report it to the task processing coroutine. When the queue length exceeds the set threshold, a new task processing coroutine is automatically added.

[0031] The task submission status monitoring module is used to monitor the submission status when a live stream task is submitted to a matched service provider. If the submission fails, the live stream task will be resubmitted to the currently matched service provider, and the number of failures of the currently matched service provider will be recorded. If the same live stream task fails to be submitted consecutively and the number of failures exceeds a preset threshold, the service provider will be switched and a new service provider will be matched for task submission.

[0032] The service provider weight adjustment module is used to reduce the service provider's task matching weight when the number of service provider failures exceeds a preset limit.

[0033] The data storage module is used to write the live stream task and the corresponding matching service provider data into the database after the live stream task is successfully submitted.

[0034] Furthermore, multiple transliteration strategies include cost-priority strategies, efficiency-priority strategies, and hybrid transliteration strategies.

[0035] The cost-first strategy is used to sort service providers from low to high based on their translation costs and real-time task matching weights after matching multiple available service providers that meet the requirements of live stream task detection, and to prioritize the service providers ranked higher for translation.

[0036] The efficiency-first strategy is used to sort service providers from high to low based on their transliteration efficiency data and real-time task matching weight after matching multiple available service providers that meet the requirements of live stream task detection. The service providers are selected first for transliteration.

[0037] The hybrid transliteration strategy is used to manually configure multiple available service providers that meet the requirements of live stream task detection, and to switch between these service providers during transliteration. The switching logic is as follows: configure the transliteration ratio for each service provider; when a service provider has been transliterating for a certain period and has reached the configured transliteration ratio, switch to the next service provider for transliteration. During the switch, first submit the live stream task to the next service provider for transliteration, and stop the transliteration of the previous service provider once the next service provider has returned a normal segment. This hybrid transliteration strategy allows for seamless switching between multiple service providers in a single live stream, avoiding missed reviews due to the limitations of a single service provider's capabilities, and improving the accuracy of live stream review.

[0038] Furthermore, the intermittent transliteration module refers to transliterating the task for a certain period of time, determining that the risk is low, and then stopping the transliteration for a certain period of time to save transliteration costs. This includes: an intelligent low-risk filtering module, an unmanned voice filtering module, and an unmanned face filtering module; The intelligent low-risk filtering module is configurable, including filtering ratio thresholds (0.1~1) and risk assessment conditions. Risk assessment conditions can be configured with either a machine-reviewed hit threshold or a human review operation threshold. Specifically, it processes the machine-reviewed risk status returned by the service provider by monitoring the slices. If the machine-reviewed model is not hit, the number of misses is accumulated; if the model is hit, the number of hits is accumulated. Based on the number of slices and the slice time interval, the translation duration of the live stream task is calculated. When the ratio of hits to misses is less than the configured low-risk condition threshold (or, if a human review operation threshold is configured, the ratio of human review operations to the total number of slices), and the number of slices reaches a preset upper limit, the current live stream task is determined to be low-risk. The service provider is notified to stop translation, and the live stream task is placed in the pending start list. The filtering ratio configuration is obtained, and it is monitored whether the translation stop duration is greater than the product of the translation duration and the preset filtering ratio. If it is greater, the service provider is notified to restart translation, and the service provider continues to monitor the slices returned by the service provider until translation is complete.

[0039] The silent speech filtering module is used to determine whether there is human speech in the current slice based on the text length of the slice returned by the service provider. If the text length of the current slice is 0, it means that the current broadcaster is not speaking, and it is judged as silent speech, and the number of silent speech slices in the current live stream task is accumulated. If the text length of the next slice is not 0, it is judged as having human speech, and the number of silent speech slices is reset to 0. When the number of silent speech slices accumulates to a certain threshold, it is determined that no one has been speaking for a period of time, that is, the current live stream task is judged as low risk, the service provider is notified to stop transcribing, and the live stream task is placed in the list to be started. It monitors whether the transcribing stop time reaches the set filtering ratio of silent speech duration (e.g., 1 / 3). If it does, the service provider restarts transcribing, continues to judge human speech, and continues to accumulate the number of silent speech slices when there is no human speech. The silent speech duration is the product of the accumulated number of silent speech slices and the slice time interval.

[0040] The face-free filtering module receives the face recognition result identifier of the live stream obtained by the business end through image recognition. When the obtained face recognition result identifier is no face, it immediately detects the cumulative number of current voice slices. If the cumulative number is not 0, it can be determined that there is no face and no voice, and the service provider is notified to stop the transliteration. At the same time, the live stream task is placed in the list to be started. When the transliteration stop time exceeds the preset threshold or a face recognition result with a face is received, the service provider restarts the transliteration and continues to monitor the face signal.

[0041] Furthermore, the slice monitoring and anomaly switching module includes a slice monitoring unit and an anomaly switching unit.

[0042] The slice monitoring unit is used to periodically scan all live stream tasks of the current transliteration, receive slices returned by each service provider and obtain the latest slice time. Based on the distance between the latest slice time and the current time, it judges transliteration anomalies (slices are sent every 10 seconds, and theoretically, the returned slices are also sent every 10 seconds). If the distance exceeds the preset threshold, it is judged that there is a transliteration anomaly.

[0043] The anomaly switching unit is used to re-match the service provider for the current live stream task when a transliteration anomaly is detected. After a new service provider is matched, a transliteration start request is sent and the transliteration of the current service provider is stopped. At the same time, the number of anomalies of the current service provider is accumulated, and when the number of anomalies accumulates to a certain threshold, the task matching weight of the current service provider is reduced, so that new tasks have a greater chance of matching with other service providers. The unit also continuously monitors the current service provider. If it does not return to normal for a long time, the task matching of the current service provider is stopped and the existing tasks are transferred to the new service provider.

[0044] Furthermore, the process for transliteration re-verification in the re-verification module includes: (1) Obtain all untranslated time segments of the live stream task.

[0045] (2) Obtain the re-inspection ratio and calculate the number of interval segments as the reciprocal of the re-inspection ratio; for example, if the re-inspection ratio is 10%, then the number of interval segments is 10.

[0046] (3) Take the first segment of all untranslated segments in the live stream task as the starting segment for re-examination, and extract the remaining re-examination segments one after another according to the number of interval segments, until the end time period of all untranslated time segments is reached.

[0047] (4) Parse the complete recording file of the live stream task (in m3u8 format) by time period and extract the recording file of the corresponding re-examination segment.

[0048] (5) Submit the recorded files of all re-examined segments to the matching service provider for transliteration.

[0049] (6) Manually review the transliterated content, obtain the number of violations based on the results of the manual review, and generate the omission rate based on the number of violations and the total number of time segments without transliteration.

[0050] (7) Adjust the re-inspection ratio and the parameters of intelligent low-risk filtering during intermittent transliteration based on the omission rate.

[0051] It should be understood that the transliteration re-check uses a proportional interval sampling method to avoid concentrating samples in the same time period, ensuring uniformity of detection coverage. After the re-check, the system supports dynamically adjusting the re-check ratio for a single task or globally based on actual conditions, further preventing the spread of illegal content. This method significantly reduces transliteration costs while ensuring content security, effectively enhancing the company's market competitiveness.

[0052] In one embodiment, such as Figure 2 As shown, a method for reviewing live stream speech-to-text conversion is provided. This method is based on the aforementioned live stream speech-to-text review system and includes the following steps: Step 1: When the service starts, the initialization configuration module performs the initialization of the live broadcast audit service access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine.

[0053] Step 2: Verify the task information of the live stream tasks submitted by the live streaming application received by the task management module, write the successfully verified live stream tasks into the Redis queue, and monitor and manage the tasks according to the task processing goroutine.

[0054] Step 3: The service provider matching module matches service providers according to the multi-transliteration strategy configured in the business, and submits each live stream task in the queue to the matched service provider in sequence for transliteration.

[0055] Step 4: Based on the intermittent transliteration module, perform intelligent low-risk filtering, no-voice filtering, and no-face filtering, and schedule various service providers to perform intermittent transliteration on the live stream task.

[0056] Step 5: According to the segment monitoring and anomaly switching module, scan all live stream tasks of the current transliteration at regular intervals, receive the segments returned by each service provider and obtain the latest segment time, judge the transliteration anomaly based on the latest segment time, and re-match the service provider of the current live stream task when the anomaly is judged.

[0057] Step 6: Based on the records of the re-inspection module, perform tasks for each live stream after intermittent transliteration is enabled. Service For the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and extract the recording file corresponding to the untranslated time segments according to the preset review ratio, and send it separately to the matching service provider for translation review.

[0058] Step 7: Finally, the real-time output transliterated content is reviewed by machine or human review through the review module to achieve review of each live stream.

[0059] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for reviewing live stream speech-to-text conversion. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0060] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0061] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the following steps: When the service starts, the initialization configuration module performs the initialization of the live broadcast review business access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine. The task management module receives live stream tasks submitted by the live streaming application, verifies the task information, writes successfully verified live stream tasks into a Redis queue, and monitors and manages the tasks based on the task processing goroutine. The service provider matching module matches service providers based on the multi-transliteration strategy configured in the business, and submits each live stream task in the queue to the matched service provider in sequence for transliteration. Based on the intermittent transliteration module, intelligent low-risk filtering, no-voice filtering, and no-face filtering are performed, and various service providers are scheduled to perform intermittent transliteration on the live stream tasks. The slice monitoring and anomaly switching module periodically scans all live stream tasks for the current transliteration, receives slices returned by each service provider and obtains the latest slice time, performs transliteration anomaly judgment based on the latest slice time, and re-matches the service provider for the current live stream task when an anomaly is judged. According to the re-inspection module records, each live stream after intermittent transliteration was enabled... ServiceFor the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and according to the preset review ratio, extract the recording file corresponding to the untranslated time segments, and send it separately to the matching service provider for translation review. Finally, the real-time transliterated content is reviewed by machine or human review through the review module to achieve review of each live stream.

[0062] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: When the service starts, the initialization configuration module performs the initialization of the live broadcast review business access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine. The task management module receives live stream tasks submitted by the live streaming application, verifies the task information, writes successfully verified live stream tasks into a Redis queue, and monitors and manages the tasks based on the task processing goroutine. The service provider matching module matches service providers based on the multi-transliteration strategy configured in the business, and submits each live stream task in the queue to the matched service provider in sequence for transliteration. Based on the intermittent transliteration module, intelligent low-risk filtering, no-voice filtering, and no-face filtering are performed, and various service providers are scheduled to perform intermittent transliteration on the live stream tasks. The slice monitoring and anomaly switching module periodically scans all live stream tasks for the current transliteration, receives slices returned by each service provider and obtains the latest slice time, performs transliteration anomaly judgment based on the latest slice time, and re-matches the service provider for the current live stream task when an anomaly is judged. According to the re-inspection module records, each live stream after intermittent transliteration was enabled... Service For the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and according to the preset review ratio, extract the recording file corresponding to the untranslated time segments, and send it separately to the matching service provider for translation review. Finally, the real-time transliterated content is reviewed by machine or human review through the review module to achieve review of each live stream.

[0063] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0064] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0065] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application.

Claims

1. A live stream speech-to-text verification system, characterized in that, The system includes: The initialization configuration module is used to initialize the live broadcast review business access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine when the service starts. The task management module is used to receive live stream tasks submitted by the live streaming application, verify the task information, write the successfully verified live stream tasks into the Redis queue, and monitor and manage the tasks according to the task processing coroutine. The service provider matching module is used to match service providers according to the multiple transliteration strategies configured in the business, and submit each live stream task in the queue to the matched service provider in sequence for transliteration; The intermittent transliteration module is used to schedule various service providers to perform intermittent transliteration of live stream tasks through intelligent low-risk filtering, voiceless filtering, and faceless filtering. The slice monitoring and anomaly switching module is used to periodically scan all live stream tasks of the current transliteration, receive slices returned by each service provider and obtain the latest slice time, judge transliteration anomalies based on the latest slice time, and re-match the service provider of the current live stream task when an anomaly is judged. The re-inspection module is used to record the tasks of each live stream after intermittent transliteration is enabled. Service For the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and according to the preset review ratio, extract the recording file corresponding to the untranslated time segments, and send it separately to the matching service provider for translation review. The review module is used to review the real-time output transliterated content by machine or human, and to review each live stream.

2. The live stream speech-to-text verification system according to claim 1, characterized in that, The live streaming review service access configuration includes the configuration of the live streaming application name, transliteration strategy, return API address, Kafka address, topic name, and token field. The initialization configuration module is also used to initialize the client object according to the configuration information corresponding to the slice return method, and to store the live streaming review service access configuration information and the client object pointer in a global variable through the live streaming application name.

3. The live stream speech-to-text verification system according to claim 1, characterized in that, The task management module includes: a queue length monitoring module, a task submission status monitoring module, a service provider weight adjustment module, and a data storage module; The queue length monitoring module is used to monitor the queue length in real time and report it to the task processing coroutine. When the queue length exceeds a set threshold, a new task processing coroutine is automatically added. The task submission status monitoring module is used to monitor the submission status when a live stream task is submitted to a matching service provider. If the submission fails, the live stream task is resubmitted to the currently matched service provider, and the number of failures of the currently matched service provider is recorded. If the same live stream task fails to be submitted consecutively and the number of failures exceeds a preset threshold, the service provider is switched and a new service provider is matched for task submission. The service provider weight adjustment module is used to reduce the service provider's task matching weight when the number of service provider failures exceeds a preset limit. The data storage module is used to write the live stream task and the corresponding matching service provider data into the database after the live stream task is successfully submitted.

4. The live stream speech-to-text verification system according to claim 1, characterized in that, The multiple transliteration strategies include cost-first strategy, efficiency-first strategy, and hybrid transliteration strategy; The cost-priority strategy is used to sort the service providers from low to high according to the transliteration cost and real-time task matching weight after matching multiple available service providers that meet the live stream task detection requirements, and to prioritize the service providers ranked higher for transliteration. The efficiency-first strategy is used to sort service providers from high to low based on their transliteration efficiency data and real-time task matching weight after matching multiple available service providers that meet the requirements of live stream task detection, and to prioritize the service providers ranked higher for transliteration. The hybrid transliteration strategy is used to manually configure multiple available service providers that meet the requirements of live stream task detection, and to switch between multiple service providers during transliteration. The switching logic is as follows: configure the transliteration ratio of each service provider. When a service provider transliterates for a certain period of time and reaches the configured transliteration ratio, switch to the next service provider for transliteration. When switching, first submit the live stream task to the next service provider for transliteration. When the next service provider has a normal slice returned, stop the transliteration of the previous service provider.

5. The live stream speech-to-text verification system according to claim 1, characterized in that, The intermittent transliteration module includes: an intelligent low-risk filtering module, an unmanned voice filtering module, and an unmanned face filtering module; The intelligent low-risk filtering module is used to process the risk assessment status returned by the service provider by monitoring the slices. If the assessment model is not hit, the number of hits is accumulated; if the assessment model is hit, the number of hits is accumulated. Based on the number of slices and the slice time interval, the transliteration duration of the live stream task is calculated. When the ratio of the number of hits to the number of hits is less than the configured low-risk condition threshold and the number of slices reaches the preset upper limit, the current live stream task is judged to be low-risk, the service provider is notified to stop transliteration, and the live stream task is placed in the list to be started. The filtering ratio configuration is obtained, and it is monitored whether the transliteration stop duration is greater than the product of the transliteration duration and the preset filtering ratio. If it is greater, the service provider is notified to restart the transliteration, and the service provider continues to monitor the slices returned by the service provider until the transliteration is completed. The silent filtering module is used to determine whether there is human voice in the current segment based on the text length of the segment returned by the service provider. If the text length of the current segment is 0, it is determined to be silent, and the number of silent segments in the current live stream task is accumulated. If the text length of the next segment is not 0, it is determined to be human voice, and the number of silent segments is reset to 0. When the number of silent segments accumulates to a certain threshold, the current live stream task is determined to be low-risk, the service provider is notified to stop transcribing, and the live stream task is placed in the list to be started. The module monitors whether the transcribing stop time reaches the set filtering ratio of silent duration. If it does, the service provider restarts transcribing, continues to judge human voice, and continues to accumulate the number of silent segments when there is no human voice. The silent duration is the product of the accumulated number of silent segments and the segment time interval. The no-face filtering module is used to receive the face recognition result identifier of the live stream obtained by the service terminal through image recognition. When the obtained face recognition result identifier is no face, it immediately detects the cumulative number of the current no-voice slice. If the cumulative number is not 0, it can be determined that there is no face and no voice. The service provider is notified to stop the transliteration and the live stream task is placed in the list to be started. When the transliteration stop time exceeds the preset threshold or a face recognition result with a face is received, the service provider restarts the transliteration and continues to monitor the face signal.

6. The live stream speech-to-text verification system according to claim 1, characterized in that, The slice monitoring and anomaly switching module includes: a slice monitoring unit and an anomaly switching unit; The slice monitoring unit is used to periodically scan all live stream tasks of the current transliteration, receive slices returned by each service provider and obtain the latest slice time, and judge transliteration anomalies based on the distance between the latest slice time and the current time. If the distance exceeds a preset threshold, it is judged that there is a transliteration anomaly. The abnormal switching unit is used to re-match the service provider for the current live stream task when a transliteration abnormality is detected. After a new service provider is matched, a transliteration start request is sent and the transliteration of the current service provider is stopped. At the same time, the number of abnormalities of the current service provider is accumulated, and when the number of abnormalities accumulates to a certain threshold, the task matching weight of the current service provider is reduced. The current service provider is continuously monitored. If it does not return to normal for a long time, the task matching of the current service provider is stopped and the existing tasks are transferred to the new service provider.

7. A live stream speech-to-text verification system according to claim 1, characterized in that, The process of transliteration re-verification by the re-verification module includes: Retrieve all untranslated time segments of the live stream task; Obtain the re-inspection ratio and calculate the number of interval segments as the reciprocal of the re-inspection ratio; The first segment among all untranslated segments of the live stream task is taken as the starting segment for re-examination, and the remaining re-examination segments are extracted successively according to the number of interval segments until the end time period of all untranslated time segments is reached. The complete recording file of the live stream task is parsed by time period, and the recording file of the corresponding review segment is extracted. Submit the recorded files of all re-examined segments to the matching service provider for transliteration; The transliterated content is manually reviewed, the number of violations is obtained based on the results of the manual review, and the omission rate is generated based on the number of violations and the total number of time segments without transliteration. Based on the aforementioned missed review rate, the re-inspection ratio and the parameters for intelligent low-risk filtering during intermittent transliteration are further adjusted.

8. A method for reviewing live stream speech-to-text conversion, characterized in that, The method is implemented based on the live stream speech-to-text review system according to any one of claims 1-7, and the method includes: When the service starts, the initialization configuration module performs the initialization of the live broadcast review business access configuration, the transliteration service provider information, the client corresponding to the slice return method, and the task processing coroutine. The task management module receives live stream tasks submitted by the live streaming application, verifies the task information, writes successfully verified live stream tasks into a Redis queue, and monitors and manages the tasks based on the task processing goroutine. The service provider matching module matches service providers based on the multi-transliteration strategy configured in the business, and submits each live stream task in the queue to the matched service provider in sequence for transliteration. Based on the intermittent transliteration module, intelligent low-risk filtering, no-voice filtering, and no-face filtering are performed, and various service providers are scheduled to perform intermittent transliteration on the live stream tasks. The slice monitoring and anomaly switching module periodically scans all live stream tasks for the current transliteration, receives slices returned by each service provider and obtains the latest slice time, performs transliteration anomaly judgment based on the latest slice time, and re-matches the service provider for the current live stream task when an anomaly is judged. According to the re-inspection module records, each live stream after intermittent transliteration was enabled... Service For the untranslated time segments, after the live broadcast ends, obtain the complete recording file, and according to the preset review ratio, extract the recording file corresponding to the untranslated time segments, and send it separately to the matching service provider for translation review. Finally, the real-time transliterated content is reviewed by machine or human review through the review module to achieve review of each live stream.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 8.

Images