Network live broadcast forensic analysis method and system

Abstract

The invention belongs to the technical field of video analysis, and provides a network live broadcast evidence obtaining analysis method, which comprises the following steps: S1, monitoring a specified target in real time, and obtaining a broadcast starting state and a live broadcast stream; s2, extracting the audio of a specified target; s3, performing voice recognition on the audio extracted in the step S2 to obtain a live broadcast text draft; and S4, analyzing the semantic characters obtained in the step S3. The invention further correspondingly provides a network live broadcast evidence obtaining and analyzing system. According to the invention, intelligent evidence obtaining and analysis integration can be realized, automatic office is realized, manpower is reduced, and working efficiency is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of video acquisition and analysis, and particularly relates to a network live broadcast evidence analysis method and system. BACKGROUND

[0002] With the rise of network media, video live broadcast wins the favor of netizens with its real-time interactivity, but at the same time, some undesirable phenomena are produced.

[0003] The main steps of the traditional monitoring and evidence analysis method are as follows: (1) collecting accounts to be monitored; (2) opening the live broadcast platform at regular intervals to check whether the account is live broadcasting; (3) discovering live broadcasting and recording the account through a mobile phone or a computer; (4) importing the video file into a video editing software after the recording is completed; (5) exporting a subtitle file and using an AI tool to analyze the subtitle text; (6) making an analysis result report after the analysis is completed.

[0004] The traditional method is complicated and basically involves manual intervention, and has the following pain points: (1) a large amount of manpower is needed for intervention; (2) only active checking whether the account is live broadcasting can be performed, real-time monitoring cannot be achieved, and key live broadcast content can be easily missed; (3) manual recording needs to occupy a computer or a mobile phone device, and the use efficiency of hardware is low; (4) high requirements are needed for computer hardware when the video is converted into a subtitle file; (5) the analysis result report needs to be manually sent to the recipient. The traditional method is complicated and limited, and the work efficiency is low. SUMMARY

[0005] In view of the problems in the background art, the application provides a network live broadcast evidence analysis method, which comprises the following steps: S1, performing real-time monitoring on a specified target to obtain a live broadcast state and a live broadcast stream; S2, extracting an audio of the specified target; S3, performing voice recognition on the extracted audio in the step S2 to obtain a live broadcast script; and S4, analyzing the semantic text obtained in the step S3. The application also provides a network live broadcast evidence analysis system.

[0006] The application also provides a network live broadcast evidence analysis system, which comprises a business logic module and a middle platform service module. The business logic module comprises the following steps: performing real-time monitoring on a specified target to obtain a live broadcast state and a live broadcast stream; extracting an audio of the specified target, and performing voice recognition on the extracted audio to obtain a live broadcast script; and analyzing the obtained semantic text. The middle platform service module provides a voice recognition interface to support the business logic module to perform voice recognition.

[0007] The technical effects of the application include intelligent evidence analysis integration, automatic office, reduction of manpower, and improvement of work efficiency.

[0008] The system has the advantages of easy use, wide applicability, real-time performance, innovation, and technology.

[0009] (1)Easy to use. When the user enters the live account homepage URL into the system platform, there is no need to pay attention to the live platform. After entering the system, the platform can distinguish the live platform according to the live URL.

[0010] (2)Widely. Most software on the market can only record a single live platform, while the system platform integrates TikTok, Kuaishou and WeChat video number, and makes unified management, The application uses IP proxy pool technology for risk control management, and based on reverse analysis and RPA technology for traffic decryption and automatic monitoring.

[0011] (3)Real-time. The response time of the system platform is seconds, and the live platform is periodically requested to obtain the live state and live stream. After the live account is entered, the account can be taken evidence basically as soon as it is broadcasted.

[0012] (4)Innovation. At present, the evidence work of WeChat video number mostly depends on sniffing technology, and the steps are complicated and prone to error, and manual intervention is needed. The system platform ingeniously uses the browser of WeChat, realizes the method of automatically obtaining video stream through real-time monitoring, and realizes the automatic evidence work without manual intervention.

[0013] (5)Technology. After extracting the audio of the live video and converting it into text, the AI interface is called for semantic analysis, and the text content is extracted, summarized and summarized, and finally the live evidence report is formed. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to make the application easier to understand, the application will be described in more detail by referring to the specific embodiments shown in the accompanying drawings. These drawings only depict typical embodiments of the application and should not be considered as limiting the scope of protection of the application.

[0015] Figure 1 The flowchart of the method of the application.

[0016] Figure 2 The framework diagram of the system of the application. DETAILED DESCRIPTION

[0017] The embodiments of the application will be described below with reference to the accompanying drawings, so that those skilled in the art can better understand the application and implement it. The embodiments listed are not considered as limiting the application, and the embodiments described below and the technical features in the embodiments can be combined with each other without conflict, wherein the same parts are denoted by the same reference numerals.

[0018] As shown in the drawings, the method of the application comprises steps S1-S4. Figure 1

[0019] ​S1, real-time monitoring of the designated target, obtaining the on-air state and live stream.

[0020] Specifically, ffmpeg (a set of open source computer programs that can be used to record, convert digital audio, video, and convert them into streams) is used as a downloader of live streams, and the downloaded videos are saved in a mass storage server. The video formats include: ts video format, flv video format. For some platforms, the live stream is captured by monitoring the target homepage state, and for some platforms, the RPA (Robotic process automation, robotic process automation) technology is used to monitor the target live state in real time. Preferably, the asyncio.run_coroutine_threadsafe method in Python is used to safely schedule asynchronous operations in a multi-threaded environment, ensuring the real-time performance and stability of multi-target monitoring.

[0021] Specifically, when using the RPA technology to monitor the target live state in real time, the Hook technology is used to open the developer tools of the built-in browser of the PC end WeChat, and the mouse and keyboard are taken over by the Shadow RPA, and a timing task is set to control the mouse and keyboard to access the target account homepage, obtain the on-air state and live stream.

[0022] In a preferred embodiment, the RPA module is independently split and executed in a virtual machine, solving the problem of exclusive operating system and saving hardware resources. Specifically, the RPA service module mentioned above is split out and run in the VMware virtual machine of the server, so it does not occupy the physical machine operating system, but only needs to provide virtual machine resources.

[0023] S2, extracting the audio of the designated target.

[0024] Specifically, in order to adapt to different formats and code rates of recorded videos and reduce bandwidth loss in the speech recognition process, the invention uses moviepy to extract the audio in the video. In order to be compatible with different formats of videos, the audio format is uniformly extracted as mp3 format, and mp3 audio is used for subsequent speech recognition tasks, which significantly reduces the recognition pressure of the speech recognition module.

[0025] S3, performing speech recognition on the audio extracted in step S2 to obtain the live script.

[0026] The platform interface is used for speech recognition, and the cloud task does not consume local server resources. The exported results can support TXT pure text format and SRT subtitle format (including complete timeline) export. Through actual use, the speech recognition module supported by the platform has high accuracy and fast speed.

[0027] Specifically, some video platforms provide a voice recognition interface to the public, supporting callers to upload files in video and audio formats for recognition, although files in video format can be directly uploaded, in order to save bandwidth and reduce packet size, the application first processes the recorded live video locally, that is, extracts the audio, which can significantly reduce the packet size. After obtaining the audio, the video platform interface is directly called for voice recognition, and the recognition result is also returned through the interface.

[0028] S4, analyze the semantic text obtained in S3.

[0029] Specifically, by calling a large model (such as the general-purpose QWEN-32B model), the live script obtained by voice recognition is summarized and analyzed, the context is associated and inferred by means of the AI large model through prompt word engineering, the risk is comprehensively judged (mainly according to the script content, the sensitive topics, characters, places, etc. of the whole live broadcast are summarized, and the next action plan of the anchor is predicted to facilitate early disposal), the video content is automatically summarized, and the related live broadcast risk is prompted. The video content of multiple anchors is summarized using the Python docx library, and a word report is formed in the specified format.

[0030] Specifically, the video analysis includes categories that can be set by the user, and is analyzed using keywords.

[0031] The innovation of the application also includes: in step S1, the live video is recorded and saved using ffmpeg; in order to automatically analyze the video content, the video must be preprocessed, otherwise directly analyzing the video content will consume more system resources. In step S2, after obtaining the saved video in S1, the audio file in the video is extracted, which can reduce bandwidth consumption and improve interface response speed. In step S3, the voice recognition interface is called to recognize the audio file, and the obtained text result has two formats: TXT pure text format and SRT timeline subtitle format, which can be used for different scenes. In step S4, a large model interface is called to analyze and summarize the text result obtained in S3 and automatically issue a word report, completely liberating manpower.

[0032] The application also provides a network live broadcast forensic analysis system, as shown in Figure 2 .

[0033] The system includes a front-end display module, which includes an access device (computer or smart phone), a front-end architecture module.

[0034] Specifically, the front-end architecture module includes a Semi Design unit (UI style component), a React unit (front-end code framework), and a Webpack unit (package compilation). The function of Semi Design is page design and style unification. The React unit completes user interaction and responsive layout, and Webpack is used to complete page packaging.

[0035] The system includes a business logic module, which includes a live monitoring unit, a voice recognition unit, and an AI intelligent summary unit.

[0036] Live monitoring unit: for a target account on a certain platform, use a scheduled task to send a request to obtain the target account's live state and live stream; for a video number, use RPA technology to monitor the target live state in real time, use a scheduled task to control the mouse and keyboard to access the target account homepage, and obtain the target account's live state and live stream. Once the live broadcast starts, the live stream address can be obtained, and ffmpeg is used to download the live stream address. The downloaded video will be saved locally.

[0037] Voice recognition unit: process the video recorded by the live monitoring unit, first format it into an mp4 video format; then extract the audio file into an mp3 format, which can significantly reduce the size of the data packet; then upload the audio file to the voice recognition interface for calling. The interface will return the voice recognition result, which will be returned in two formats: TXT plain text format and SRT subtitle format.

[0038] AI intelligent summary unit: for the recognition result of the voice recognition unit, a large model (such as the qwen-32b model) is called for intelligent summary according to the preset prompt word. The large model analyzes the sensitivity of the content and prompts. After the large model analysis is completed, the analysis results of multiple target accounts are summarized and formatted into a word report using the python-docx library.

[0039] The system includes a middle platform service module, which includes: voice recognition interface 1: mainly supports the voice recognition service of the business logic layer, provides voice recognition API. Voice recognition interface 2: mainly supports the voice recognition service of the business logic layer, provides voice recognition API. Qwen-32b: mainly supports the AI intelligent summary service of the business logic layer, provides large model capabilities.

[0040] The system includes a data storage module, including a SQLite database and a video storage library. The SQLite database mainly stores the data of the user interaction part, such as the target account entered by the user, the task record of the voice recognition service, the data record of the AI intelligent summary service, etc. The video storage library is used to store the live streaming obtained by the live monitoring service (downloaded as a video). The above embodiments are only the preferred specific embodiments of the present application. The present specification uses the phrases "in an embodiment", "in another embodiment", "in yet another embodiment" or "in other embodiments", which all refer to one or more of the same or different embodiments according to the present disclosure. The usual changes and replacements made by those skilled in the art within the scope of the technical solutions of the present application should be included in the protection scope of the present application.

Claims

1. A method for forensic analysis of live streaming, characterized in that, include: S1, to monitor the specified target in real time and obtain the broadcast status and live stream; S2, extract the audio from the specified target; S3, perform speech recognition on the audio extracted in step S2 to obtain the live transcript; S4 analyzes the semantic text obtained in S3.

2. The method for network live streaming forensic analysis according to claim 1, characterized in that, In step S1, ffmpeg is used to download the live stream.

3. The method for network live streaming forensic analysis according to claim 1, characterized in that, In step S1, RPA is used to monitor the target live stream status in real time: Hook technology is used to enable the developer tools of the built-in browser, Shadow Blade RPA is used to take over the mouse and keyboard, a scheduled task is set up to control the mouse and keyboard at regular intervals to access the target account homepage and obtain the live stream status and live stream.

4. The method for network live streaming forensic analysis according to claim 3, characterized in that, The RPA module is separated and executed in a virtual machine.

5. The method for network live streaming forensic analysis according to claim 3, characterized in that, In step S4, the large model is called to summarize and analyze the transcript of the live broadcast obtained from speech recognition. Through prompt word engineering, the AI ​​large model is used to infer the context, summarize the video content, and warn of live broadcast risks.

6. The method for network live streaming forensic analysis according to claim 1, characterized in that, In step S2, moviepy is used to extract the audio from the video.

7. A network live streaming evidence collection and analysis system, characterized in that, include: The business logic module includes: real-time monitoring of a specified target to obtain the broadcast status and live stream; extracting the audio of the specified target and performing speech recognition on the extracted audio to obtain the live transcript; and analyzing the obtained semantic text. Mid-platform service module: Provides a speech recognition interface to support business logic modules in performing speech recognition.

8. The network live streaming forensics analysis system according to claim 7, characterized in that, The business logic module includes: Live Streaming Monitoring Unit: For target accounts, use scheduled tasks to send requests to obtain the target account's live streaming status and live stream; or, when using RPA technology to monitor the target's live streaming status in real time, periodically control the mouse and keyboard to access the target account's homepage to obtain the target account's live streaming status and live stream. The speech recognition unit processes the video recorded by the live monitoring unit, first converting it to MP4 video format; then extracting the audio file into MP3 format; and finally uploading the audio file to the speech recognition interface for processing, which will return the speech recognition result. AI Intelligent Summary Unit: Based on the recognition results of the speech recognition unit, the large model is invoked according to preset prompts to perform intelligent summarization. The large model analyzes the sensitivity of the content and provides prompts.

9. The network live streaming forensics analysis system according to claim 7, characterized in that, Use RPA to monitor the target live stream status in real time: Use Hook technology to enable the developer tools of the built-in browser, use Shadow Blade RPA to take over the mouse and keyboard, set up a scheduled task, and control the mouse and keyboard at regular intervals to access the target account's homepage and obtain the live stream status and live stream.

10. The network live streaming forensics analysis system according to claim 7, characterized in that, Use moviepy to extract audio from a video.

Images