Video stream playing and analyzing system and working method thereof

Abstract

The invention provides a video stream playing and analyzing system and a working method thereof, the system performs video stream media playing and video analysis, and the system comprises a multi-video stream playing module, a multi-channel playing module, a video stream comparison module, a code stream size detection module and a video stream broadcasting module; the method comprises the following steps: S1, acquiring chip channel information; s2, acquiring information of a video stream; basic information of the code stream; s3, according to functions, performing S4 playing, S5 detection and S6 comparison respectively; s4, multi-video playing: playing each video stream; s5, code stream size detection: counting code stream information of 256 frames; respectively displaying the code stream information of each frame and the statistical information of the first 256 frames; s6, video stream comparison: respectively acquiring video stream information of the video A to be detected and the video B to be compared; respectively extracting and displaying images in the two videos; obtaining a video C with a mark after frame difference; judging whether the display end displays A or B; after determining a display video, fusing the video with the C; and displaying the final video.

Description

Technical Field

[0001] This invention belongs to the field of video processing technology, and specifically relates to a video stream playback and analysis system and its working method. Background Technology

[0002] Streaming media technology is a technology that transmits audio, video, and multimedia content in a streaming format over the internet. This technology allows users to play audio and video content in real time while it is being transmitted, without waiting for the entire file to download. Below are some key concepts and characteristics of streaming media technology:

[0003] Real-time performance: Streaming technology allows users to start playing audio and video content immediately without waiting for the entire file to download.

[0004] Segmented transmission: Audio and video content is divided into small segments (or streams), transmitted segment by segment and played in real time, reducing latency.

[0005] Adaptive bitrate: Automatically adjusts the bitrate and resolution of audio and video based on the user's network bandwidth and device performance to achieve a better playback experience.

[0006] Multimedia formats: Streaming media can support a variety of audio and video formats, such as MP4, HLS (HTTP Live Streaming), RTMP (Real-Time Messaging Protocol), etc.

[0007] Streaming media server: Used to store, encode, and transmit audio and video content, providing streaming media services.

[0008] The video playback system mainly decodes network data or local video data, converts it into images visible to the human eye, and displays it on the playback panel.

[0009] A video playback system is primarily used to play various video files, providing a user-friendly interface and functions for watching video content.

[0010] A video bitrate analysis system is mainly used to analyze the bitrate information of video files, helping users understand the data rate and video quality of video files.

[0011] However, the shortcomings of existing technologies are:

[0012] 1. Existing video playback systems require opening multiple software systems to play multiple video streams, which is not very convenient;

[0013] 2. Existing video playback systems cannot display the bitrate of the video stream. Real-time display and statistical analysis of bitrate control are not possible.

[0014] 3. Existing playback systems cannot compare video streams, and the differences between each image in the video stream cannot be displayed in real time.

[0015] In addition, the terminology commonly used in this technology includes:

[0016] Ffmpeg is an open-source, cross-platform multimedia processing tool that can be used to process audio, video, and multimedia streams. It includes a rich set of tools and libraries that can perform audio and video encoding / decoding, conversion, and streaming media processing. Summary of the Invention

[0017] In order to solve the above problems, the purpose of this application is to make:

[0018] 1. Enables the playback and analysis system to play multiple video streams simultaneously;

[0019] 2. Combined with Junzheng's chip, it can simultaneously display video streams from different data channels;

[0020] 3. The playback and analysis system can view the bitrate in real time and display the bitrate of each frame in a histogram through statistics;

[0021] 4. The playback and analysis system can compare video streams, and different parts of the video streams will be marked with red blocks.

[0022] Specifically, the present invention provides a video stream playback and analysis system, which performs video streaming media playback and video analysis, including: a multi-video stream playback module, a multi-channel playback module, a video stream comparison module, and a bitstream size detection module; wherein,

[0023] The multi-video stream playback module is used to simultaneously play multiple single video streams;

[0024] First, it is necessary to implement playback of a single video stream. This implementation involves the following three threads working together to control the playback:

[0025] Control thread: It acts as a bridge between the user and the system, receiving user instructions and providing relevant feedback based on those instructions;

[0026] Playback thread: As the system runs, the images on the display interface are constantly refreshed, creating the effect of playing video;

[0027] Exception handling thread: This thread is responsible for monitoring for any abnormal situations.

[0028] Secondly, in the case of multiple video playback, while the system acquires video stream A, multiple threads are used. These multiple threads are in the underlying driver of the software program, and multiple threads handle multiple things. That is, multiple tasks are executed simultaneously in this program, and they are parallel in time. Video streams B, C, etc. are acquired at the same time; multiple video streams, namely video stream A, video stream B, video stream C, etc., are simultaneously handed over to the decoder for decoding.

[0029] The playback process of a single video stream takes a video stream as input and outputs a series of images. Multiple video streams can be acquired using different threads and then sent to the playback process of the single video stream. The video processing, i.e., the system control logic for single video stream playback, handles the video streams. Each single video stream has its own corresponding decoder. After processing, an image and some information corresponding to the image are obtained. This information includes the width and height of the image and the size of the video stream. The image is then displayed on the corresponding interface.

[0030] The multi-channel playback module includes: acquiring chip channel information; acquiring video stream information; and then playing each video stream separately.

[0031] For multi-channel video stream playback, it is for the chip that can simultaneously output multiple video streams of different resolutions. Multiple different streams can be obtained by encoder. The chip can obtain the number of channels of the current video stream, dynamically adjust the number of video streams displayed in real time, and display all video streams of one chip at the same time.

[0032] The bitstream size detection module includes: acquiring chip channel information; acquiring basic information of the bitstream, i.e., the video stream; statistically analyzing the bitstream information of 256 frames; displaying the bitstream information of each frame and the statistical information of the first 256 frames respectively;

[0033] After obtaining the video stream of a chip (which can be set by the user, or multiple streams can be used, or it can be a single stream), the video stream of the corresponding channel is decoded, and after decoding, it is displayed on the corresponding interface, thus enabling the simultaneous display of video streams.

[0034] The video stream comparison module: Assume it acquires the video stream information of the test video A and the comparison video B respectively; extracts and displays images from both videos; and obtains a marked video C after frame difference analysis; determines whether the display should show video A or video B; after determining the video to be displayed, it merges it with video C; and displays the final video. This module supports a maximum of two video comparisons, a rather stringent condition requiring accurate frame matching. The final comparison result is as follows: Figure 9 As shown;

[0035] In image testing, it's common to compare the test video and a comparison video simultaneously. Based on the multi-video stream playback technology mentioned above, the system can acquire multiple video streams concurrently. Let's assume two video streams, A and B, are displayed simultaneously. Let A be the test video and B be the comparison video. When this function module is activated, the system first acquires two frames from each video. These two frames are then processed using a frame difference method to obtain the resulting image after frame difference. If the display terminal chooses to show video A, the frame-differenced image is merged with the original image A, and the overlapping parts are marked. The same process is repeated for video B, resulting in a fused image of A and the frame difference with markings.

[0036] The three threads in the multi-video stream playback module further include:

[0037] Control Thread: The first thread is the control class thread, which acts as a bridge between the user and the system, receiving user commands and providing relevant feedback based on those commands.

[0038] The control thread first needs to obtain the chip's IP address, and then retrieve the video stream information from the chip based on the IP address. If the video stream information can be output normally, it passes the video stream information to the playback thread. If the stream cannot be output, playback is stopped immediately, and the relevant information is displayed. The judgment condition for normal output is as follows: there is a time interval between the start of acquiring the bitstream and receiving the first frame header. If the first frame header is received within the specified time (internal default is 1 second), it is considered that the image can be output normally. If the frame header is not received or the timeout occurs, it is considered that the image cannot be output.

[0039] Playback thread: This thread only starts when the chip is outputting images normally. After obtaining the video information from the video stream in the chip, it begins to extract frames from the video stream. The frames in the video stream are matched with the local decoder to obtain a suitable decoder. By traversing the sequence of all decoders supported by the PC, the video stream is put into the decoding chip one by one. The one that can be successfully decoded is considered a suitable decoder. The suitable decoder is used to decode the video stream into an RGB image. Then, the scaling function is used to match the size of the display interface and display it in the image display interface. As the system runs, the image on the display interface is constantly refreshed, thus creating the effect of playing video.

[0040] Exception Handling Thread: This thread is responsible for monitoring for any abnormal situations. If an exception handling interruption is triggered, it first checks if the frame exists in the video stream. If the frame is missing, it may be due to network fluctuations, causing the frame to be incomplete or its information to be lost. The system discards this frame and retrieves a new frame from the video stream for processing. If the frame itself is fine, the chip will transmit the frame header and trailer information. If both the frame header and trailer are obtained, it is considered a complete frame. However, if the system network is disconnected, it checks if the network is disconnected. If it is, it reconnects to the network. If three consecutive reconnection attempts fail, the system will display an error message indicating a network connection failure, and the system will revert to initialization. If the issue is due to a bitstream abnormality, the system will reacquire the bitstream. If there is no bitstream, playback will stop. If the bitstream is temporarily lost due to network issues or other reasons, simply reacquiring the bitstream will allow playback to resume.

[0041] In the multi-video stream playback module:

[0042] The control thread receives user commands and provides relevant feedback based on those commands, including common functions such as pausing the video, resuming video playback, and taking screenshots of the video.

[0043] The scaling algorithm used in the playback thread is the nearest neighbor interpolation algorithm, which includes: calculating the scaling ratio to obtain the resolution of the target image, i.e., the scaled image; then, using the scaled image, finding the position in the original image based on the scaled coordinates; and replacing the pixel value of the position in the original image with the pixel value of the current point.

[0044] The exception handling thread's exception issues include exception handling when the image is outputting data normally. When the video stream is running normally, exceptions such as video stream stuttering or disconnection may occur due to abnormal chip disconnection or abnormal network fluctuations.

[0045] The bitstream size detection module further includes:

[0046] Bitrate detection is one of the functions of video stream debugging. The bitrate size can indicate whether the bitrate meets the basic requirements of the encoder and whether the encoder is outputting video according to specific frames, including how many frames are between I-frames and how many P-frames follow each I-frame. Based on this information, it can be determined whether the encoder is outputting normally. The bitrate channel display function can capture a segment of the video stream and display the bitrate size of each frame in the captured video stream. It can also detect whether the bitrate fluctuation is within a controllable range over a period of time. Bitrate detection is based on the current bitrate size transmitted from the decoding chip and synchronized to the bitrate image. The system stores the bitrate information of each frame and plots this information in a table to see the changes in the bitrate over a period of time.

[0047] The system's application scenarios include debugging issues with Ingenic's T-series chips and analyzing video streams. It involves functions that interact with the chip. Using this system, one can see the differences in images in video streams between Ingenic's T-series chips and other comparison devices under the same scenario, as well as the differences in bitrate size under the same scenario, facilitating comparison with other devices.

[0048] The chip in the system can obtain the current number of video stream channels and dynamically adjust the number of video streams displayed in real time, so as to achieve the function of displaying all video streams of one chip at the same time.

[0049] This application also relates to a method for operating a video stream playback and analysis system, the method being applicable to the aforementioned system, comprising:

[0050] S1, obtain chip channel information;

[0051] S2, retrieve video stream information; retrieve the bitstream, i.e., the basic information of the video stream.

[0052] S3, according to functional requirements, perform step S4 playback, step S5 detection and step S6 comparison respectively;

[0053] S4, if it is a multi-video playback function, then each video stream will be played separately;

[0054] S5, if it is a bitstream size detection function, then further: count the bitstream information of 256 frames; display the bitstream information of each frame and the statistical information of the first 256 frames respectively;

[0055] S6, if it is a video stream comparison function, then further: obtain the video stream information of the test video A and the comparison video B respectively; extract the images from the two videos and display them respectively; and obtain the marked video C after frame difference; determine whether the display end should display video A or video B; after determining the video to be displayed, merge it with video C; and display the final video.

[0056] Step S4 includes:

[0057] For multi-channel video stream playback, for chips capable of simultaneously outputting multiple video streams of different resolutions, these different streams can be encoded to obtain multiple different video streams. The chip's information can obtain the current number of video stream channels and dynamically adjust the number of video streams displayed in real time, achieving the function of simultaneously displaying all video streams from one chip. Assuming there are 3 channels, information for video streams CH0, 1, and 2 is obtained respectively through step S2. Further, multi-video stream playback for simultaneously playing multiple single video streams is performed, including:

[0058] Video decoding is performed on A, B, and C respectively; the results are displayed on interfaces A, B, and C respectively; further including:

[0059] First, it is necessary to implement the playback of a single video stream, which involves the following three threads working together to complete the control: Control thread: The first thread is the control class thread, which acts as a bridge between the user and the system, receives user commands, and makes relevant feedback based on the user commands, including the functions of pausing the video, resuming video playback, and taking screenshots of the video.

[0060] The control thread first needs to obtain the chip's IP address. Based on the IP address, it retrieves the video stream information from the chip. If the video stream information appears normally, it passes the video stream information to the playback thread. If the stream cannot be output, playback stops immediately, and the relevant information is displayed. The playback thread only starts when it is certain that the chip is outputting images normally. After obtaining the video stream information from the chip, it begins to extract frames from the video stream. It matches the frames in the video stream with the local decoder to obtain a suitable decoder. Using this suitable decoder, the video stream is decoded into an RGB image. Then, the scaling function is used to match the size of the display interface and display it on the image display interface. As the system runs, the image on the display interface is continuously refreshed, creating the effect of playing video.

[0061] Exception handling thread: Used to monitor for any abnormal situations;

[0062] If an exception handling interruption is triggered, the system first checks if the frame exists in the video stream. If the frame is missing, it may be due to network fluctuations, causing the frame to be incomplete or its information to be lost. The system discards this frame and retrieves a new frame from the video stream for processing. If the frame itself is fine, the chip will transmit the frame header and trailer information. If both the frame header and trailer are obtained, it is considered a complete frame. However, if the network is disconnected, the system checks if the network is disconnected. If it is, it tries to reconnect to the network. If three consecutive attempts to reconnect to the network fail, the system will display an error message indicating a network connection failure, and the system will return to initialization. If the problem is due to a bitstream error, the system will reacquire the bitstream. If there is no bitstream, playback will stop. If the bitstream is temporarily lost due to network issues or other reasons, simply reacquire the bitstream and playback will resume.

[0063] Secondly, in the case of multiple video playback, while the system acquires video stream A, multiple threads are used to acquire video stream B simultaneously. Both video streams A and B are simultaneously handed over to the decoder for decoding. From the system software perspective, the playback process of a single video stream can be seen as a black box: the input is a video stream, and the output is a series of images. Acquiring multiple video streams, such as video streams A and B, can be done using different threads, and then sent to the black box of single video stream playback. The video processing core can be understood as the system control logic for single video stream playback to process it. After processing, an image and its corresponding information are obtained, and then the image is displayed on the corresponding interface.

[0064] Step S5 further includes:

[0065] After acquiring the video stream of a chip, the video stream of the corresponding channel is decoded. After decoding, it is displayed on the corresponding interface, thus enabling simultaneous display of the video stream. The bit stream size detection function is a very important function in video stream debugging. From the bit stream size, we can see whether the bit stream meets the basic requirements of the encoder and whether the encoder is outputting video according to specific frames, including how many frames are between I-frames and how many P-frames follow the I-frames. Based on this information, we can determine whether the encoder is outputting normally.

[0066] The bitstream channel display function can capture a segment of video stream and display the bitstream size of each frame in the captured video stream; it can also detect whether the bitstream fluctuation is within a controllable range over a period of time. The bitstream size detection is based on the current bitstream size transmitted from the decoding chip and synchronized to the bitstream image. The system stores the bitstream information of each frame and plots this information in a table, which shows the changes in the bitstream over a period of time.

[0067] Step S6 further includes:

[0068] Video stream comparison is a common feature in image testing, where a test video and a comparison video are compared simultaneously. Based on multi-video stream playback technology, the system can simultaneously acquire and display, for example, test video A and comparison video B. Let's assume A is the test video and B is the comparison video. When this function is activated, the system first acquires two frames from each video. These two frames are then processed using a frame difference method to obtain the resulting image after frame difference. If the display device chooses to show video A, the frame-differenced image is merged with the original image A, and the overlapping parts are marked. The process is repeated for video B, and so on. This results in a marked image of A and the merged frame-differenced image.

[0069] Therefore, the advantage of this application is:

[0070] 1. Resolved the issue of incomparability between video streams;

[0071] 2. A solution was found that allows simultaneous display of video streams from different data channels using video streaming.

[0072] 3. Solved the problem of simultaneously playing multiple video streams within a single playback system in multi-camera projects;

[0073] 4. The interface for viewing the bitrate of the video stream has been improved, making it easier for users to quickly find the bitrate and debug the correspondence between the bitrate and the network. Attached Figure Description

[0074] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0075] Figure 1 This is a schematic diagram of the functional framework of the video stream playback and analysis system of the present invention.

[0076] Figure 2 This is a schematic diagram of the control flow for the first step of the multi-video playback process in this invention: the playback of a single video stream.

[0077] Figure 3 This is a schematic diagram illustrating how, in the case of multiple video playback in this invention, the system simultaneously acquires video streams A and B and hands them over to the decoder for decoding.

[0078] Figure 4 This is a flowchart illustrating the function of simultaneously displaying all video streams from one or more chips in this invention.

[0079] Figure 5This is a schematic diagram illustrating the process of simultaneously displaying video streams in this invention.

[0080] Figure 6 This is a screenshot of a table showing how the bitstream information of each frame is plotted on a single table.

[0081] Figure 7 This is a flowchart illustrating the process of implementing the video stream comparison function in this invention.

[0082] Figures 8(1) and 8(2) are schematic diagrams of the two consecutive frames of video in the example of this invention.

[0083] Figure 9 These are schematic diagrams of the markings after the video playback is completed, as shown in the examples in Figures 8(1) and 8(2).

[0084] Figure 10 This is a schematic diagram illustrating the final effect of the invention.

[0085] Figure 11 This is a schematic diagram of the steps in the method of the present invention. Detailed Implementation

[0086] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0087] This invention designs a video stream playback and analysis system and method for video streaming playback and analysis. Specifically, it is a video stream playback and analysis system customized for Ingenic chips, primarily used for debugging issues with Ingenic's T-series chips and analyzing video streams. This mainly involves functions that interact with the chip, such as specific functions required by Ingenic chips, and compatibility only with Ingenic chips. Using this system and method, differences in image quality and bitrate can be observed between Ingenic T-series chips and other comparison devices under the same conditions, primarily for comparison with other devices.

[0088] The video stream playback and analysis system designed in this invention mainly includes the following five functional modules, such as: Figure 1 As shown, it includes: a multi-video stream playback module, a multi-channel playback module, a video stream comparison module, and a bitrate detection module; some systems also include a video stream broadcasting module. Video stream broadcasting mainly involves acquiring the streams of all currently playing devices within the local area network. Video stream broadcasting refers to traversing all IPs within the local area network, determining if a video stream exists on that IP, and recording it if it does for user selection. This function is not the core function of this application; it is a bypass function and will not be elaborated further. Among these:

[0089] The multi-video stream playback module is used to simultaneously play multiple single video streams. Multi-video stream playback can be understood as the superposition of multiple single video stream playbacks. Multi-video stream playback is achieved by playing single video streams. The core is to play one video stream using one thread and one display interface. Multi-video stream playback involves multiple threads and multiple display interfaces.

[0090] First, it is necessary to implement playback of a single video stream, such as... Figure 2 As shown, the playback of a single video stream is controlled by the following three threads:

[0091] Control Thread: The first thread is the control class thread, whose main role is to act as a bridge between the user and the system, receive user commands, and make relevant responses based on the user commands, such as common functions like pausing video, resuming video playback, and taking screenshots of video.

[0092] The first step for the control thread is to obtain the chip's IP address. Based on the IP address, it retrieves the video stream information from the chip. If the video stream information can be output normally, it is passed to the playback thread. If the stream cannot be output, playback is stopped immediately, and the relevant information is displayed.

[0093] The playback thread is activated only when the chip is outputting images normally. After acquiring the video information from the video stream in the chip, it extracts frames from the video stream, matches them with the local decoder to obtain a suitable decoder, and uses this suitable decoder to decode the video stream into an RGB image. Then, it uses a scaling function to match the size of the display interface and displays it on the image display interface. As the system runs, the image on the display interface is continuously refreshed, creating the effect of playing video. The scaling algorithm used can be the nearest neighbor interpolation algorithm. The main process is to calculate the scaling ratio to obtain the resolution of the target image (the scaled image), and then, using the scaled image and its coordinates, find its position in the original image, and replace the pixel value of the current point with the pixel value of the position in the original image.

[0094] The exception handling thread primarily handles exceptions when the image data is being transmitted normally. During normal video streaming, issues such as video stream interruptions or drops may occur due to chip malfunctions or network fluctuations. This thread monitors for these anomalies. If an exception handling interruption is triggered, it first checks if the frame exists in the video stream. If the frame is missing, it might be due to network fluctuations causing the frame to be incomplete or its information to be lost. The system discards this frame and retrieves a new frame from the video stream for processing. If the frame itself is intact (the chip transmits the frame header and trailer), and both are obtained, it's considered a complete frame. However, if the network is disconnected, the system checks for network disconnection. If disconnected, it reconnects. If three consecutive reconnection attempts fail, the system displays an error message indicating a network connection failure and performs a complete system reset. If the problem is due to an abnormal bitstream, the system will reacquire the bitstream. If there is no bitstream, playback will stop. If the bitstream is temporarily lost due to network or other reasons, simply reacquire the bitstream and playback will resume.

[0095] Secondly, in the case of multi-video playback, while the system acquires video stream A, multiple threads simultaneously acquire video stream B, and both video streams are simultaneously sent to the decoder for decoding. From the system software perspective, the playback process of a single video stream can be seen as a black box: the input is a video stream, and the output is a series of images. Acquiring video streams A and B can be done using different threads, and then they are sent to the single-video stream playback black box. The video processing core can be understood as the system control logic for single-video stream playback, which processes the data. After processing, an image and its corresponding information are obtained, and then the image is displayed on the corresponding interface, such as... Figure 3 As shown, this includes: assuming that information from video streams A and B is obtained respectively; corresponding video decoding of A and B is performed respectively; and the information is displayed in interfaces A and B respectively.

[0096] The multi-channel playback module includes: acquiring chip channel information; acquiring video stream information; and then playing each video stream separately.

[0097] For multi-channel video stream playback, this function is primarily designed specifically for chips like those from Ingenic Semiconductor. Ingenic's series of chips can communicate with the system via the network to obtain internal chip information. For example, Ingenic's T-series chips can simultaneously output up to three different resolution video streams. These three different streams are encoded to produce three different video streams. In traditional video playback systems, only one stream can be displayed at a time, which is inconvenient when verifying scaling algorithms, requiring frequent interface switching and making it impossible to see the difference between the normal and scaled streams. This invention addresses this by allowing the chip to obtain the current number of video stream channels and dynamically adjust the number of video streams displayed in real-time, achieving the function of simultaneously displaying all video streams from a single chip. Figure 4 As shown, the process includes: acquiring chip channel information; assuming there are 3 channels, acquiring video stream information for CH0, 1, and 2 respectively; performing video decoding A, B, and C respectively; and displaying the information in interfaces A, B, and C respectively.

[0098] The bitstream size detection module: such as Figure 5 As shown, it includes: obtaining chip channel information; obtaining basic information of the bitstream; statistically analyzing the bitstream information of 256 frames; displaying the bitstream information of each frame and the statistical information of the first 256 frames respectively;

[0099] After obtaining the video stream from a chip (which can be multiple streams or a single stream, as set by the user), the corresponding video stream is decoded and then displayed on the corresponding interface, thus enabling simultaneous display of video streams.

[0100] Bitrate detection is a crucial function in video stream debugging. The bitrate size reveals whether the bitrate meets the encoder's basic requirements and whether the encoder is outputting video in a specific frame-by-frame manner, such as the interval between I-frames and the number of P-frames following each I-frame. This information helps determine if the encoder is outputting correctly. The bitrate channel display function allows you to capture a segment of the video stream and easily display the bitrate size of each frame. It can also detect bitrate fluctuations within a controllable range over a period of time. Bitrate detection is primarily based on the current bitrate size transmitted from the decoding chip, synchronized to the bitrate image. The system stores the bitrate information for each frame and plots this information in a table, clearly showing the bitrate changes over a period of time. A screenshot of the table is shown below. Figure 6 As shown.

[0101] The video stream comparison module: acquires video stream information of the test video A and the comparison video B respectively; extracts and displays images from both videos; obtains a marked video C after frame difference; determines whether to display video A or video B; after determining the video to be displayed, merges it with video C; and displays the final video.

[0102] The video stream comparison function is also a very distinctive feature of this invention. During image testing, it's common to compare test and comparison videos. The conventional approach is to extract images from both videos and use image comparison tools to compare the differences, or to visually distinguish the differences between the video streams. However, this method has significant margins of error, and some details are imperceptible to the naked eye. Using image analysis tools, in certain scenarios, some problematic videos are fleeting and cannot be saved in time, making debugging impossible and causing considerable trouble for debugging. This invention effectively solves this problem. The specific process is as follows: Figure 7 As shown. Based on the multi-video stream playback technology mentioned above, the system can simultaneously acquire and display two video streams, A and B. Assume that A is the video to be tested and B is the comparison video. When the system starts this function, it first acquires two frames from the video to be tested and the comparison video respectively. The two frames are processed using the frame difference method to obtain the image after the frame difference. If the display end chooses to display video A, the image after the frame difference is fused with the original image A. The overlapping part is marked in red. If the display end chooses to display video B, the same applies. Then an image with red markings is obtained by fusing A and the frame difference. For better understanding, the two frames of video are shown in Figure 8(1) and Figure 8(2). The marked image after the video playback is completed is shown in Figure 8(1). Figure 9 As shown in the image, displaying this image in the interface will show the differences between the original video and the comparison video.

[0103] like Figure 10 The image shown is a demonstration of the final effect of this system.

[0104] This application discloses a working method for a video stream playback and analysis system, applicable to any of the aforementioned systems, such as... Figure 11 As shown, it includes:

[0105] S1, obtain chip channel information;

[0106] S2, retrieve video stream information; retrieve the bitstream, i.e., the basic information of the video stream.

[0107] S3, according to functional requirements, perform step S4 playback, step S5 detection and step S6 comparison respectively;

[0108] S4, if it is a multi-video playback function, then each video stream will be played separately;

[0109] S5, if it is a bitstream size detection function, then further: count the bitstream information of 256 frames; display the bitstream information of each frame and the statistical information of the first 256 frames respectively;

[0110] S6, if it is a video stream comparison function, then further: obtain the video stream information of the test video A and the comparison video B respectively; extract the images from the two videos and display them respectively; and obtain the marked video C after frame difference; determine whether the display end should display video A or video B; after determining the video to be displayed, merge it with video C; and display the final video.

[0111] Step S4 includes:

[0112] For multi-channel video stream playback, for chips capable of simultaneously outputting multiple video streams of different resolutions, these different streams can be encoded to obtain multiple different video streams. The chip's information can obtain the current number of video stream channels and dynamically adjust the number of video streams displayed in real time, achieving the function of simultaneously displaying all video streams from one chip. Assuming there are 3 channels, information for video streams CH0, 1, and 2 is obtained respectively through step S2. Further, multi-video stream playback for simultaneously playing multiple single video streams is performed, including:

[0113] Video decoding is performed on A, B, and C respectively; the results are displayed on interfaces A, B, and C respectively; further including:

[0114] First, it is necessary to implement the playback of a single video stream, which involves the following three threads working together to complete the control: Control thread: The first thread is the control class thread, which acts as a bridge between the user and the system, receives user commands, and makes relevant feedback based on the user commands, including the functions of pausing the video, resuming video playback, and taking screenshots of the video.

[0115] The control thread first needs to obtain the chip's IP address. Based on the IP address, it retrieves the video stream information from the chip. If the video stream can be output normally, it passes the video stream information to the playback thread. If it cannot output, it immediately stops playback and displays the relevant information. The playback thread only starts when it is sure that the chip is outputting normally. After obtaining the video stream information from the chip, it begins to extract frames from the video stream. It matches the frames in the video stream with the local decoder to obtain a suitable decoder. Using the suitable decoder, the video stream is decoded into an RGB image. Then, the scaling function is used to match the size of the display interface and display it on the image display interface. As the system runs, the image on the display interface is continuously refreshed, creating the effect of playing video.

[0116] Exception handling thread: Used to monitor for any abnormal situations;

[0117] If an exception handling interruption is triggered, the system first checks if the frame exists in the video stream. If the frame is missing, it may be due to network fluctuations, causing the frame to be incomplete or its information to be lost. The system discards this frame and retrieves a new frame from the video stream for processing. If the frame itself is fine, the chip will transmit the frame header and trailer information. If both the frame header and trailer are obtained, it is considered a complete frame. However, if the network is disconnected, the system checks if the network is disconnected. If it is, it tries to reconnect to the network. If three consecutive attempts to reconnect to the network fail, the system will display an error message indicating a network connection failure, and the system will return to initialization. If the problem is due to a bitstream error, the system will reacquire the bitstream. If there is no bitstream, playback will stop. If the bitstream is temporarily lost due to network issues or other reasons, simply reacquire the bitstream and playback will resume.

[0118] Secondly, in the case of multiple video playback, while the system acquires video stream A, multiple threads are used to acquire video stream B simultaneously. Both video streams A and B are simultaneously handed over to the decoder for decoding. From the perspective of system software, the playback process of a single video stream can be seen as a black box: the input is a video stream, and the output is a series of images. Acquiring multiple video streams, such as video streams A and B, can be done using different threads, and then sent to the black box of single video stream playback. The video processing core can be understood as the system control logic of single video stream playback for processing. After processing, an image and some information corresponding to the image are obtained, and then the image is displayed on the corresponding interface.

[0119] Step S5 further includes:

[0120] After acquiring the video stream of a chip, the video stream of the corresponding channel is decoded. After decoding, it is displayed on the corresponding interface, thus enabling simultaneous display of the video stream. The bit stream size detection function is a very important function in video stream debugging. From the bit stream size, we can see whether the bit stream meets the basic requirements of the encoder and whether the encoder is outputting video according to specific frames, including how many frames are between I-frames and how many P-frames follow the I-frames. Based on this information, we can determine whether the encoder is outputting normally.

[0121] The bitstream channel display function can capture a segment of video stream and display the bitstream size of each frame in the captured video stream; it can also detect whether the bitstream fluctuation is within a controllable range over a period of time. The bitstream size detection is based on the current bitstream size transmitted from the decoding chip and synchronized to the bitstream image. The system stores the bitstream information of each frame and plots this information in a table, which shows the changes in the bitstream over a period of time.

[0122] Step S6 further includes:

[0123] Video stream comparison is a common feature in image testing, where a test video and a comparison video are compared simultaneously. Based on multi-video stream playback technology, the system can simultaneously acquire and display, for example, test video A and comparison video B. Let's assume A is the test video and B is the comparison video. When this function is activated, the system first acquires two frames from each video. These two frames are then processed using a frame difference method to obtain the resulting image after frame difference. If the display device chooses to show video A, the frame-differenced image is merged with the original image A, and the overlapping parts are marked. The process is repeated for video B, and so on. This results in a marked image of A and the merged frame-differenced image.

[0124] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A video stream playback and analysis system, characterized in that, The system performs video streaming playback and video analysis, including: a multi-video stream playback module, a multi-channel playback module, a video stream comparison module, and a bitrate detection module; wherein, The multi-video stream playback module is used to simultaneously play multiple single video streams; First, it is necessary to implement playback of a single video stream. This implementation involves the following three threads working together to control the playback: Control thread: It acts as a bridge between the user and the system, receiving user instructions and providing relevant feedback based on those instructions; Playback thread: As the system runs, the images on the display interface are constantly refreshed, creating the effect of playing video; Exception handling thread: This thread is responsible for monitoring for any abnormal situations. Secondly, in the case of multiple video playback, while the system acquires video stream A, multiple threads are used. These multiple threads are in the underlying driver of the software program, and multiple threads handle multiple things. That is, multiple tasks are executed simultaneously in this program, and they are parallel in time. Video streams B, C, etc. are acquired at the same time; multiple video streams, namely video stream A, video stream B, video stream C, etc., are simultaneously handed over to the decoder for decoding. The playback process of a single video stream takes a video stream as input and outputs a series of images. Multiple video streams can be acquired using different threads and then sent to the playback process of the single video stream. The video processing, i.e., the system control logic for single video stream playback, handles the video streams. Each single video stream has its own corresponding decoder. After processing, an image and some information corresponding to the image are obtained. This information includes the width and height of the image and the size of the video stream. The image is then displayed on the corresponding interface. The multi-channel playback module includes: acquiring chip channel information; acquiring video stream information; and then playing each video stream separately. For multi-channel video stream playback, it is for the chip that can simultaneously output multiple video streams of different resolutions. Multiple different streams can be obtained by encoder. The chip can obtain the number of channels of the current video stream, dynamically adjust the number of video streams displayed in real time, and display all video streams of one chip at the same time. The bitstream size detection module includes: acquiring chip channel information; acquiring basic information of the bitstream, i.e., the video stream; statistically analyzing the bitstream information of 256 frames; displaying the bitstream information of each frame and the statistical information of the first 256 frames respectively; After obtaining the video stream of a chip (which can be set by the user, or multiple streams can be used, or it can be a single stream), the video stream of the corresponding channel is decoded, and after decoding, it is displayed on the corresponding interface, thus enabling the simultaneous display of video streams. The video stream comparison module: assumes that it acquires video stream information of the test video A and the comparison video B respectively; extracts and displays images from both videos respectively; and obtains a marked video C after frame difference; determines whether the display should show video A or video B; after determining the video to be displayed, it merges it with video C; and displays the final video. In image testing, it's common to compare the test video and a comparison video simultaneously. Based on the multi-video stream playback technology mentioned above, the system can acquire multiple video streams concurrently. Let's assume two video streams, A and B, are displayed simultaneously. Let A be the test video and B be the comparison video. When this function module is activated, the system first acquires two frames from each video. These two frames are then processed using a frame difference method to obtain the resulting image after frame difference. If the display terminal chooses to show video A, the frame-differenced image is merged with the original image A, and the overlapping parts are marked. The same process is repeated for video B, resulting in a fused image of A and the frame difference with markings.

2. The video stream playback and analysis system according to claim 1, characterized in that, The three threads in the multi-video stream playback module further include: Control Thread: The first thread is the control class thread, which acts as a bridge between the user and the system, receiving user commands and providing relevant feedback based on those commands. The control thread first needs to obtain the chip's IP address, and then retrieve the video stream information from the chip based on the IP address. If the video stream information can be output normally, it passes the video stream information to the playback thread. If the stream cannot be output, playback is stopped immediately, and the relevant information is displayed. The judgment condition for normal output is as follows: there is a time interval between the start of acquiring the bitstream and receiving the first frame header. If the first frame header is received within the specified time (internal default is 1 second), it is considered that the image can be output normally. If the frame header is not received or the timeout occurs, it is considered that the image cannot be output. Playback thread: This thread only starts when the chip is outputting images normally. After obtaining the video information from the video stream in the chip, it begins to extract frames from the video stream. The frames in the video stream are matched with the local decoder to obtain a suitable decoder. By traversing the sequence of all decoders supported by the PC, the video stream is put into the decoding chip one by one. The one that can be successfully decoded is considered a suitable decoder. The suitable decoder is used to decode the video stream into an RGB image. Then, the scaling function is used to match the size of the display interface and display it in the image display interface. As the system runs, the image on the display interface is constantly refreshed, thus creating the effect of playing video. Exception Handling Thread: This thread is responsible for monitoring for any abnormal situations. If an exception handling interruption is triggered, it first checks if the frame exists in the video stream. If the frame is missing, it may be due to network fluctuations, causing the frame to be incomplete or its information to be lost. The system discards this frame and retrieves a new frame from the video stream for processing. If the frame itself is fine, the chip will transmit the frame header and trailer information. If both the frame header and trailer are obtained, it is considered a complete frame. However, if the system network is disconnected, it checks if the network is disconnected. If it is, it reconnects to the network. If three consecutive reconnection attempts fail, the system will display an error message indicating a network connection failure, and the system will revert to initialization. If the issue is due to a bitstream abnormality, the system will reacquire the bitstream. If there is no bitstream, playback will stop. If the bitstream is temporarily lost due to network issues or other reasons, simply reacquiring the bitstream will allow playback to resume.

3. The video stream playback and analysis system according to claim 2, characterized in that, In the multi-video stream playback module: The control thread receives user commands and provides relevant feedback based on those commands, including common functions such as pausing the video, resuming video playback, and taking screenshots of the video. The scaling algorithm used in the playback thread is the nearest neighbor interpolation algorithm, which includes: calculating the scaling ratio to obtain the resolution of the target image, i.e., the scaled image; then, using the scaled image, finding the position in the original image based on the scaled coordinates; and replacing the pixel value of the position in the original image with the pixel value of the current point. The exception handling thread's exception issues include exception handling when the image is outputting data normally. When the video stream is running normally, exceptions such as video stream stuttering or disconnection may occur due to abnormal chip disconnection or abnormal network fluctuations.

4. The video stream playback and analysis system according to claim 1, characterized in that, The bitstream size detection module further includes: Bitrate detection is one of the functions of video stream debugging. The bitrate size can indicate whether the bitrate meets the basic requirements of the encoder and whether the encoder is outputting video according to specific frames, including how many frames are between I-frames and how many P-frames follow each I-frame. Based on this information, it can be determined whether the encoder is outputting normally. The bitrate channel display function can capture a segment of the video stream and display the bitrate size of each frame in the captured video stream. It can also detect whether the bitrate fluctuation is within a controllable range over a period of time. Bitrate detection is based on the current bitrate size transmitted from the decoding chip and synchronized to the bitrate image. The system stores the bitrate information of each frame and plots this information in a table to see the changes in the bitrate over a period of time.

5. The video stream playback and analysis system according to claim 1, characterized in that, The system's application scenarios include debugging issues with Ingenic's T-series chips and analyzing video streams. It involves functions that interact with the chip. Using this system, one can see the differences in images in video streams between Ingenic's T-series chips and other comparison devices under the same scenario, as well as the differences in bitrate size under the same scenario, facilitating comparison with other devices.

6. The video stream playback and analysis system according to claim 1, characterized in that, The chip in the system can obtain the current number of video stream channels and dynamically adjust the number of video streams displayed in real time, so as to achieve the function of displaying all video streams of one chip at the same time.

7. A method for operating a video stream playback and analysis system, characterized in that, The method is applicable to any of the systems described in claims 1-6 above, comprising: S1, obtain chip channel information; S2, retrieve video stream information; retrieve the bitstream, i.e., the basic information of the video stream. S3, according to functional requirements, perform step S4 playback, step S5 detection and step S6 comparison respectively; S4, if it is a multi-video playback function, then each video stream will be played separately; S5, if it is a bitstream size detection function, then further: count the bitstream information of 256 frames; display the bitstream information of each frame and the statistical information of the first 256 frames respectively; S6, if it is a video stream comparison function, then further: obtain the video stream information of the test video A and the comparison video B respectively; extract the images from the two videos and display them respectively; and obtain the marked video C after frame difference; determine whether the display end should display video A or video B; after determining the video to be displayed, merge it with video C; and display the final video.

8. The working method of a video stream playback and analysis system according to claim 7, characterized in that, Step S4 includes: For multi-channel video stream playback, for chips capable of simultaneously outputting multiple video streams of different resolutions, these different streams can be encoded to obtain multiple different video streams. The chip's information can obtain the current number of video stream channels and dynamically adjust the number of video streams displayed in real time, achieving the function of simultaneously displaying all video streams from one chip. Assuming there are 3 channels, information for video streams CH0, 1, and 2 is obtained respectively through step S2. Further, multi-video stream playback for simultaneously playing multiple single video streams is performed, including: Video decoding is performed on A, B, and C respectively; the results are displayed on interfaces A, B, and C respectively; further including: First, playback of a single video stream needs to be implemented, which involves the following three threads working together to control it: Control Thread: The first thread is the control class thread, which acts as a bridge between the user and the system, receiving user commands and making relevant responses based on the user commands, including functions such as pausing video, resuming video playback, and taking screenshots of video. The control thread first needs to obtain the chip's IP address. Based on the IP address, it retrieves the video stream information from the chip. If the video stream information appears normally, it passes the video stream information to the playback thread. If the stream cannot be output, playback stops immediately, and the relevant information is displayed. The playback thread only starts when it is certain that the chip is outputting images normally. After obtaining the video stream information from the chip, it begins to extract frames from the video stream. It matches the frames in the video stream with the local decoder to obtain a suitable decoder. Using this suitable decoder, the video stream is decoded into an RGB image. Then, the scaling function is used to match the size of the display interface and display it on the image display interface. As the system runs, the image on the display interface is continuously refreshed, creating the effect of playing video. Exception handling thread: Used to monitor for any abnormal situations; If an exception handling interruption is triggered, the system first checks if the frame exists in the video stream. If the frame is missing, it may be due to network fluctuations, causing the frame to be incomplete or its information to be lost. The system discards this frame and retrieves a new frame from the video stream for processing. If the frame itself is fine, the chip will transmit the frame header and trailer information. If both the frame header and trailer are obtained, it is considered a complete frame. However, if the network is disconnected, the system checks if the network is disconnected. If it is, it tries to reconnect to the network. If three consecutive attempts to reconnect to the network fail, the system will display an error message indicating a network connection failure, and the system will return to initialization. If the problem is due to a bitstream error, the system will reacquire the bitstream. If there is no bitstream, playback will stop. If the bitstream is temporarily lost due to network issues or other reasons, simply reacquire the bitstream and playback will resume. Secondly, in the case of multiple video playback, while the system acquires video stream A, multiple threads are used to acquire video stream B simultaneously. Both video streams A and B are simultaneously handed over to the decoder for decoding. From the system software perspective, the playback process of a single video stream can be seen as a black box: the input is a video stream, and the output is a series of images. Acquiring multiple video streams, such as video streams A and B, can be done using different threads, and then sent to the black box of single video stream playback. The video processing core can be understood as the system control logic for single video stream playback to process it. After processing, an image and its corresponding information are obtained, and then the image is displayed on the corresponding interface.

9. The working method of a video stream playback and analysis system according to claim 8, characterized in that, Step S5 further includes: After acquiring the video stream of a chip, the video stream of the corresponding channel is decoded. After decoding, it is displayed on the corresponding interface, thus enabling simultaneous display of the video stream. The bit stream size detection function is a very important function in video stream debugging. From the bit stream size, we can see whether the bit stream meets the basic requirements of the encoder and whether the encoder is outputting video according to specific frames, including how many frames are between I-frames and how many P-frames follow the I-frames. Based on this information, we can determine whether the encoder is outputting normally. The bitstream channel display function can capture a segment of video stream and display the bitstream size of each frame in the captured video stream; it can also detect whether the bitstream fluctuation is within a controllable range over a period of time. The bitstream size detection is based on the current bitstream size transmitted from the decoding chip and synchronized to the bitstream image. The system stores the bitstream information of each frame and plots this information in a table, which shows the changes in the bitstream over a period of time.

10. The working method of a video stream playback and analysis system according to claim 8, characterized in that, Step S6 further includes: Video stream comparison is a common feature in image testing, where a test video and a comparison video are compared simultaneously. Based on multi-video stream playback technology, the system can simultaneously acquire and display, for example, test video A and comparison video B. Let's assume A is the test video and B is the comparison video. When this function is activated, the system first acquires two frames from each video. These two frames are then processed using a frame difference method to obtain the resulting image after frame difference. If the display device chooses to show video A, the frame-differenced image is merged with the original image A, and the overlapping parts are marked. The process is repeated for video B, and so on. This results in a marked image of A and the merged frame-differenced image.

Images