Abnormality analysis method of video stream, electronic equipment and vehicle

Abstract

The invention provides a video stream anomaly analysis method, which comprises the following steps: processing received video stream source data to play video content corresponding to the video stream source data; determining target video stream source data corresponding to the abnormal video content in response to determining that the playing of the video content is abnormal; and based on the target video stream source data, determining a reason for abnormal video playing. According to the method, the target video stream source data corresponding to the abnormal video content is determined, so that the technical effect of tracing when the video playing is abnormal is realized, the abnormal reason can be quickly positioned, and the practicability of the method is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method for analyzing abnormalities in a video stream, an electronic device, and a vehicle. Background Art

[0002] Existing in-vehicle video streams are mainly processed by in-vehicle applications developed by car manufacturers, which involves framing and decoding from the hardware layer to the application layer. Problems are very likely to occur in these processing links at the application layer, resulting in abnormal video playback. These abnormalities are difficult to predict and difficult to trace through traditional log records. Summary of the invention

[0003] In view of this, the purpose of the present application is to propose a method, electronic device and vehicle for analyzing anomalies in video streams, so as to solve the problem that anomalies in video playback cannot be traced.

[0004] Based on the above purpose, the present application provides a method for analyzing abnormalities in a video stream, comprising:

[0005] Processing the received video stream source data to play the video content corresponding to the video stream source data;

[0006] In response to determining that the video content playback is abnormal, determining target video stream source data corresponding to the abnormal video content;

[0007] Based on the target video stream source data, the cause of the abnormal video playback is determined.

[0008] Further, the video stream source data is marked with a timestamp;

[0009] In response to determining that the video playback is abnormal, determining the target video stream source data corresponding to the abnormal video content includes:

[0010] In response to determining that the video playback is abnormal, determining a first time point at which the video playback is abnormal and a second time point at which the video playback stops being abnormal;

[0011] Determine, based on the first time point and the second time point, a first time period in which the video playback is abnormal;

[0012] Target video stream source data having a timestamp including the first time period is determined from the stored video stream source data.

[0013] Further, the determining the cause of the abnormal video playback based on the target video stream source data includes:

[0014] Analyze the target video stream source data to determine whether the target video stream source data is abnormal;

[0015] In response to determining that the target video stream source data is abnormal, determining that the cause of the abnormal video playback is a problem with a sending end of the video stream source data;

[0016] In response to determining that there is no abnormality in the target video stream source data, it is determined based on the target video stream source data whether there is a problem in the processing of the target video stream source data.

[0017] Furthermore, the processing of the video stream source data includes:

[0018] Performing frame processing on the video stream source data to obtain frame data, and performing frame sequence integrity detection on the frame data;

[0019] In response to determining that there is abnormal frame sequence data in the frame data, marking the abnormal frame sequence data;

[0020] Decoding the framed data to obtain decoded data, and detecting the frame sequence integrity of the decoded data;

[0021] In response to determining that frame sequence abnormal data exists in the decoded data, the frame sequence abnormal data is marked.

[0022] Further, the determining the cause of the abnormal video playback based on the target video stream source data includes:

[0023] Based on the target video stream source data, determining target frame data and target decoded data corresponding to the target video stream source data from the stored frame data and decoded data;

[0024] Determine whether the target frame data and the target decoded data have a frame sequence abnormality mark;

[0025] In response to determining that both the target frame data and the target decoded data have frame sequence abnormality marks, determining that the cause of the abnormal video playback is a problem in the frame processing process;

[0026] In response to determining that the target frame data does not have a frame sequence abnormality mark, and the target decoded data has a frame sequence abnormality mark, determining that the cause of the abnormal video playback is a problem in the decoding process;

[0027] In response to determining that neither the target frame data nor the target decoded data has a frame sequence abnormality mark, it is determined whether there is a problem with the image content of the target frame data and the target decoded data.

[0028] Further, the determining whether there is a problem with the image content of the target frame data and the target decoded data includes:

[0029] Analyzing the image contents of the target frame data and the target decoded data respectively to determine whether the target frame data and the target decoded data contain abnormal image content data;

[0030] In response to determining that both the target frame data and the target decoded data have abnormal image content data, determining that there is a problem in the frame processing process;

[0031] In response to determining that the target frame data does not contain abnormal image content data, and the target decoded data contains abnormal image content data, it is determined that there is a problem in the decoding process.

[0032] Further, after determining that there is a problem in the framing process, the method further includes:

[0033] Analyze the framing algorithm corresponding to the framing processing to locate the loopholes in the framing algorithm.

[0034] Further, after determining that there is a problem in the decoding process, the method further includes:

[0035] Repair the decoder corresponding to the decoding process to locate the decoder fault.

[0036] Based on the same inventive concept, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0037] Based on the same inventive concept, the present disclosure also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method as described above.

[0038] From the above, it can be seen that the present application provides a method for analyzing anomalies in a video stream, which processes the video stream source data to play the video content corresponding to the video stream source data, and monitors the played video content for anomalies while playing the video content. When an abnormality occurs in the video content during playback, the target video stream source data corresponding to the abnormal video content is determined, and on this basis, the cause of the abnormality in the video content playback is determined to achieve anomaly monitoring during video playback. By determining the target video stream source data corresponding to the abnormal video content, the technical effect of tracing back when an abnormality occurs in video playback is achieved, which is conducive to quickly locating the cause of the abnormality and improving the practicality of the present method. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A schematic diagram of the process structure of a method for analyzing abnormalities in a video stream according to an embodiment of the present application;

[0041] Figure 2 This is a schematic diagram of the processing process of video stream source data in an embodiment of the present application;

[0042] Figure 3 This is a schematic diagram of the structure of a device for analyzing abnormalities in a video stream according to an embodiment of the present application;

[0043] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0045] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0046] As described in the background technology, with the development of in-vehicle entertainment systems, real-time transmission and processing of in-vehicle video streams has become one of the key technologies in in-vehicle applications. However, existing in-vehicle video streams mainly rely on in-vehicle applications developed by car manufacturers for processing, which involves framing and decoding issues from the hardware layer to the application layer. Due to the complexity of the in-vehicle environment, these links are prone to problems, resulting in abnormal playback images. These abnormal problems are difficult to predict and difficult to trace through traditional log records. For example, when the screen is distorted, stuck or jittery, there may not be any significant abnormal records in the process, making the problem difficult to restore and analyze. There is currently no effective solution to comprehensively trace and deal with these problems.

[0047] Since there are corresponding quality requirements when the vehicle leaves the factory: before the vehicle leaves the factory, the quality of the transmission and playback of the on-board video stream must be guaranteed, that is, the vehicle factory must detect and repair possible problems in the transmission process of the on-board video stream in advance. However, when the on-board video playback has a distorted screen, freezes or jitters, the process may not have any significant abnormal records, making the problem difficult to restore and analyze. Therefore, this application proposes a video stream abnormality analysis method to monitor the transmission and playback process of the on-board video stream before the vehicle leaves the factory, and quickly locate the problem, so as to improve the efficiency of discovering and solving problems, thereby ensuring the efficiency and quality of vehicle delivery.

[0048] It should be noted that the real-time transmission and processing of the in-vehicle video stream described in this case is carried out on the basis of a normal network, that is, the process does not need to consider abnormal video playback caused by network problems. The process is that the in-vehicle camera transmits the captured video in real time to the in-vehicle application layer for processing and plays it on the in-vehicle display, so as to realize the real-time transmission, processing and playback of the in-vehicle video stream.

[0049] This method can be applied to the abnormal analysis of the vehicle-mounted video stream, and can also be applied to the abnormal analysis of the video stream of the monitoring device, that is, this method is applicable to any scenario of transmitting the video stream to the relevant device in real time for playback.

[0050] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0051] The present application provides a method for analyzing abnormalities in a video stream, which is executed by a vehicle controller or a video stream transmission controller, hereinafter referred to as the controller, and refers to Figure 1 , the method comprising:

[0052] Step S100, processing the received video stream source data to play the video content corresponding to the video stream source data;

[0053] Specifically, the video stream needs to be processed during transmission in order to play the video content on the display. The controller controls the vehicle application layer to perform frame processing, decoding processing and rendering processing on the received video stream source data in sequence, and then the display can play the video content.

[0054] Step S200, in response to determining that the video content playback is abnormal, determining the target video stream source data corresponding to the abnormal video content;

[0055] Specifically, the controller monitors the video content played on the display. When the video content played on the display is abnormal (such as screen distortion, freeze or jitter, etc.), the controller locates the target video stream source data corresponding to the abnormal video content.

[0056] Exemplarily, the controller monitors the video content played by the display through a video monitoring system. When the display screen is abnormal, the video monitoring system sends an abnormal signal to the controller, allowing the controller to determine that the played video content is abnormal, so that the controller can determine the target video stream source data corresponding to the abnormal video content.

[0057] In addition, you can use methods such as using a camera to shoot, using test software and using professional instruments to detect abnormal conditions such as screen distortion, freezes, and jitters on the monitor, including: 1. Using a camera to shoot: By shooting the monitor screen and then playing back the video to observe whether there is flickering or irregular movement, this method can quickly detect whether the monitor has obvious screen flicker or movement problems; 2. Using test software such as SMTT 2.0 and PixPerAn, you can use different test modes to detect the monitor's response time and screen flicker. These software can provide more professional test results and help identify specific problems with the monitor; 3. Using professional instruments such as spectrometers and oscilloscopes, you can detect the monitor's brightness, color and screen flicker, so as to obtain more accurate test results; The above methods can effectively detect abnormal conditions such as screen distortion, freezes, and jitters on the monitor.

[0058] The video monitoring system can be composed of a device installed with display anomaly detection software to monitor the video content played on the display.

[0059] Step S300, based on the target video stream source data, determining the cause of the abnormal video playback.

[0060] Specifically, based on the target video stream source data, the controller analyzes the target video stream source data and the data in the corresponding processing process respectively to determine the cause of the abnormal video content, that is, the cause of the abnormality in video playback.

[0061] Exemplarily, the controller analyzes the target video stream source data (the target video stream source data may be sent to a detection audio and video playback software for playback) to determine whether there is a problem with the target video stream source data. If there is a problem with the target video stream source data, it is determined that the cause of the abnormal video playback is a problem with the hardware end for obtaining the video stream source data. The hardware end may be a camera. The hardware end may then be troubleshooted to determine the fault and repair it. If there is no problem with the target video stream source data, the data in the processing process corresponding to the target video stream source data is analyzed to determine whether there is a problem with the data in the processing process. This allows problems to be confirmed step by step, which is beneficial to improving the efficiency of problem discovery.

[0062] In this embodiment, the video stream source data is processed to play the video content corresponding to the video stream source data, and when the video content is played, the played video content is monitored for abnormalities. When an abnormality occurs in the video content during playback, the target video stream source data corresponding to the abnormal video content is determined, and on this basis, the cause of the abnormality in the video content playback is determined to achieve abnormal monitoring during the video playback. By determining the target video stream source data corresponding to the abnormal video content, the technical effect of tracing back when an abnormality occurs in the video playback is achieved, which is conducive to quickly locating the cause of the abnormality and improving the practicality of the method.

[0063] In some embodiments, the video stream source data is marked with a timestamp;

[0064] Step S200: in response to determining that the video playback is abnormal, determining the target video stream source data corresponding to the abnormal video content, including:

[0065] Step S201, in response to determining that the video playback is abnormal, determining a first time point at which the video playback is abnormal and a second time point at which the video playback stops being abnormal;

[0066] Specifically, when the controller determines that the video playback is abnormal, it determines a first time point when the abnormality occurs, and when the controller determines that the video playback stops being abnormal, it determines a second time point when the abnormality stops.

[0067] Exemplarily, when the video surveillance system monitors that the video playback screen of the display has an abnormality, it sends an abnormality signal to the controller, and the controller determines the time point as the first time point when receiving the abnormality signal; when the video surveillance system monitors that the video playback screen of the display stops abnormally, that is, when the video playback screen of the display returns to normal, it sends a stop abnormality signal to the controller, and the controller determines the time point as the second time point when receiving the stop abnormality signal.

[0068] Step S202, determining a first time period during which video playback is abnormal based on the first time point and the second time point;

[0069] Specifically, based on the known first time point and the second time point, the controller can determine the first time period in which the video playback occurs. The starting time point of the first time period is a preset time length set in advance based on the first time point, and the end time point is the second time point. The starting time point of the first time period is advanced considering that the process of video stream transmission requires time, and the video surveillance system also requires time to send a signal to the controller. The preset time length is used to make up for the time difference to avoid the mismatch between the target video stream source data determined by the controller based on the first time period and the abnormal content on the display screen, which is beneficial to improving the accuracy of abnormal analysis of the method.

[0070] Exemplarily, the first time point is 14:00:00, the second time point is 14:00:10, the preset time length is 10 seconds, and the first time period is 13:59:50-14:00:10.

[0071] Step S203: determining target video stream source data having a timestamp including the first time period from the stored video stream source data.

[0072] Specifically, the stored video stream source data has a timestamp mark, and the video stream source data is stored according to time periods, so the timestamp of the video stream source data is the time period; based on the first time period, the controller searches for video stream source data whose timestamp is within the first time period in the stored video stream source data, and determines it as the target video stream source data.

[0073] Exemplarily, the first time period is 14:00:00-14:01:00, and the video stream source data is stored in segments, each segment storing 10 seconds of video stream source data, wherein the timestamps of the video stream source data included therein are 13:59:51-14:00:00, 14:00:01-14:00:10, 14:00:11-14:00:20, 14:00:21-14:00:30, 14:00:31-14:00:40, 14:00:41-14:00:50, and 14:00:51-14:01:00. Since 14:00:00 is the end time point of a timestamp, all the video stream source data corresponding to the above timestamps are determined as the target video stream source data.

[0074] In this embodiment, the process of determining the target video stream source data is described in detail. When storing the video stream source data, the video stream source data is stored with a timestamp so as to determine the target video stream source data, and the first time period of the picture abnormality is extended based on the time when the picture abnormality is monitored, thereby avoiding missing abnormal video stream source data, which is beneficial to improving the analysis efficiency and analysis accuracy of the method, and further conducive to the promotion and application of the method.

[0075] In some embodiments, step S300: determining the cause of abnormal video playback based on the target video stream source data includes:

[0076] Step S301, analyzing the target video stream source data to determine whether the target video stream source data is abnormal;

[0077] Specifically, the controller analyzes the target video stream source data to determine whether there is any abnormality in the target video stream source data. The analysis process includes frame sequence abnormality analysis and image content abnormality analysis. As long as there is an analysis abnormality, it can be determined that there is an abnormality in the target video stream source data. If there is no abnormality in both analyses, it is determined that there is no abnormality in the target video stream source data.

[0078] Among them, the process of analyzing the target video stream source data can be a manual analysis or an image data analysis system. During the analysis process, the image content is checked. If the image content has abnormal content such as screen distortion, jitter, etc., it is determined that the target video stream source data corresponding to the image content is abnormal.

[0079] Exemplarily, the target video stream source data is sent to an audio and video playback software for detection, and the target video stream source data is played. If the displayed image content has abnormal conditions such as screen distortion, freezes or jitters during playback, it is determined that there is a problem with the target video stream source data. If the displayed image content does not have abnormal conditions such as screen distortion, freezes or jitters during playback, it is determined that there is no problem with the target video stream source data.

[0080] Step S302, in response to determining that the target video stream source data is abnormal, determining that the cause of the abnormal video playback is a problem with the sending end of the video stream source data;

[0081] Specifically, when it is determined that there is an abnormality in the target video stream source data, the controller determines that the cause of the abnormal video playback is the sending end of the video stream source data, that is, there is a problem with the hardware end for obtaining the video stream source data, resulting in the abnormality in the target video stream source data. At this time, the problem is quickly located without the need to analyze the data in the processing process.

[0082] Step S303, in response to determining that there is no abnormality in the target video stream source data, determining whether there is a problem in the processing of the target video stream source data based on the target video stream source data.

[0083] Specifically, when it is determined that there is no abnormality in the target video stream source data, that is, the target video stream source data is normal, the controller needs to further analyze the data in the processing process corresponding to the target video stream source data to determine whether there is a problem in the processing process.

[0084] In this embodiment, the target video stream source data is first analyzed to determine whether there is a problem with the data source. When there is a problem with the video stream source data, the cause of the problem can be determined, thereby avoiding some unnecessary abnormal analysis (i.e., avoiding data analysis after the target video stream source data is processed), which is beneficial to improving the abnormal analysis efficiency of the method. When there is no problem with the video stream source data, the data in the video stream source data processing process is analyzed to further determine the problem, thereby achieving step-by-step positioning of the problem, which can not only improve the efficiency of locating the problem, but also help to further correct related problems, and is beneficial to the promotion and application of the method.

[0085] In some embodiments, reference Figure 2 , the processing process of the video stream source data includes:

[0086] Step S101, performing frame processing on the video stream source data to obtain frame data, and performing frame sequence integrity detection on the frame data;

[0087] Specifically, the sending end of the video stream source data is the hardware layer, the hardware layer sends the video stream source data to the application layer, the controller stores the video stream source data (it can be stored in the cloud or locally after ZIP compression), and performs frame processing on the video stream source data through a frame algorithm to obtain frame data. The controller marks the frame data with a timestamp and then stores it (it can be stored in the cloud or locally after ZIP compression), and performs frame sequence integrity detection on the frame data during the storage process to achieve an initial inspection of the frame data.

[0088] It should be noted that the sending ends of the video stream source data are different, and the frame sequences of the frame data obtained after frame processing of the video stream source data are different. Therefore, when the controller performs frame sequence integrity detection, the standard frame sequence based on it is a pre-set standard frame sequence corresponding to the sending end.

[0089] Exemplarily, the standard frame sequence corresponding to the transmitter of the video stream source data is: I frame + PPS frame + SPS frame + 29 P frames, then the standard frame sequence based on which the controller performs frame sequence integrity detection is: I frame + PPS frame + SPS frame + 29 P frames.

[0090] Among them, I frame (key frame): an independently decoded frame, containing complete image data, and is a reference frame of the video stream; PPS frame (Picture Parameter Set): an information frame containing image parameters in the video stream; SPS frame (Sequence Parameter Set): an information frame containing video sequence parameters; P frame (prediction frame): encoded based on the previous I frame or P frame, and only contains data of the changed part; the number of P frames is also related to the sending end of the video stream source data, so the number of P frames, and the order of I frames, PPS frames and SPS frames correspond to the sending end of the video stream source data, that is, the standard frame sequence based on which the controller performs frame sequence integrity detection corresponds to the sending end of the video stream source data.

[0091] Step S102, in response to determining that there is abnormal frame sequence data in the frame data, marking the abnormal frame sequence data;

[0092] Specifically, when the controller performs frame sequence integrity detection on the frame data, if the frame type in the frame data is different from the corresponding frame type in the standard frame sequence, it is determined that there is frame sequence abnormal data in the frame data, and the controller marks the determined frame sequence abnormal data so that the frame sequence abnormal data can be quickly determined subsequently, which is beneficial to improving the efficiency of the method.

[0093] Exemplarily, the standard frame sequence is: I frame + PPS frame + SPS frame + 29 P frames, and there is a frame sequence in the frame data: I frame + PPS frame + SPS frame + 28 P frames. The controller determines that the frame data corresponding to the frame sequence is abnormal frame sequence data and marks it.

[0094] Step S103, decoding the frame data to obtain decoded data, and detecting the frame sequence integrity of the decoded data;

[0095] Specifically, the framed data is transmitted to a decoder at the application layer to decode the framed data to obtain the decoded data. The controller marks the decoded data with a timestamp and stores it (it may be ZIP compressed and then stored in the cloud or locally), and detects the frame sequence integrity of the decoded data during the storage process.

[0096] It should be noted that the controller performs frame sequence integrity detection on both the frame data and the decoded data, and the standard frame sequence is the same, which will not be elaborated here. In addition, on the basis of the frame sequence integrity detection on the frame data, the frame sequence integrity detection is further performed on the decoded data to avoid the situation where anomalies occur in the frame data during the decoding process and the cause of the fault cannot be located, which is beneficial to improving the accuracy and practicality of the method.

[0097] Step S104: in response to determining that there is abnormal frame sequence data in the decoded data, marking the abnormal frame sequence data.

[0098] Specifically, when the controller performs a frame sequence integrity check on the decoded data, if the frame type in the decoded data is different from the corresponding frame type in the standard frame sequence, it is determined that there is abnormal frame sequence data in the decoded data. The controller marks the determined abnormal frame sequence data so that the abnormal frame sequence data can be quickly determined subsequently, which is beneficial to improving the efficiency of the method.

[0099] In this embodiment, the video stream source data is framed and decoded to obtain the framed data and the decoded data, and the frame sequence integrity of the framed data and the decoded data is detected during the process of storing the framed data and the decoded data to preliminarily identify and mark abnormal data, thereby improving the efficiency of abnormal analysis of the video stream, which is conducive to the promotion and application of the method.

[0100] In some embodiments, in step S300, determining the cause of abnormal video playback based on the target video stream source data includes:

[0101] Step S304, based on the target video stream source data, determining target frame data and target decoded data corresponding to the target video stream source data from the stored frame data and decoded data;

[0102] Specifically, the frame data and the decoded data are both marked with a timestamp when stored, and the timestamp of the target video stream source data is determined based on the target video stream source data, and the target frame data and target decoded data with the timestamp of the target video stream source data are confirmed in the frame data and the decoded data.

[0103] Exemplarily, the timestamps of the target video stream source data are 14:00:01-14:00:02, the timestamps of the frame data are 14:00:00, 14:00:01, 14:00:02, and 14:00:03, and the timestamps of the decoded data are 14:00:00, 14:00:01, 14:00:02, and 14:00:03, then the frame data with timestamps of 14:00:01 and 14:00:02 are determined as target frame data, and the decoded data with timestamps of 14:00:01 and 14:00:02 are determined as target decoded data.

[0104] Step S305, determining whether the target frame data and the target decoded data have a frame sequence abnormality mark;

[0105] Specifically, the controller performs a mark check on the target frame data and the target decoded data to determine whether there is a frame sequence abnormality mark therein.

[0106] It should be noted that the frame data and the decoded data have been subjected to frame sequence anomaly detection during the storage process, and the data with frame sequence anomalies have been marked with frame sequence anomalies. The controller only needs to detect the marks of the target frame data and the target decoded data to determine whether there are frame sequence anomaly marks. The judgment process is fast, which is conducive to improving the analysis efficiency of the method.

[0107] Step S306, in response to determining that both the target segmented frame data and the target decoded data have frame sequence abnormality marks, determining that the cause of the abnormal video playback is a problem in the segmented frame processing process;

[0108] Specifically, when the controller detects that both the target framing data and the target decoding data include data with frame sequence exception marks, it is determined that both the target framing data and the target decoding data store frame sequence exception data. On the premise that it has been confirmed that there is no exception in the target video stream source data, it can be determined that there is a problem in the process of framing the target video stream source data by the application layer, so that both the target framing data and the target decoding data have frame sequence exceptions, thereby achieving the technical effect of accurately and quickly locating the problem.

[0109] Step S307, in response to determining that the target segmented frame data does not have a frame sequence abnormality mark, and the target decoded data has a frame sequence abnormality mark, determining that the cause of the abnormal video playback is a problem in the decoding process;

[0110] Specifically, when the controller detects that the target frame data has no frame sequence exception mark, and the target decoded data has a frame sequence exception mark, it is determined that there is no frame sequence exception mark data in the target frame data, and there is a frame sequence exception mark in the target decoded data. On the premise that it has been confirmed that there is no abnormality in the target video stream source data and that there is no frame sequence exception data in the target frame data, there is frame sequence exception data in the target decoded data, it can be determined that there is a problem in the process of decoding the target frame data by the application layer, resulting in a frame sequence exception in the target decoded data.

[0111] Step S308 , in response to determining that neither the target frame data nor the target decoded data has a frame sequence abnormality mark, it is determined whether there is a problem with the image content of the target frame data and the target decoded data.

[0112] Specifically, when the controller detects that there are no frame sequence abnormality marks in the target frame data and the target decoded data, and on the premise that there are no abnormalities in the target video stream source data, it can further judge to determine whether there are problems with the image content of the target frame data and the target decoded data, which is conducive to further discovering and locating the problem.

[0113] In this embodiment, on the premise that it has been determined that there is no abnormality in the target video stream source data, the frame sequence abnormality marks of the target framing data and the target decoding data corresponding to the target video stream source data are detected to determine whether there are frame sequence abnormality data in the target framing data and the target decoding data, so as to further determine whether there is an abnormality in the processing process, and to realize the judgment of whether the processing process causes frame sequence abnormality. After determining that there is no problem causing frame sequence abnormality in the processing process, the image content of the target framing data and the target decoding data is detected to further determine whether there is an abnormality in the processing process causing image content abnormality, so as to achieve the technical effect of hierarchical positioning problems, which is conducive to improving the analysis efficiency and accuracy of the method.

[0114] In some embodiments, in step S303E: determining whether there is a problem with the image content of the target frame data and the target decoded data includes:

[0115] Step S303E-1, analyzing the image contents of the target frame data and the target decoded data respectively, to determine whether the target frame data and the target decoded data contain abnormal image content data respectively;

[0116] Specifically, on the premise that it has been determined that there is no abnormality in the target video stream source data, and that there is no frame sequence abnormality in the target framing data and the target decoding data, the controller analyzes the image content of the target framing data and the target decoding data respectively to determine whether there is abnormal image content data in the target framing data and the target decoding data respectively. The analysis process can be performed manually or by an image data analysis tool. During the analysis process, the image content is checked. If the image content has abnormal content such as screen distortion and jitter, it is determined that the data corresponding to the image content has abnormal image content data.

[0117] The image content checking and analysis of the target frame data and the target decoded data may be a decryption check to determine whether there is an abnormality in the image content.

[0118] Exemplarily, the target frame data and the target decoded data are respectively sent to an audio and video player for detection to be played. During the playback, it is checked whether the image content displayed on the monitor has screen distortion, freezes or jitters. If so, it is determined that there is a problem with the data corresponding to the image content.

[0119] Step S303E-2, in response to determining that both the target frame data and the target decoded data have abnormal image content data, determining that there is a problem in the frame processing process;

[0120] Specifically, if it is determined that both the target frame data and the target decoded data have abnormal image content data, it is determined that there is a problem in the process of framing the target video stream source data, resulting in abnormal image content in both the target frame data and the target decoded data.

[0121] Step S303E-3, in response to determining that the target frame data does not contain abnormal image content data, and the target decoded data contains abnormal image content data, it is determined that there is a problem in the decoding process.

[0122] Specifically, if it is determined that the target frame data does not contain abnormal image content data, and the target decoded data contains abnormal image content data, it can be determined that there is a problem in the process of decoding the target frame data, which leads to the presence of abnormal image content data in the target decoded data.

[0123] In this embodiment, on the premise that it has been determined that there is no abnormality in the target video stream source data, and that there is no frame sequence abnormality in the target framing data and the target decoded data, the image contents of the target framing data and the target decoded data are checked and analyzed to respectively determine whether there is an abnormality in the image contents of the target framing data and the target decoded data, so as to further determine the process causing the abnormality, thereby achieving the technical effect of hierarchical positioning of the problem, which is conducive to improving the analysis efficiency and accuracy of the method.

[0124] In some embodiments, after determining that there is a problem in the framing process, the method includes:

[0125] Step S400: analyzing the framing algorithm corresponding to the framing process to locate loopholes in the framing algorithm.

[0126] Specifically, after determining that there is a problem in the frame processing process, the frame algorithm is analyzed and detected to determine the loopholes in the frame algorithm, and the frame algorithm is repaired based on the determined loopholes, thereby achieving the technical effect of quickly locating the problem and quickly repairing the problem, which is conducive to the practicality and promotion and application of the method.

[0127] Specifically, the framing algorithms mainly include inter-frame difference method, background difference method, optical flow method, etc.; in actual use, multiple framing algorithms will be used in combination to achieve a good framing effect.

[0128] There are several ways to detect and analyze problems with the framing algorithm:

[0129] 1. Code review: Review the algorithm code to check for logical errors, improper data processing, or implementation defects. This can be done through code walkthroughs, unit testing, etc.

[0130] 2. Comparative analysis: Compare the framing algorithm with other known good framing algorithms, analyze the differences between the two, and identify the loopholes of the analysis algorithm.

[0131] Through the above method, problems existing in the frame segmentation algorithm can be detected, so that corresponding improvements and optimizations can be made according to the problems obtained.

[0132] After the problem with the framing algorithm is determined, the framing algorithm is manually repaired according to the problem.

[0133] In some embodiments, after determining that there is a problem in the decoding process, the method further includes:

[0134] Step S500: inspect and repair the decoder corresponding to the decoding process to locate the fault of the decoder.

[0135] Specifically, after determining that there is a problem in the decoding process, the decoder is analyzed and detected to determine the fault in the decoder, and the decoder is repaired based on the determination of the fault, thereby achieving the technical effect of quickly locating the problem and quickly repairing the problem, which is conducive to the practicality and popularization and application of the method.

[0136] Specifically, the analysis and detection of the decoder mainly involves the following aspects:

[0137] 1. Setting check: Check the decoder settings to determine whether the settings are correct.

[0138] 2. Hardware check: Check whether the decoder port is damaged.

[0139] 3. Power supply check: Check whether the power supply of the decoder is normal.

[0140] 4. Voltage changes: Observe whether there are voltage changes between the ports of the decoder to determine whether the decoder is working properly.

[0141] 5. Fuse inspection: Check whether the fuse of the decoder is burned out.

[0142] Through the above steps, the fault of the decoder can be systematically detected, so that corresponding maintenance measures can be taken. If no abnormality is detected in the above steps, the decoder is updated.

[0143] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0144] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0145] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a device for analyzing abnormalities in a video stream.

[0146] refer to Figure 3 , the video stream abnormality analysis device comprises:

[0147] The processing module 100 is configured to process the received video stream source data to play the video content corresponding to the video stream source data;

[0148] The first determination module 200 is configured to determine the target video stream source data corresponding to the abnormal video content in response to determining that the video content playback is abnormal;

[0149] The second determination module 300 is configured to determine the cause of the abnormality in video playback based on the target video stream source data.

[0150] In some embodiments, the video stream source data is marked with a timestamp;

[0151] The first determining module 200 is further configured to:

[0152] In response to determining that the video playback is abnormal, determining a first time point at which the video playback is abnormal and a second time point at which the video playback stops being abnormal;

[0153] Determine, based on the first time point and the second time point, a first time period in which the video playback is abnormal;

[0154] Target video stream source data having a timestamp including the first time period is determined from the stored video stream source data.

[0155] In some embodiments, the second determining module 300 is further configured to:

[0156] Analyze the target video stream source data to determine whether the target video stream source data is abnormal;

[0157] In response to determining that the target video stream source data is abnormal, determining that the cause of the abnormal video playback is a problem with a sending end of the video stream source data;

[0158] In response to determining that there is no abnormality in the target video stream source data, it is determined based on the target video stream source data whether there is a problem in the processing of the target video stream source data.

[0159] In some embodiments, the processing module 100 is further configured to:

[0160] Performing frame processing on the video stream source data to obtain frame data, and performing frame sequence integrity detection on the frame data;

[0161] In response to determining that there is abnormal frame sequence data in the frame data, marking the abnormal frame sequence data;

[0162] Decoding the framed data to obtain decoded data, and detecting the frame sequence integrity of the decoded data;

[0163] In response to determining that frame sequence abnormal data exists in the decoded data, the frame sequence abnormal data is marked.

[0164] In some embodiments, the second determining module 300 is further configured to:

[0165] Based on the target video stream source data, determining target frame data and target decoded data corresponding to the target video stream source data from the stored frame data and decoded data;

[0166] Determine whether the target frame data and the target decoded data have a frame sequence abnormality mark;

[0167] In response to determining that both the target frame data and the target decoded data have frame sequence abnormality marks, it is determined that there is a problem in the frame processing process;

[0168] In response to determining that the target frame data does not have a frame sequence abnormality mark, and the target decoded data has a frame sequence abnormality mark, determining that there is a problem in the decoding process;

[0169] In response to determining that neither the target frame data nor the target decoded data has a frame sequence abnormality mark, it is determined whether there is a problem with the image content of the target frame data and the target decoded data.

[0170] In some embodiments, the second determining module 300 is further configured to:

[0171] Analyzing the image contents of the target frame data and the target decoded data respectively to determine whether the target frame data and the target decoded data contain abnormal image content data;

[0172] In response to determining that both the target frame data and the target decoded data have abnormal image content data, determining that there is a problem in the frame processing process;

[0173] In response to determining that the target frame data does not contain abnormal image content data, and the target decoded data contains abnormal image content data, it is determined that there is a problem in the decoding process.

[0174] In some embodiments, the apparatus further includes a first positioning module, wherein the first positioning module is configured to:

[0175] Analyze the framing algorithm corresponding to the framing processing to locate the loopholes in the framing algorithm.

[0176] In some embodiments, the apparatus further includes a second positioning module, wherein the second positioning module is configured to:

[0177] Repair the decoder corresponding to the decoding process to locate the decoder fault.

[0178] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0179] The device of the above embodiment is used to implement the corresponding video stream abnormality analysis method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0180] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for analyzing the abnormality of the video stream described in any of the above embodiments is implemented.

[0181] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0182] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0183] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0184] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0185] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0186] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0187] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0188] The electronic device of the above embodiment is used to implement the corresponding video stream abnormality analysis method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0189] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the video stream anomaly analysis method as described in any of the above embodiments.

[0190] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0191] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the video stream anomaly analysis method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0192] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0193] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0194] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0195] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0196] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0197] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0198] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0199] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0200] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A method for analyzing abnormalities in a video stream, characterized in that: include: Processing the received video stream source data to play the video content corresponding to the video stream source data; In response to determining that the video content playback is abnormal, determining target video stream source data corresponding to the abnormal video content; Based on the target video stream source data, the cause of the abnormal video playback is determined.

2. The method according to claim 1, characterized in that The video stream source data is marked with a timestamp; In response to determining that the video playback is abnormal, determining the target video stream source data corresponding to the abnormal video content includes: In response to determining that the video playback is abnormal, determining a first time point at which the video playback is abnormal and a second time point at which the video playback stops being abnormal; Determine, based on the first time point and the second time point, a first time period in which the video playback is abnormal; Target video stream source data having a timestamp including the first time period is determined from the stored video stream source data.

3. The method according to claim 1, characterized in that The determining, based on the target video stream source data, a reason why the video playback is abnormal includes: Analyze the target video stream source data to determine whether the target video stream source data is abnormal; In response to determining that the target video stream source data is abnormal, determining that the cause of the abnormal video playback is a problem with a sending end of the video stream source data; In response to determining that there is no abnormality in the target video stream source data, it is determined based on the target video stream source data whether there is a problem in the processing of the target video stream source data.

4. The method according to claim 1, characterized in that: The processing process of the video stream source data includes: Performing frame processing on the video stream source data to obtain frame data, and performing frame sequence integrity detection on the frame data; In response to determining that there is abnormal frame sequence data in the frame data, marking the abnormal frame sequence data; Decoding the framed data to obtain decoded data, and detecting the frame sequence integrity of the decoded data; In response to determining that frame sequence abnormal data exists in the decoded data, the frame sequence abnormal data is marked.

5. The method according to claim 4, characterized in that The determining, based on the target video stream source data, a reason why the video playback is abnormal includes: Based on the target video stream source data, determining target frame data and target decoded data corresponding to the target video stream source data from the stored frame data and decoded data; Determine whether the target frame data and the target decoded data have a frame sequence abnormality mark; In response to determining that both the target frame data and the target decoded data have frame sequence abnormality marks, determining that the cause of the abnormal video playback is a problem in the frame processing process; In response to determining that the target frame data does not have a frame sequence abnormality mark, and the target decoded data has a frame sequence abnormality mark, determining that the cause of the abnormal video playback is a problem in the decoding process; In response to determining that neither the target frame data nor the target decoded data has a frame sequence abnormality mark, it is determined whether there is a problem with the image content of the target frame data and the target decoded data.

6. The method according to claim 5, characterized in that The determining whether there is a problem with the image content of the target frame data and the target decoded data includes: Analyzing the image contents of the target frame data and the target decoded data respectively to determine whether the target frame data and the target decoded data contain abnormal image content data; In response to determining that both the target frame data and the target decoded data have abnormal image content data, determining that there is a problem in the frame processing process; In response to determining that the target frame data does not contain abnormal image content data, and the target decoded data contains abnormal image content data, it is determined that there is a problem in the decoding process.

7. The method according to claim 5 or 6, characterized in that: After determining that there is a problem in the framing process, the method further comprises: Analyze the framing algorithm corresponding to the framing processing to locate the loopholes in the framing algorithm.

8. The method according to claim 5 or 6, characterized in that: After determining that there is a problem in the decoding process, the method further comprises: Repair the decoder corresponding to the decoding process to locate the decoder fault.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A vehicle, characterized in that: Comprising an electronic device as claimed in claim 9.

Images