Video diagnosis method and electronic equipment

Abstract

The invention provides a video diagnosis method and electronic equipment, relates to the technical field of multimedia, and is used for reducing resource consumption of video quality diagnosis. The method comprises the following steps: receiving a plurality of video data packets; determining at least one target video data packet corresponding to the target frame according to the mark values and the timestamps of the plurality of video data packets; the mark value is used for representing whether the video data packet is the last data packet corresponding to the target frame; and determining a video diagnosis result of the target frame according to the serial number of the at least one target video data packet.

Description

Technical Field

[0001] The present disclosure relates to the field of multimedia technology, and in particular to a video diagnosis method and electronic equipment. Background Art

[0002] With rising public safety awareness, cameras are becoming increasingly common in everyday life and production. By transmitting video data captured by cameras to receiving devices, various security incidents can be prevented. However, this process can lead to degradation of video quality due to network issues, hardware failures, camera movement, or obstructions.

[0003] Therefore, after receiving video data, the receiving device must first decode the video data and then perform a quality assessment on the decoded video data. If the decoded video has quality issues, it may not be possible to effectively prevent them. However, the decoding process consumes a large amount of computing resources. Therefore, after acquiring video data, how to determine video quality with minimal resources is an urgent problem that needs to be solved. Summary of the Invention

[0004] In order to solve the above technical problems, the present disclosure provides a video diagnosis method and an electronic device, which can reduce resource consumption of video quality diagnosis.

[0005] The technical solutions disclosed in this disclosure are as follows:

[0006] In a first aspect, the present disclosure provides a video diagnosis method, which is applied to an electronic device, and the method includes: receiving multiple video data packets; determining at least one target video data packet corresponding to a target frame based on the tag values and timestamps of the multiple video data packets; the tag value is used to indicate whether the video data packet is the last data packet corresponding to the target frame; and determining a video diagnosis result of the target frame based on the serial number of at least one target video data packet.

[0007] In combination with the first aspect, in another possible implementation, the video diagnosis result of the target frame is determined based on the serial number of at least one target video data packet, including: based on the serial number of at least one target video data packet, when the serial number of at least one target video data packet does not conform to a preset arrangement, determining that the video diagnosis result of the target frame is abnormal.

[0008] In combination with the first aspect, in another possible implementation, when the serial number of at least one target video data packet does not conform to a preset arrangement, the video diagnosis result of the target frame is determined to be abnormal, including: when the serial number of at least one target video data packet is discontinuous, the video diagnosis result of the target frame is determined to be packet loss; when the serial number of at least one target video data packet is repeated, the video diagnosis result of the target frame is determined to be packet duplication.

[0009] In combination with the first aspect, in another possible implementation, at least one target video data packet corresponding to a target frame is determined based on the tag values and timestamps of multiple video data packets, including: determining at least one video data packet corresponding to the target frame based on the timestamp of each video data packet in the multiple video data packets; the timestamp of at least one video data packet corresponding to the target frame is the same; when there is a video data packet with a tag value having a preset value in at least one video data packet, determining the at least one video data packet as the at least one target video data packet corresponding to the target frame.

[0010] In combination with the first aspect, in another possible implementation method, at least one target video data packet corresponding to the target frame is determined based on the tag values and timestamps of multiple video data packets. Before that, the method also includes: storing multiple video data packets according to a preset structure, the preset structure includes a serial number, a timestamp and a tag value.

[0011] In combination with the first aspect, in another possible implementation, the preset structure further includes a valid length of the video data packet and video data of the video data packet.

[0012] In combination with the first aspect, in another possible implementation, the method further includes: within a preset time window, if no video data packet is received within a preset time length, determining that the video diagnosis result is a video transmission interruption at the video transmitting end.

[0013] In a second aspect, embodiments of the present disclosure provide a video diagnostic device that can be applied to an electronic device to implement the method described in the first aspect. The functions of the video diagnostic device can be implemented using hardware or by executing corresponding software on the hardware. The hardware or software includes one or more modules corresponding to the aforementioned functions, such as a receiving module and a processing module.

[0014] The receiving module is configured to receive a plurality of video data packets.

[0015] The processing module is configured to determine at least one target video data packet corresponding to a target frame based on the tag values and timestamps of multiple video data packets; the tag value is used to indicate whether the video data packet is the last data packet corresponding to the target frame; and determine the video diagnosis result of the target frame based on the serial number of at least one target video data packet.

[0016] In combination with the second aspect, in another possible implementation, the processing module is further configured to determine that the video diagnosis result of the target frame is abnormal based on the serial number of at least one target video data packet, if the serial number of at least one target video data packet does not conform to a preset arrangement.

[0017] In combination with the second aspect, in another possible implementation, the processing module is further configured to determine that the video diagnosis result of the target frame is packet loss when there is discontinuity in the serial number of at least one target video data packet; and to determine that the video diagnosis result of the target frame is packet duplication when there is duplication in the serial number of at least one target video data packet.

[0018] In combination with the second aspect, in another possible implementation, the processing module is further configured to determine at least one video data packet corresponding to the target frame based on the timestamp of each video data packet in the multiple video data packets; the timestamp of at least one video data packet corresponding to the target frame is the same; and when there is a video data packet with a mark value of a preset value in at least one video data packet, the at least one video data packet is determined as the at least one target video data packet corresponding to the target frame.

[0019] In combination with the second aspect, in another possible implementation, the processing module is further configured to store multiple video data packets according to a preset structure, where the preset structure includes a sequence number, a timestamp, and a tag value.

[0020] In combination with the second aspect, in another possible implementation, the preset structure further includes a valid length of the video data packet and video data of the video data packet.

[0021] In combination with the second aspect, in another possible implementation, the processing module is further configured to determine that the video diagnosis result is a video transmission interruption if no video data packet is received within a preset time window.

[0022] In a third aspect, an electronic device is provided, comprising: a processor and a memory; the memory is used to store computer instructions, and when the electronic device is running, the processor executes the computer instructions stored in the memory to enable the electronic device to perform the video diagnosis method as described in the first aspect above.

[0023] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the computer-readable storage medium is run on an electronic device, the electronic device can execute the video diagnosis method as described in the first aspect.

[0024] In a fifth aspect, a computer program product comprising computer instructions is provided. When the computer instructions are executed on an electronic device, the electronic device can execute the video diagnosis method as described in the first aspect.

[0025] In a sixth aspect, a device (for example, a chip system) is provided, which includes a processor for supporting an electronic device to implement the functions involved in the first aspect above. In one possible design, the device also includes a memory for storing program instructions and data necessary for the electronic device. When the device is a chip system, it can be composed of a chip or include a chip and other discrete devices.

[0026] In this disclosure, the above names do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this disclosure, they are within the scope of the claims of this disclosure and their equivalents.

[0027] These and other aspects of the present disclosure will become more apparent from the following description.

[0028] The technical solution provided by this disclosure offers the following advantages over existing technologies: After receiving multiple video data packets, the at least one target video data packet corresponding to a target frame can be accurately determined by examining the tag value and timestamp of each video data packet. Subsequently, based on the sequence number of the at least one target video data packet, the quality of the target frame can be preliminarily assessed, allowing potential issues to be promptly identified and addressed. This not only enables a preliminary assessment of the quality of the target frame, but also significantly improves the processing efficiency of video quality assessment.

[0029] Compared to related technologies that require decoding multiple video packets before evaluating the video quality of the target frame, the present invention uses the header information of the video packet (such as the tag value, timestamp, and sequence number) to make a preliminary judgment. This can quickly identify possible quality issues without decoding the entire video packet. This not only speeds up video quality detection, but also significantly reduces the consumption of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0031] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 A schematic diagram of displaying video data with a distorted screen provided by an embodiment of the present disclosure;

[0033] Figure 2 A schematic diagram of the structure of a video diagnosis system provided in an embodiment of the present disclosure;

[0034] Figure 3 This is one of the flowcharts of a video diagnosis method provided by an embodiment of the present disclosure;

[0035] Figure 4 A schematic diagram of a process for transmitting audio / video data based on the RTSP media service protocol framework provided in an embodiment of the present disclosure;

[0036] Figure 5 The second flowchart of a video diagnosis method provided by an embodiment of the present disclosure;

[0037] Figure 6 The third flowchart of a video diagnosis method provided by an embodiment of the present disclosure;

[0038] Figure 7 This is a fourth flow chart of a video diagnosis method provided in an embodiment of the present disclosure;

[0039] Figure 8 This is a fifth flow chart of a video diagnosis method provided in an embodiment of the present disclosure;

[0040] Figure 9 The sixth flowchart of a video diagnosis method provided in an embodiment of the present disclosure;

[0041] Figure 10 A structural block diagram of a video diagnostic device provided in an embodiment of the present disclosure;

[0042] Figure 11 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0045] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0046] With society's growing emphasis on public safety, cameras have become a part of our daily lives. Cameras capture video data and then transmit it to a receiving device. The receiving device then performs in-depth analysis to promptly identify and prevent potential security threats.

[0047] However, the above process may have problems such as network instability, equipment failure, camera position change or occlusion, which may affect the quality of video data. Therefore, after obtaining video data, the video data quality can be diagnosed to ensure that the video data is of qualified quality for further analysis.

[0048] In some examples, image recognition technology can be used to diagnose the quality of video data. For example, image processing or machine learning classification technology can be used to detect video data for quality issues such as snowflakes, camera offline, screen distortion, color cast, etc. Figure 1 However, image recognition technology has a low overall recognition rate and poor scene adaptability; it also requires large-scale parameter adjustments in different scenarios, resulting in low work efficiency.

[0049] In other examples, video data quality can be diagnosed using trained deep learning models. This approach can significantly improve the accuracy of video quality diagnosis. However, it also consumes significant resources, including but not limited to decoding the video data, processing the trained deep learning model, and training the model. Therefore, after acquiring video images, how to diagnose video quality using limited resources has become a pressing issue.

[0050] To address the above issues, embodiments of the present disclosure provide a video diagnostic method. After receiving multiple video data packets, the method accurately determines the at least one target video packet corresponding to a target frame by examining the tag value and timestamp of each video packet. Subsequently, based on the sequence number of the at least one target video packet, the quality of the target frame can be preliminarily assessed, allowing potential issues to be promptly identified and addressed. This method not only enables a preliminary assessment of the quality of the target frame but also significantly improves the processing efficiency of video quality assessment.

[0051] Compared to related technologies that require decoding multiple video packets before assessing the video quality of a target frame, the disclosed embodiment uses header information (such as tag value, timestamp, and sequence number) in the video packet to make a preliminary assessment. This allows for rapid identification of potential quality issues without decoding the entire video packet. This not only speeds up video quality testing but also significantly reduces computing resource consumption.

[0052] The video diagnosis method provided by the embodiment of the present disclosure is described below. Those skilled in the art will appreciate that with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiment of the present disclosure are also applicable to similar technical problems.

[0053] The present disclosure provides a video diagnosis system, which can be applied to Figure 2 In the implementation environment shown, the implementation environment may include a sending end device 01 and a receiving end device 02. The sending end device 01 and the receiving end device 02 may communicate with each other via a wired or wireless communication method.

[0054] The transmitting device 01 is used to obtain video data, encode the video data to obtain multiple video data packets, and send the multiple video data packets to the receiving device 02. In the embodiment of the present disclosure, the transmitting device 01 may include at least one camera. Figure 2 The example in which the sending end device 01 includes multiple cameras is taken as an example, and no specific limitation is made to this.

[0055] Receiving device 02 is configured to receive multiple video data packets from transmitting device 01; determine at least one target video data packet corresponding to a target frame based on the tag values and timestamps of the multiple video data packets; the tag value is used to indicate whether the video data packet is the last data packet; and determine the video diagnosis result of the target frame based on the sequence number of the at least one target video data packet. In the disclosed embodiment, there may be one or more receiving devices 02. Receiving device 02 may be a server, a server cluster consisting of multiple servers, or a cloud computing service center. Figure 2 The example in which the receiving device 02 includes multiple servers is taken as an example, and no specific limitation is made to this.

[0056] For example, the transmitting device 01 and the receiving device 02 may be two independent devices, or may be a single device with integrated functions, which is not limited in this embodiment of the present invention.

[0057] based on Figure 2 The video diagnostic system shown, refer to Figure 3 , a flow chart of a video diagnosis method provided by an embodiment of the present disclosure. Figure 3 As shown, the method may include steps 301 to 303.

[0058] Step 301: Receive multiple video data packets.

[0059] Exemplarily, the multiple video data packets may be Real-time Transport Protocol (RTP) packets based on User Datagram Protocol (UDP).

[0060] In some embodiments, an electronic device (also referred to as a receiving device) may receive multiple video data packets from a sending device. For example, the sending device may be a camera.

[0061] In some scenarios, the sending device (such as a camera) can convert the light signal captured by the sensor into a digital signal to form multiple frames of video data.

[0062] However, raw video data is typically very large. For example, for a video with 720 resolution (1280x720 pixels) and 30 frames per second, if each pixel uses 24-bit color (3 bytes), the data generated per second is approximately 87MB. The specific calculation formula is: 1280 × 720 × 3 × 33 / (1024 × 1024) = 87MB / s. This means approximately 5.22GB of data is generated in one minute, making it difficult to transmit directly over the network.

[0063] Therefore, after obtaining multiple frames of video data, the sending device can use video encoding and decoding technology to efficiently compress the collected multiple frames of video data, and the compressed video data packets are sent out through network protocols such as RTP / RTSP (Real-Time Streaming Protocol).

[0064] In some examples, video data is a series of image frames arranged in time sequence, and these image frames often have strong correlations. Based on this characteristic, intra-frame compression and inter-frame compression can generally be used to efficiently compress multi-frame video data.

[0065] Intra-frame compression considers only the information within the current frame, not adjacent frames, similar to the compression of static images. Typically, intra-frame compression uses a lossy compression algorithm. While the efficiency of single-frame data compression is limited, it ensures that each encoded frame can be independently decoded and displayed. This is generally applicable to keyframes or I-frames (intra-frame encoded independent frames).

[0066] Inter-frame compression uses the high similarity between consecutive frames to reduce redundant information. This compression method, also known as temporal compression, encodes the differences between previous and subsequent frames, significantly reducing the amount of data required to store or transmit. Common inter-frame compression techniques also include P-frames and B-frames. P-frames, also known as predictive coding frames, predict the content of the current frame based on the information of the previous frame. B-frames, also known as bidirectional predictive coding frames, refer not only to the previous frame but also to the subsequent frame for more efficient compression. The above compression method can significantly reduce the size of video data while maintaining good data quality.

[0067] After compressing the multi-frame video data collected by the transmitting device using the above compression method, the compressed multi-frame video data packet can be sent to the receiving device using video data transmission technology. For example, the compressed multi-frame video data packet can be transmitted to the receiving device in the form of a real-time stream.

[0068] Figure 4 The following is a schematic diagram showing an exemplary process of transmitting audio / video data based on the RTSP media service protocol framework.

[0069] like Figure 4 As shown, after the transmitting device obtains the audio / video data, it can encode the audio / video data and then encapsulate the encoded audio / video data into RTP packets (i.e., multiple video data packets) through RTP for transmission. At the same time, control information and quality monitoring data are transmitted through the Real-time Transport Control Protocol (RTCP) to monitor transmission quality. UDP has the characteristics of being connectionless and low-latency. Therefore, RTP packets, control information, quality monitoring data, etc. can be transmitted through the User Datagram Protocol (UDP).

[0070] During the transmission of RTP packets, RTSP control information can be transmitted through the Transmission Control Protocol (TCP). During the transmission process, the data packets transmitted by UDP and TCP are sent to the receiving device through the IP protocol.

[0071] Figure 4The transmission process shown primarily involves the RTP protocol, a protocol for transmitting real-time data over computer networks. It is widely used in audio, video, and other multimedia applications to provide real-time data transmission, synchronization, and load control. The following describes the operating principles and characteristics of RTP.

[0072] RTP has the characteristics of real-time data transmission, data segmentation and identification, synchronization, load type and encoding, delay and jitter buffer, and RTP control protocol, namely RTCP.

[0073] Real-time data transmission refers to the ability of RTP to transmit real-time data, such as audio and video. It provides low latency and high throughput transmission capabilities to meet the needs of real-time communications, such as voice calls, video conferencing, and live streaming services.

[0074] Data segmentation and identification: RTP typically segments audio / video data into multiple small data packets (also called RTP packets). Each RTP packet includes a sequence number and timestamp, which help the receiving device reassemble the data and maintain data timing. RTP also provides an identification mechanism to facilitate the receiving device to correctly parse and process the data.

[0075] Synchronicity means that RTP can use timestamps to ensure data synchronization. The sending device assigns a timestamp to each RTP packet, and the receiving device uses these timestamps to restore the original timing of the data to ensure that the audio / video is played correctly and synchronously on the receiving device.

[0076] Payload type and encoding support: RTP supports a variety of real-time data payload types, including audio, video, and other multimedia data. It provides flexible payload type identification and encoding mechanisms to adapt to different application requirements. For example, for audio data, RTP supports multiple audio codecs, such as G.711 and G.729.

[0077] Delay & Jitter Buffering (D&J) refers to the use of delay and jitter buffering mechanisms in RTP to mitigate delay variations and jitter in network transmission. The receiving device can use a buffer to receive and store RTP packets and adjust the jitter buffering strategy to mitigate delay fluctuations and provide a smooth real-time data playback experience.

[0078] The RTP Control Protocol (RTCP) is a companion protocol designed to support RTP's load control and statistics collection. RTP works in conjunction with the RTP Control Protocol (RTCP). RTCP enables participants to exchange information such as transmission quality, receiver reports, and sender feedback, optimizing data transmission and improving performance.

[0079] Step 302: Determine at least one target video data packet corresponding to the target frame according to the tag values and timestamps of the multiple video data packets.

[0080] The tag value is used to indicate whether the video data packet is the last data packet corresponding to the target frame.

[0081] After acquiring the plurality of video data packets, at least one target video data packet corresponding to the target frame can be determined based on the tag values and timestamps of the plurality of video data packets. Thus, after determining the at least one target video data packet corresponding to the target frame, it can be determined based on the at least one target video data packet corresponding to the target frame whether the video diagnosis result of the target frame is abnormal.

[0082] In some examples, the tag value may be a marker value. Generally, the header of each video data packet in the plurality of video data packets (ie, RTP packets) includes a bit of a marker value (marker bit), and the bit of the marker value is "M".

[0083] If the bit position of the marker value is 1, that is, the marker value of the video data packet is 1, it means that the video data packet is the last data packet corresponding to the current frame. In other words, when the marker value of a video data packet is 1, it indicates that the current group of continuous RTP packets has ended and constitutes a complete frame data.

[0084] If the bit position of the marker value is 0, that is, the marker value of the video data packet is 0, it means that the video data packet is not the last data packet corresponding to the current frame. In other words, when the marker value of a video data packet is 0, it indicates that the current group of continuous RTP packets has not ended and cannot form a complete frame data.

[0085] For example, when the receiving device receives a video data packet, each time a video data packet is received, the frame to which the video data packet belongs and whether the video data packet is the last data packet in the frame can be determined based on the tag value and timestamp of the video data packet.

[0086] Therefore, the tag value can be used to determine whether the current video data packet is the last part of the target frame, and the timestamp can ensure the time synchronization of the video data packets, thereby accurately determining at least one target video data packet corresponding to the target frame.

[0087] In some embodiments, as Figure 5 As shown in the above Figure 3 Based on the illustrated embodiment, before determining at least one target video data packet corresponding to the target frame according to the tag values and timestamps of the plurality of video data packets in step 302 , the method further includes step 501 .

[0088] Step 501: Store multiple video data packets according to a preset structure.

[0089] The preset structure includes a sequence number, a timestamp, and a tag value.

[0090] In some examples, after receiving multiple video data packets, the electronic device may store the multiple video data packets according to a preset structure. For example, the electronic device may store the multiple video data packets in a container of a preset structure. For example, the container is a map container. The preset structure is std::map<int,RtpData> RtpDataMap. That is, the map container is a container that exists in the form of key-value pairs.

[0091] As can be seen from the above content, the key part of the map container uses the int type, and the value of this int type represents the sequence number of the video data packet. The sequence number is used to identify the order of each video data packet and determine the integrity of the video data packet.

[0092] The value part uses a custom structure called RtpData, which is used to store the relevant content of each video data packet. For example, the structure includes: a sequence number (the same as the sequence number used as the key), a timestamp, and a marker value.

[0093] In some examples, the structure RtpData in the preset structure also includes information such as the valid length of the video data packet and the video data of the video data packet.

[0094] Table 1 exemplarily shows a plurality of video data packets in a map container.

[0095] Table 1

[0096]

[0097] According to Table 1, the map container includes information of 6 video data packets.

[0098] The payload type of the first video data packet is DynamiRTP-Type-96; the source is 0x6D1 E8276; the sequence number is 36894, the generation time is 1946110350, and the tag value is 1.

[0099] The payload type of the second video packet is DynamiRTP-Type-96; the source is 0x6D1 E8276; the sequence number is 36895, the generation time is 1946117550, and the tag value is 1.

[0100] The payload type of the third video packet is DynamiRTP-Type-96; the source is 0x6D1 E8276; the sequence number is 36896, the generation time is 1946135550, and the tag value is 0.

[0101] The payload type of the fourth video packet is DynamiRTP-Type-96; the source is 0x6D1 E8276; the sequence number is 36897, the generation time is 1946135550, and the tag value is 0.

[0102] The payload type of the fifth video packet is DynamiRTP-Type-96; the source is 0x6D1 E8276; the sequence number is 36898, the generation time is 1946135550, and the tag value is 0.

[0103] The payload type of the sixth video packet is DynamiRTP-Type-96; the source is 0x6D1 E8276; the sequence number is 36899, the generation time is 1946135550, and the tag value is 1.

[0104] Storing multiple video data packets in a map container allows electronic devices to quickly find, insert, or update video data packets based on their sequence numbers, while also maintaining the order of the video data packets. This approach can also be applied to scenarios where sequential processing of RTP streams is required, such as real-time audio and video transmission.

[0105] In some embodiments, as Figure 6 As shown in the above Figure 3 Based on the illustrated embodiment, the above step 302 of determining at least one target video data packet corresponding to the target frame according to the tag values and timestamps of the plurality of video data packets includes steps 601 and 602.

[0106] Step 601: Determine at least one video data packet corresponding to a target frame according to the timestamp of each video data packet among a plurality of video data packets.

[0107] The timestamp of at least one video data packet corresponding to the target frame is the same.

[0108] As can be seen from the above, when data is transmitted via RTP, one frame of data can be split into multiple RTP packets. Therefore, for one frame of data, the timestamps of multiple video data packets corresponding to the frame of data are the same.

[0109] Therefore, after the electronic device receives multiple video data packets, it can determine at least one video data packet corresponding to the target frame according to the time stamp of each video data packet in the multiple video data packets.

[0110] For example, referring to Table 1, it can be seen that the timestamps of the 3rd video data packet, the 4th video data packet, the 5th video data packet and the 6th video data packet are the same, so the 3rd video data packet, the 4th video data packet, the 5th video data packet and the 6th video data packet can be considered as video data packets corresponding to the same frame.

[0111] Step 602: When there is a video data packet with a mark value of a preset value in at least one video data packet, determine the at least one video data packet as at least one target video data packet corresponding to the target frame.

[0112] For example, it can be determined whether there is a video data packet with a tag value equal to a preset value in at least one video data packet. If a video data packet with a tag value equal to the preset value exists in at least one video data packet, it indicates that the electronic device has completely received all video data packets corresponding to the frame, and thus the at least one video data packet can be determined as the at least one target video data packet. If a video data packet with a tag value equal to the preset value does not exist in at least one video data packet, it indicates that the electronic device has not completely received all video data packets corresponding to the frame, and the electronic device can continue to wait for receiving other video data packets corresponding to the frame until all video data packets corresponding to the frame are received.

[0113] By checking the timestamp and tag value, it can be ensured that the selected video data packets belong to the same video frame. This not only improves the accuracy and efficiency of data processing, but also enhances the robustness of the system, enabling it to better perform real-time video transmission services.

[0114] Step 303: Determine a video diagnosis result of a target frame according to the sequence number of at least one target video data packet.

[0115] When at least one target video data packet corresponding to the target frame is determined, the video diagnosis result of the target frame may be determined according to the sequence numbers of all the target video data packets.

[0116] In some embodiments, as Figure 7 As shown in the above Figure 3 Based on the illustrated embodiment, step 303 , determining a video diagnosis result of a target frame according to a sequence number of at least one target video data packet, includes step 701 .

[0117] Step 701: When the sequence number of at least one target video data packet does not conform to a preset arrangement, determine that a video diagnosis result of a target frame is abnormal.

[0118] Among them, the preset arrangement method includes sequential arrangement.

[0119] After determining at least one target video data packet corresponding to the target frame, it can be determined whether the video data packet corresponding to the target frame is abnormal based on the sequence number of the at least one target video data packet.

[0120] For example, it can be determined whether the sequence number of at least one target video data packet is arranged in a preset arrangement; if the sequence number of at least one target video data packet does not conform to the preset arrangement, the video diagnosis result of the target frame is determined to be abnormal. If the sequence number of at least one target video data packet conforms to the preset arrangement, it is determined that the video data packet corresponding to the target frame does not have an abnormality.

[0121] In some embodiments, as Figure 8 As shown in the above Figure 7 Based on the illustrated embodiment, step 701, according to the serial number of at least one target video data packet, when the serial number of at least one target video data packet does not conform to the preset arrangement, determines that the video diagnosis result of the target frame is abnormal, including steps 801-802.

[0122] Step 801: When the sequence number of at least one target video data packet is discontinuous, determine that the video diagnosis result of the target frame is packet loss.

[0123] Step 802: When the sequence number of at least one target video data packet is repeated, determine that the video diagnosis result of the target frame is a packet duplication.

[0124] In some examples, when the serial number of at least one target video data packet does not conform to the preset arrangement, it may specifically be that: the serial number of at least one target video data packet is discontinuous, or the serial number of at least one target video data packet is repeated.

[0125] If the sequence number of at least one target video data packet is discontinuous, it can be considered that the video data packet corresponding to the target frame has packet loss, that is, the video diagnosis result of the target frame is packet loss. If the sequence number of at least one target video data packet is repeated, it can be considered that the video data packet corresponding to the target frame has packet duplication, that is, the video diagnosis result of the target frame is packet duplication.

[0126] For example, if the serial numbers of at least one target video data packet corresponding to the target frame are 201, 202, 203, 206, 208, 209, and 210, respectively, it indicates that packet loss occurs in the target frame, and the serial numbers of the lost video data packets are 204, 205, and 207.

[0127] If the serial numbers of at least one target video data packet corresponding to the target frame are 301, 302, 303, 303, 304, 304, and 305 respectively, it indicates that there are packet duplications in the target frame, and the serial numbers of the duplicated video data packets are 303 and 304.

[0128] For another example, in the video data packets shown in Table 1, since the 3rd video data packet, the 4th video data packet, the 5th video data packet and the 6th video data packet belong to the video data packets corresponding to the same frame, and the mark value of the 6th video data packet is 1, it means that all data packets corresponding to the frame have been received, and according to the serial number, it can be seen that the serial numbers of these 4 video data packets are arranged in order as 36896, 36897, 36898, and 36899, which are continuous and there is no duplication, so it can be determined that the video diagnosis result of the frame is normal.

[0129] It should be noted that the embodiment of the present disclosure does not limit the execution order between step 801 and step 802. For example, only step 801 may be executed; only step 802 may be executed; or both steps 801 and 802 may be executed simultaneously, which may be determined based on actual usage requirements.

[0130] In some examples, when it is determined that packet loss exists in a target frame, the packet loss ratio of the target frame can be determined based on the serial number of at least one target video data packet. When the packet loss ratio of the target frame exceeds a first value, the video diagnosis result is determined to be a distorted screen.

[0131] Referring to step 802 above, when the serial number of at least one target video data packet corresponding to the target frame is: 201, 202, 203, 206, 208, 209, 210 (a total of 7), and the serial numbers of the lost video data packets are 204, 205 and 207 (a total of 3), then the packet loss ratio of the target frame is 3 / (7+3)=0.3.

[0132] Exemplarily, the first value is 20%, that is, the packet loss ratio of the target frame exceeds 20%, and the video diagnosis result is considered to be a distorted screen.

[0133] In other examples, when it is determined that there is packet duplication in the target frame, the repetition ratio of the target frame can be determined based on the serial number of at least one target video data packet. When the repetition ratio of the target frame exceeds a second value, the video diagnosis result is determined to be a packet abnormality.

[0134] Referring to step 802 above, when the serial number of at least one target video data packet corresponding to the target frame is: 301, 302, 303, 303, 304, 304, 305 (a total of 7), and the serial numbers of the repeated video data packets are 303 and 304 (a total of 2), the repetition ratio of the target frame is 2 / (7)≈0.28.

[0135] Exemplarily, the second value is 20%, that is, the repetition ratio of the target frame exceeds 20%, and the video diagnosis result is considered to be packet repetition.

[0136] In some embodiments, as Figure 9 As shown in the above Figure 3 Based on the illustrated embodiment, the method further includes step 901 .

[0137] Step 901: If no video data packet is received within a preset time window, it is determined that the video diagnosis result is that the video transmitter is disconnected.

[0138] In some examples, if no video data packet is received within a preset time window for a preset duration, it is considered that the video transmitter has a flow interruption. For example, the preset duration may be 10 seconds.

[0139] The disclosed embodiments provide a video diagnostic method that, after receiving multiple video data packets, accurately identifies at least one target video packet corresponding to a target frame by examining the tag value and timestamp of each video packet. Subsequently, based on the target video packet sequence number, a preliminary assessment of the target frame's quality can be performed, allowing potential issues to be promptly identified and addressed. This not only enables a preliminary assessment of the target frame's quality but also significantly improves the processing efficiency of video quality assessment.

[0140] Compared to related technologies that require decoding multiple video packets before evaluating the video quality of the target frame, the present invention uses the header information of the video packet (such as the tag value, timestamp, and sequence number) to make a preliminary judgment. This can quickly identify possible quality issues without decoding the entire video packet. This not only speeds up video quality detection, but also significantly reduces the consumption of computing resources.

[0141] In some examples, after preliminarily judging the video quality using the header information of the video data packet, the video data packet can be further decoded if necessary, and the video quality can be detected using image recognition technology or deep learning models to further improve the accuracy of video quality detection.

[0142] The above mainly introduces the solution provided by the embodiment of the present disclosure from the perspective of method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0143] The embodiments of the present disclosure can divide the electronic device into functional modules according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present disclosure is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0144] Corresponding to the method in the aforementioned embodiment, the present disclosure also provides a decision-making device. This decision-making device is used to implement the aforementioned video diagnosis method. The functions of the decision-making device can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions.

[0145] For example, Figure 10 A schematic diagram of the structure of a video diagnostic device is shown. Figure 10 As shown, the decision-making device may include: a receiving module 1001 and a processing module 1002.

[0146] The receiving module 1001 is configured to receive multiple video data packets. Figure 3 , the receiving module 1001 can be used to execute step 301.

[0147] The processing module 1002 is configured to determine at least one target video data packet corresponding to the target frame according to the tag values and timestamps of the plurality of video data packets; the tag value is used to indicate whether the video data packet is the last data packet corresponding to the target frame; and determine the video diagnosis result of the target frame according to the sequence number of the at least one target video data packet. Figure 3 , the processing module 1002 can be used to execute step 302 and step 303.

[0148] Optionally, the processing module 1002 is further configured to determine that the video diagnosis result of the target frame is abnormal when the sequence number of at least one target video data packet does not conform to the preset arrangement. Figure 7 , the processing module 1002 can be used to execute step 701.

[0149] Optionally, the processing module 1002 is further configured to determine that the video diagnosis result of the target frame is packet loss when the sequence number of at least one target video data packet is discontinuous; and to determine that the video diagnosis result of the target frame is packet duplication when the sequence number of at least one target video data packet is repeated. Figure 8 , the processing module 1002 can be used to execute step 801 and step 802.

[0150] Optionally, the processing module 1002 is further configured to determine at least one video data packet corresponding to the target frame based on the timestamp of each video data packet in the plurality of video data packets; the timestamp of the at least one video data packet corresponding to the target frame is the same; and when there is a video data packet with a tag value of a preset value in the at least one video data packet, determine the at least one video data packet as the at least one target video data packet corresponding to the target frame. For example, in combination with Figure 6 , the processing module 1002 can be used to execute step 601 and step 602.

[0151] Optionally, the processing module 1002 is further configured to store multiple video data packets according to a preset structure, the preset structure including a sequence number, a timestamp and a tag value. Figure 5 , the processing module 1002 can be used to execute step 501.

[0152] Optionally, the preset structure further includes a valid length of the video data packet and video data of the video data packet.

[0153] Optionally, the processing module 1002 is further configured to determine that the video diagnosis result is a video transmitter disconnection if no video data packet is received within a preset time window. Figure 9 , the processing module 1002 can be used to execute step 901.

[0154] Of course, the path selection device provided by the embodiments of the present disclosure includes but is not limited to the above modules. For example, the path selection device may further include a storage module 1003. The storage module 1003 may be used to store program code of the write path selection device, and may also be used to store data generated during operation of the write path selection device, such as data in a write request.

[0155] Figure 11 A structural diagram of an electronic device provided in an embodiment of the present disclosure, such as Figure 11 As shown, the electronic device 1100 includes one or more processors 1101 and a memory 1102 .

[0156] The processor 1101 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1100 to perform desired functions.

[0157] The memory 1102 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and the processor 1101 may execute the program instructions to implement the video diagnosis method, live broadcast parameter data processing method, and / or other desired functions of the various embodiments of the present disclosure described above.

[0158] In one example, the electronic device 1100 may further include an input device 1103 and an output device 1104 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0159] Of course, to simplify, Figure 11 Only some of the components related to the present disclosure in the electronic device 1100 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 1100 may further include any other appropriate components.

[0160] In addition to the above methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes the steps of the video diagnosis method in the various embodiments above.

[0161] The computer program product may be written in any combination of one or more programming languages to implement the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0162] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the video diagnosis method according to various embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.

[0163] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0164] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0165] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are merely illustrative examples and do not imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "including," "comprising," "having," and the like are open-ended words that refer to "including but not limited to" and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to" and can be used interchangeably therewith.

[0166] It should also be noted that in the apparatus, device, and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0167] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0168] The above description has been provided for the purpose of illustration and description. In addition, this description is not meant to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A video diagnostic method, applied to an electronic device, comprising: receiving a plurality of video data packets; Determining at least one target video data packet corresponding to a target frame according to the tag values and timestamps of the plurality of video data packets; The tag value is used to indicate whether the video data packet is the last data packet corresponding to the target frame; The video diagnosis result of the target frame is determined according to the serial number of the at least one target video data packet.

2. The method according to claim 1, characterized in that Determining the video diagnosis result of the target frame according to the sequence number of the at least one target video data packet includes: In a case where the sequence number of the at least one target video data packet does not conform to a preset arrangement, it is determined that the video diagnosis result of the target frame is abnormal.

3. The method according to claim 2, characterized in that When the sequence number of the at least one target video data packet does not conform to a preset arrangement, determining that the video diagnosis result of the target frame is abnormal includes: When the sequence number of the at least one target video data packet is discontinuous, determining that the video diagnosis result of the target frame is packet loss; In a case where the sequence number of the at least one target video data packet is repeated, it is determined that the video diagnosis result of the target frame is a packet duplication.

4. The method according to claim 1, wherein The step of determining at least one target video data packet corresponding to a target frame according to the tag values and timestamps of the plurality of video data packets comprises: determining, based on the timestamp of each video data packet in the plurality of video data packets, at least one video data packet corresponding to the target frame; wherein the timestamp of the at least one video data packet corresponding to the target frame is the same; When a video data packet with a mark value of a preset value exists in the at least one video data packet, the at least one video data packet is determined as at least one target video data packet corresponding to the target frame.

5. The method according to any one of claims 1 to 4, characterized in that Before determining at least one target video data packet corresponding to the target frame based on the tag values and timestamps of the multiple video data packets, the method further includes: The plurality of video data packets are stored according to a preset structure, where the preset structure includes the sequence number, the timestamp, and the tag value.

6. The method according to claim 5, characterized in that The preset structure also includes the effective length of the video data packet and the video data of the video data packet.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: In the preset time window, if no video data packet is received within the preset time length, it is determined that the video diagnosis result is that the video sending end is disconnected.

8. A video diagnostic device, characterized in that: include: A receiving module configured to receive a plurality of video data packets; a processing module configured to determine at least one target video data packet corresponding to a target frame according to the tag values and timestamps of the plurality of video data packets; The tag value is used to indicate whether the video data packet is the last data packet corresponding to the target frame; and the video diagnosis result of the target frame is determined according to the sequence number of the at least one target video data packet.

9. An electronic device, characterized in that: The video diagnosis method comprises a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the video diagnosis method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the video diagnosis method according to any one of claims 1 to 7.

Images