Video anomaly backtracking method and device

Abstract

Embodiments of the present application relate to the technical field of equipment inspection, and particularly relate to a video abnormal backtracking method and device. The method is applied to an application server and comprises: acquiring video code stream data, uplink sending monitoring traffic and uplink receiving monitoring traffic; determining a traffic abnormal time and a traffic abnormal degree coefficient of a camera based on the uplink sending monitoring traffic and the uplink receiving monitoring traffic; determining a picture quality abnormal segment from the video code stream data and determining a picture quality abnormal time and a picture quality abnormal degree value; setting a first edge between a camera graph node and a gateway graph node according to the traffic abnormal degree coefficient; determining a target picture quality abnormal time adjacent to the traffic abnormal time, and setting a second edge between the camera graph node and the gateway graph node according to the corresponding picture quality abnormal degree value. Embodiments of the present application can construct the association between traffic abnormality and unclear picture quality to realize traffic backtracking.

Description

[Technical Field]

[0001] This invention relates to the field of equipment inspection technology, and in particular to a video anomaly backtracking method and device. [Background Technology]

[0002] Current video network surveillance systems consist of resource points (cameras), transmission networks, gateways, gateway acquisition servers, and application servers. In these systems, abnormal traffic often leads to unclear video quality. Existing resource points (cameras) lack traffic monitoring programs, and the gateways only provide real-time traffic query functions, failing to store historical traffic data, thus preventing traffic backtracking.

[0003] Therefore, how to achieve video traffic backtracking is the core issue that this proposal aims to address. [Summary of the Invention]

[0004] This invention provides a method and device for tracing video anomalies. By constructing a knowledge graph that can trace traffic, this invention enables intuitive observation and identification of situations where the network is not interrupted but video quality is unclear due to abnormal traffic.

[0005] In a first aspect, embodiments of the present invention provide a video anomaly backtracking method, the method being applied to an application server, comprising:

[0006] Acquire the video stream data of the camera and the uplink transmission monitoring traffic of the camera, as well as the uplink reception monitoring traffic of the camera generated by the gateway;

[0007] Based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic, determine the time of traffic anomaly of the camera and the traffic anomaly degree coefficient at the time of traffic anomaly;

[0008] Identify the segments with abnormal image quality from the video stream data, and determine the time of the abnormal image quality segment and the corresponding degree of abnormal image quality.

[0009] Based on the traffic anomaly severity coefficient corresponding to the time of the traffic anomaly, a first side is set between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively.

[0010] Determine a target image quality anomaly time that is adjacent to the traffic anomaly time from the image quality anomaly times, and set a second side between the camera map node and the gateway map node according to the image quality anomaly degree value of the target image quality anomaly time.

[0011] Wherein, both the first edge and the second edge are associated with the image quality abnormality segment at the time of the target image quality abnormality.

[0012] In one possible implementation, determining the moment of abnormal traffic from the camera based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic includes:

[0013] Determine a first difference between the uplink transmitted monitoring traffic and the uplink received monitoring traffic with the same time value;

[0014] Determine whether the first difference is greater than a preset threshold;

[0015] If the first difference is greater than the preset threshold, the abnormal traffic time is determined based on the corresponding time value.

[0016] In one possible implementation, determining the traffic anomaly severity coefficient at the moment of traffic anomaly based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic includes:

[0017] The second difference is obtained by subtracting the first difference corresponding to the time of the traffic anomaly from the preset threshold.

[0018] The ratio of the second difference to the preset threshold is determined as the traffic anomaly degree coefficient at the corresponding time of traffic anomaly.

[0019] In one possible implementation, determining the quality-abnormal segment from the video stream data, and determining the time of the quality-abnormality segment and the corresponding quality-abnormality level, includes:

[0020] Convert the images in the video stream data into grayscale images;

[0021] The gradient value of the grayscale image is calculated based on the Sobel operator;

[0022] Based on the gradient value of the grayscale image, the image quality sharpness value of the corresponding image is determined;

[0023] Images with image quality clarity values ​​less than a clarity threshold are identified as image quality abnormal segments, wherein the time corresponding to the image quality abnormal segment is the image quality abnormal moment;

[0024] The degree of image quality abnormality of the image quality abnormal segment is determined based on the image quality sharpness value of the image quality abnormal segment.

[0025] In one possible implementation, setting a first edge between the camera map node and the gateway map node corresponding to the camera and the gateway respectively, based on the traffic anomaly severity coefficient corresponding to the time of the traffic anomaly, includes:

[0026] Based on the traffic anomaly severity coefficient at the time of the traffic anomaly, the color and thickness of the first side are set between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively.

[0027] In one possible implementation, setting a second side between the camera map node and the gateway map node based on the image quality anomaly level value at the time of the target image quality anomaly includes:

[0028] Based on the degree of image quality abnormality at the moment of the target image quality abnormality, the color and thickness of the second side are set between the camera map node and the gateway map node.

[0029] One possible implementation further includes: if the first difference is not greater than the preset threshold, then the normal traffic time is determined according to the corresponding time value;

[0030] A third side is set between the camera node and the gateway node when the traffic is normal, and the third side is used to indicate that the traffic is normal.

[0031] This invention constructs a correlation between abnormal traffic and unclear image quality using a knowledge graph. The degree of image and traffic anomalies is represented by the degree of edge variation, thereby enabling backtracking of information using knowledge graph technology and assisting analysts in observing the causes of image quality malfunctions.

[0032] Secondly, embodiments of the present invention provide an application server, comprising:

[0033] The acquisition module is used to acquire the video bitstream data of the camera and the uplink transmission monitoring traffic of the camera, as well as the uplink reception monitoring traffic of the camera generated by the gateway.

[0034] The first determining module is used to determine the time of abnormal traffic of the camera and the degree coefficient of abnormal traffic at the time of abnormal traffic based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic.

[0035] The second determining module is used to determine the abnormal image quality segment from the video bitstream data, and to determine the abnormal image quality segment at the time of the abnormal image quality and the corresponding abnormal image quality degree value.

[0036] The first setting module is used to set a first side between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively, according to the traffic abnormality degree coefficient corresponding to the time of the traffic abnormality.

[0037] The second setting module is used to determine a target image quality abnormality time that is adjacent to the traffic abnormality time from the image quality abnormality time, and to set a second side between the camera map node and the gateway map node according to the image quality abnormality degree value of the target image quality abnormality time.

[0038] In one possible implementation, the first determining module is specifically used to determine a first difference between the uplink transmission monitoring traffic and the uplink reception monitoring traffic with the same time value;

[0039] Determine whether the first difference is greater than a preset threshold;

[0040] If the first difference is greater than the preset threshold, the abnormal traffic time is determined based on the corresponding time value.

[0041] In one possible implementation, the first determining module is further specifically used to subtract the first difference corresponding to the abnormal traffic time from the preset threshold to obtain a second difference;

[0042] The ratio of the second difference to the preset threshold is determined as the traffic anomaly coefficient at the corresponding time when the traffic is normal.

[0043] In one possible implementation, the second determining module is specifically used to convert the image in the video stream data into a grayscale image;

[0044] The gradient value of the grayscale image is calculated based on the Sobel operator;

[0045] Based on the gradient value of the grayscale image, the image quality sharpness value of the corresponding image is determined;

[0046] Images with image quality clarity values ​​less than a clarity threshold are identified as image quality abnormal segments, wherein the time corresponding to the image quality abnormal segment is the image quality abnormal moment;

[0047] The degree of image quality abnormality of the image quality abnormal segment is determined based on the image quality sharpness value of the image quality abnormal segment.

[0048] In one possible implementation, the first setting module is specifically used to set the color and thickness of the first side between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively, according to the traffic anomaly degree coefficient at the time of the traffic anomaly.

[0049] In one possible implementation, the second setting module is specifically used to set the color and thickness of a second side between the camera map node and the gateway map node based on the degree of image quality abnormality at the moment of the target image quality abnormality.

[0050] One possible implementation also includes: a third determining module, used to determine the normal traffic time based on the corresponding time value if the first difference is not greater than the preset threshold;

[0051] The third setting module is used to set a third side between the camera map node and the gateway map node when the traffic is normal, and the third side is used to indicate that the traffic is normal.

[0052] Thirdly, embodiments of the present invention provide an electronic device, comprising:

[0053] At least one processor; and

[0054] At least one memory communicatively connected to the processor, wherein:

[0055] The memory stores program instructions that can be executed by the processor, and the processor can execute the method provided in the first aspect by calling the program instructions.

[0056] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that cause the computer to perform the method provided in the first aspect.

[0057] It should be understood that the second to fourth aspects of this specification are consistent with the technical solutions of the first aspect of this specification, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, so they will not be described again. [Attached Image Description]

[0058] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 A schematic diagram of a video network monitoring system provided in an embodiment of the present invention;

[0060] Figure 2 A flowchart of a video anomaly backtracking method provided in an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of a knowledge graph provided in an embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of the structure of an application server provided in an embodiment of the present invention;

[0063] Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

Detailed Implementation Methods

[0064] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0065] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of this specification.

[0066] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0067] This invention relates to a video network monitoring system called the Skynet video network monitoring system. The Skynet system includes resource points (cameras), a transmission network, a gateway, a gateway acquisition server, and an application server. The resource points (cameras) send their performance monitoring results and video stream bitrates to the gateway acquisition server via the transmission network and gateway. The application server then uses the performance data collected by the gateway acquisition server to perform various application analyses. Currently, existing resource points (cameras) lack traffic monitoring programs, and the gateway can only view the current traffic. In network systems, abnormal traffic often leads to unclear image quality.

[0068] The present invention first proposes a video network monitoring system. Figure 1 This is a schematic diagram of a video network monitoring system provided in an embodiment of the present invention, such as... Figure 1As shown, the video network monitoring system 100 includes resource points 101, a gateway 102, a gateway acquisition server 103, and an application server 104. Resource points 101 contain P cameras. The system connects to the gateway acquisition server via an internal network routing configuration, and then to a Passive Optical Network (PON) gateway or a Packet Transport Network (PTN) gateway to collect uplink transmission monitoring traffic, video stream data, and uplink reception monitoring traffic from each camera in resource points 101. Each camera uploads uplink transmission monitoring traffic and video stream data to the gateway 102. The gateway 102 acquires the uplink transmission monitoring traffic and video stream data, generates uplink reception monitoring traffic, and uploads these data to the gateway acquisition server 103. The gateway acquisition server 103 stores the uplink transmission monitoring traffic, video stream data, and uplink reception monitoring traffic. Application server 104 obtains uplink transmission monitoring traffic, video stream data and uplink reception monitoring traffic from gateway acquisition server 103.

[0069] This invention provides a video playback method, which is applied to application server 104. For example... Figure 2 As shown, Figure 2 This is a flowchart of a video anomaly backtracking method provided by an embodiment of the present invention. The method includes:

[0070] Step 201: The application server obtains the video stream data of the camera and the uplink transmission monitoring traffic of the camera, as well as the uplink reception monitoring traffic of the camera generated by the gateway.

[0071] In some embodiments, taking camera 1 in resource point 101 as an example, camera 1 uploads uplink transmission monitoring traffic and video stream data at multiple times to gateway 102. Gateway 102 sends the uplink transmission monitoring traffic and video stream data at multiple times, as well as the generated uplink reception monitoring traffic at multiple times, to gateway acquisition server 103. Gateway acquisition server 103 stores the uplink transmission monitoring traffic, video stream data, and uplink reception monitoring traffic at multiple times. Application server 104 obtains the uplink transmission monitoring traffic, video stream data, and uplink reception monitoring traffic at multiple times from gateway acquisition server.

[0072] Step 202: The application server determines the time of traffic anomaly of the camera and the degree coefficient of traffic anomaly at the time of traffic anomaly based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic.

[0073] In some embodiments, taking the uplink transmission monitoring traffic and uplink reception monitoring traffic of camera 1 at time X1 as an example, application server 104 determines whether there is a traffic anomaly at time X1 based on the difference between the uplink transmission monitoring traffic and the uplink reception monitoring traffic. If a traffic anomaly exists, a traffic anomaly degree coefficient is calculated.

[0074] Specifically, determining the time of abnormal traffic for the camera based on the uplink transmission monitoring traffic and the uplink reception monitoring traffic includes: calculating a first difference between the uplink transmission monitoring traffic and the uplink reception monitoring traffic at the same time; determining whether the first difference is greater than a preset threshold; and if the first difference is greater than the preset threshold, then the time is determined as the time of abnormal traffic.

[0075] Let the uplink receive monitoring traffic value at time X1 be Q1 and the uplink transmit monitoring traffic value be Q2. Let the threshold be Qy. Application server 104 calculates Q1-Q2=Qx, where Qx is the difference between the two traffic values. If Qx is greater than Qy, then it is determined that there is a traffic anomaly at time X1, and time X1 is the time of traffic anomaly.

[0076] Specifically, based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic, the traffic anomaly degree coefficient at the time of the traffic anomaly is determined, including: subtracting the first difference at the time of the traffic anomaly from the preset threshold to obtain a second difference; and determining the ratio of the second difference to the preset threshold as the traffic anomaly degree coefficient at the first time.

[0077] After application server 104 determines that the difference Qx is greater than the threshold Qy, the degree of anomaly is calculated based on the percentage exceeding the threshold. The calculation is k = Qx - Qy / Qy, where k represents the traffic anomaly degree coefficient.

[0078] Step 203: The application server determines the abnormal image quality segment from the video stream data, and determines the time of the abnormal image quality segment and the corresponding degree of abnormal image quality.

[0079] In some embodiments, the application server 104 calls the video stream data to perform image quality anomaly analysis, find the image quality anomaly segment, the time point when the image quality anomaly occurs, and the degree of image quality anomaly.

[0080] Specifically, the images in the video stream data are converted into grayscale images; the gradient values ​​of the grayscale images are calculated based on the Sobel operator; the image quality sharpness value of the corresponding image is determined according to the gradient values ​​of the grayscale images; images with image quality sharpness values ​​less than a sharpness threshold are identified as image quality aberration segments, wherein the time corresponding to the image quality aberration segment is the image quality aberration moment; and the image quality aberration degree value of the image quality aberration segment is determined according to the image quality sharpness value of the image quality aberration segment.

[0081] When performing image quality anomaly analysis on application server 104, a sharpness anomaly detection technique is used. The specific steps are as follows: Step 1: Convert the acquired image to a grayscale image. Step 2: Calculate the horizontal and vertical gradient values ​​of the grayscale image using horizontal and vertical Sobel operators, and select the larger value as the image gradient value. Step 3: Calculate the histogram of the gradient image and filter out low gradient values ​​that do not represent edge information. Step 4: Calculate the mean and variance of the remaining gradient values. Step 5: Calculate the image sharpness value using the mean and variance. The calculation formula is shown below:

[0082]

[0083]

[0084] Where, μ grad The mean value calculated for the residual gradient, count del The formula represents the number of low gradient values ​​filtered out based on a threshold t, where m×n is the image size, and the normalized mean is... Multiplication is used to account for the proportion of pixels involved in the mean calculation. Hist grad [i] is the truth function for the gradient value, and SharpValue is the image sharpness value.

[0085] Application server 104 compares the image quality clarity value with a preset threshold. If the image quality clarity value is less than the preset threshold, it determines that the image has an image quality abnormality, i.e., the image is identified as an image quality abnormal segment. The time when the image quality abnormal segment occurs is defined as the image quality abnormality time and the image quality abnormality degree value. If there are I image quality abnormality times, the corresponding number of image quality abnormality times is also I.

[0086] Step 204: The application server sets a first edge between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively, based on the traffic anomaly degree coefficient corresponding to the time of the traffic anomaly.

[0087] In some embodiments, the application server 104 creates a first side that represents the degree of traffic anomaly, with cameras and network management systems both regarded as traffic nodes, and constructs a knowledge graph in combination with the first side.

[0088] Specifically, based on the traffic anomaly severity coefficient at the time of the traffic anomaly, the color and thickness of the first side are set between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively.

[0089] A first edge is set between the camera map node and the gateway map node. The thickness and color of the first edge are determined by the traffic anomaly degree coefficient k. Different thicknesses or colors indicate different degrees of traffic anomalies.

[0090] Step 205: The application server determines the target image quality abnormality time that is adjacent to the traffic abnormality time from the image quality abnormality time, and sets a second side between the camera map node and the gateway map node according to the image quality abnormality degree value of the target image quality abnormality time.

[0091] In some embodiments, the application server 104 correlates times of image quality anomalies with times of traffic anomalies based on a proximity algorithm. Specifically, for a time of traffic anomaly, a neighboring time is found among I times of image quality anomalies and identified as the target time of image quality anomaly. If a correlation result exists, a second edge representing the degree of image quality anomaly is created next to the corresponding first edge representing the degree of traffic anomaly.

[0092] Specifically, setting a second side between the camera map node and the gateway map node based on the image quality abnormality level value at the time of the target image quality abnormality includes: setting the color and thickness of the second side between the camera map node and the gateway map node based on the image quality abnormality level value at the time of the target image quality abnormality.

[0093] A second edge is set next to the first edge between the camera graph node and the gateway graph node. The thickness and color of the second edge are determined by the image quality anomaly level value; different thicknesses or colors indicate different levels of image quality anomalies. The image quality anomaly segments are overlaid on the knowledge graph, and the screen is set to play when the first or second edge is touched.

[0094] Specifically, it also includes: if the first difference is not greater than the preset threshold, then the normal traffic time is determined according to the corresponding time value; a third side is set between the camera map node and the gateway map node at the normal traffic time, and the third side is used to indicate that the traffic is normal.

[0095] If the traffic difference Qx is less than the threshold Qy, then it is determined that there is no traffic anomaly at time X1, and time X1 is a time of normal traffic. Therefore, a third side representing normal traffic is set between the camera map node and the gateway map node.

[0096] Taking time X1 as an example, the knowledge graph created at time X1 is as follows: Figure 3 As shown, Figure 3This is a schematic diagram of a knowledge graph structure provided in an embodiment of the present invention. The diagram contains P camera graph nodes and one gateway graph node. It is clearly visible that the first and second edges between camera graph node 1 and the gateway graph node are thicker than the first and second edges between camera graph node p and the gateway graph node, indicating that at time X1, the traffic anomaly of camera 1 is higher than that of camera p. The dashed edge between camera graph node 2 and the gateway graph node represents the third edge, indicating normal traffic and signifying that the camera does not experience traffic anomalies.

[0097] This invention constructs a knowledge graph that allows for backtracking of network traffic, enabling intuitive observation and identification of situations where network connectivity is not interrupted but abnormal traffic results in unclear image quality. It can store collected traffic data by time, thus achieving traffic backtracking for 7 days or even longer.

[0098] Figure 4 This is a schematic diagram of the structure of an application server provided in an embodiment of the present invention, such as... Figure 5 As shown, the application server 300 may include: an acquisition module 301, a first determination module 302, a second determination module 303, a first setting module 304, and a second setting module 305. The acquisition module 301 is used to acquire video stream data from the camera and the uplink transmission monitoring traffic of the camera, as well as the uplink reception monitoring traffic of the camera generated by the gateway. The first determination module 302 is used to determine the time of traffic anomaly of the camera and the degree coefficient of traffic anomaly at the time of traffic anomaly based on the uplink transmission monitoring traffic and the uplink reception monitoring traffic. The second determination module 303 is used to determine the image quality anomaly segment from the video stream data, and to determine the time of image quality anomaly at which the image quality anomaly segment is located and the corresponding image quality anomaly value. The first setting module 304 is used to set a first edge between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively, according to the degree coefficient of traffic anomaly at the time of traffic anomaly. The second setting module 305 is used to determine a target image quality abnormality time that is adjacent to the traffic abnormality time from the image quality abnormality time, and set a second side between the camera map node and the gateway map node according to the image quality abnormality degree value of the target image quality abnormality time.

[0099] In the above embodiments of the present invention, optionally, the first determining module 302 is specifically used to determine a first difference between the uplink transmission monitoring traffic and the uplink reception monitoring traffic with the same time value; determine whether the first difference is greater than a preset threshold; if the first difference is greater than the preset threshold, then determine the time of traffic abnormality based on the corresponding time value.

[0100] In the above embodiments of the present invention, optionally, the first determining module 302 is further configured to subtract the first difference corresponding to the abnormal traffic time from the preset threshold to obtain a second difference; and to determine the ratio of the second difference to the preset threshold as the traffic abnormality coefficient at the corresponding normal traffic time.

[0101] In the above embodiments of the present invention, optionally, the second determining module 303 is specifically used to convert the image in the video stream data into a grayscale image; calculate the gradient value of the grayscale image based on the Sobel operator; determine the image quality sharpness value of the corresponding image according to the gradient value of the grayscale image; determine the image with the image quality sharpness value less than the sharpness threshold as an image quality abnormal segment, wherein the time corresponding to the image quality abnormal segment is the image quality abnormal moment; and determine the image quality abnormality degree value of the image quality abnormal segment according to the image quality sharpness value of the image quality abnormal segment.

[0102] In the above embodiments of the present invention, optionally, the first setting module 304 is specifically used to set the color and thickness of the first side between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively, according to the traffic abnormality degree coefficient at the time of the traffic abnormality.

[0103] In the above embodiments of the present invention, optionally, the second setting module 305 is specifically used to set the color and thickness of a second side between the camera map node and the gateway map node according to the degree of image quality abnormality at the time of the target image quality abnormality.

[0104] In the above embodiments of the present invention, optionally, it further includes: a third determining module, used to determine the normal traffic time according to the corresponding time value if the first difference is not greater than the preset threshold; and a third setting module, used to set a third side between the camera map node and the gateway map node at the normal traffic time, the third side being used to indicate that the traffic is normal.

[0105] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0106] like Figure 5 As shown, the electronic device described above may include at least one processor; and at least one memory communicatively connected to the processor, wherein the memory stores program instructions executable by the processor, and the processor can execute this specification by calling the program instructions. Figure 2 The illustrated embodiment provides a video anomaly backtracking method.

[0107] The aforementioned electronic device can be a device capable of recognizing user gestures, such as a cloud server. This specification does not limit the specific form of the electronic device in the embodiments. It is understood that the electronic device here refers to the machine mentioned in the method embodiments.

[0108] Figure 5 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present invention is shown. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0109] like Figure 5 As shown, the electronic device is represented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: one or more processors 410, a communication interface 420, a memory 430, and a communication bus 440 connecting different system components (including the memory 430 and the processing unit 410).

[0110] Communication bus 440 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MAC) buses, Enhanced ISA buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses.

[0111] Electronic devices typically include a variety of computer-readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, and removable and non-removable media.

[0112] Memory 430 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 430 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0113] A program / utility having a set (at least one) of program modules can be stored in memory 430. Such program modules include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules typically perform the functions and / or methods described in the embodiments of the present invention.

[0114] Processor 410 executes various functional applications and data processing by running programs stored in memory 430, such as implementing the present invention. Figure 2 The illustrated embodiment provides a video anomaly backtracking method.

[0115] This invention provides a computer-readable storage medium storing computer instructions that cause a computer to execute the present specification. Figure 2 The illustrated embodiment provides a video anomaly backtracking method.

[0116] The aforementioned computer-readable storage medium may be any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in connection with an instruction execution system, apparatus, or device.

[0117] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0118] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0119] Computer program code for performing the operations described herein can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0120] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0121] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0122] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this specification, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0123] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this specification includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which the embodiments of this specification pertain.

[0124] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0125] It should be noted that the terminals involved in the embodiments of this specification may include, but are not limited to, personal computers (PCs), personal digital assistants (PDAs), wireless handheld devices, tablet computers, mobile phones, MP3 players, MP4 players, etc.

[0126] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0127] Furthermore, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0128] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0129] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A video anomaly backtracking method, characterized in that, The method is applied to an application server and includes: Acquire the video stream data of the camera and the uplink transmission monitoring traffic of the camera, as well as the uplink reception monitoring traffic of the camera generated by the gateway; Based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic, determine the time of traffic anomaly of the camera and the traffic anomaly degree coefficient at the time of traffic anomaly; Determine the segments with abnormal image quality from the video stream data, and determine the time of the abnormal image quality segment and the corresponding degree of abnormal image quality. Based on the traffic anomaly severity coefficient corresponding to the time of the traffic anomaly, a first edge is set between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively. Determine a target image quality anomaly time that is adjacent to the traffic anomaly time from the image quality anomaly times, and set a second side between the camera map node and the gateway map node according to the image quality anomaly degree value of the target image quality anomaly time. Wherein, both the first side and the second side are associated with the image quality abnormality segment at the moment of the target image quality abnormality, the image quality abnormality segment is superimposed on the knowledge graph, and it is set to play when the first side or the second side is touched; The step of setting a first edge between the camera map node and the gateway map node corresponding to the camera and the gateway respectively, based on the traffic anomaly severity coefficient corresponding to the time of the traffic anomaly, includes: Based on the traffic anomaly severity coefficient at the time of the traffic anomaly, the color and thickness of the first side are set between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively. The step of setting a second side between the camera map node and the gateway map node based on the image quality anomaly level value at the time of the target image quality anomaly includes: Based on the degree of image quality abnormality at the moment of the target image quality abnormality, the color and thickness of the second side are set between the camera map node and the gateway map node.

2. The method according to claim 1, characterized in that, The step of determining the moment of abnormal traffic from the camera based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic includes: Determine a first difference between the uplink transmitted monitoring traffic and the uplink received monitoring traffic with the same time value; Determine whether the first difference is greater than a preset threshold; If the first difference is greater than the preset threshold, the abnormal traffic time is determined based on the corresponding time value.

3. The method according to claim 2, characterized in that, The step of determining the traffic anomaly severity coefficient at the moment of traffic anomaly based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic includes: The second difference is obtained by subtracting the first difference corresponding to the time of the traffic anomaly from the preset threshold. The ratio of the second difference to the preset threshold is determined as the traffic anomaly degree coefficient at the corresponding time of traffic anomaly.

4. The method according to claim 1, characterized in that, The step of determining the image quality abnormality segment from the video bitstream data, and determining the time of the image quality abnormality segment and the corresponding image quality abnormality degree value, includes: Convert the images in the video stream data into grayscale images; The gradient value of the grayscale image is calculated based on the Sobel operator; Based on the gradient value of the grayscale image, the image quality sharpness value of the corresponding image is determined; Images with image quality clarity values ​​less than a clarity threshold are identified as image quality abnormal segments, wherein the time corresponding to the image quality abnormal segment is the image quality abnormal moment; The degree of image quality abnormality of the image quality abnormal segment is determined based on the image quality sharpness value of the image quality abnormal segment.

5. The method according to claim 2, characterized in that, The method further includes: If the first difference is not greater than the preset threshold, then the normal flow time is determined according to the corresponding time value; A third side is set between the camera map node and the gateway map node at the time when the traffic is normal, and the third side is used to indicate that the traffic is normal.

6. An application server, characterized in that, include: The acquisition module is used to acquire the video bitstream data of the camera and the uplink transmission monitoring traffic of the camera, as well as the uplink reception monitoring traffic of the camera generated by the gateway. The first determining module is used to determine the time of abnormal traffic of the camera and the degree coefficient of abnormal traffic at the time of abnormal traffic based on the uplink transmitted monitoring traffic and the uplink received monitoring traffic. The second determining module is used to determine the abnormal image quality segment from the video bitstream data, and to determine the abnormal image quality segment at the time of the abnormal image quality and the corresponding abnormal image quality degree value. The first setting module is used to set a first side between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively, according to the traffic abnormality degree coefficient corresponding to the time of the traffic abnormality. The second setting module is used to determine a target image quality abnormality time that is adjacent to the traffic abnormality time from the image quality abnormality time, and set a second side between the camera map node and the gateway map node according to the image quality abnormality degree value of the target image quality abnormality time. Wherein, both the first side and the second side are associated with the image quality abnormality segment at the moment of the target image quality abnormality, the image quality abnormality segment is superimposed on the knowledge graph, and it is set to play when the first side or the second side is touched; The first setting module is specifically used to set the color and thickness of the first side between the camera map node and the gateway map node corresponding to the camera and the gateway, respectively, according to the traffic anomaly degree coefficient at the time of the traffic anomaly. The second setting module is specifically used to set the color and thickness of the second side between the camera map node and the gateway map node based on the degree of image quality abnormality at the moment of the target image quality abnormality.

7. An electronic device, characterized in that, include: At least one processor; as well as At least one memory communicatively connected to the processor, wherein: The memory stores program instructions that can be executed by the processor, and the processor can execute the method as described in any one of claims 1 to 5 by calling the program instructions.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause the computer to perform the method as described in any one of claims 1 to 5.

Images