Abnormal analysis method and system for intelligent surveillance video

Abstract

The present invention provides an intelligent monitoring video abnormality analysis method and system, which belongs to the field of data processing technology, and is used to realize intelligent monitoring video abnormality analysis through lightweight data interaction between an A-IoT device and a reader. The method includes: the reader sends a first R2D message to the A-IoT device, the A-IoT device is deployed in a closed road section, the first R2D message includes a first read command, and the first read command indicates to read the data in the first storage area of ​​the A-IoT device; the reader receives M1 first D2R messages returned by the A-IoT device according to the first R2D message, each of the M1 first D2R messages contains a corresponding video frame, a total of M1 video frames, each of the M1 video frames contains a picture of the closed road section, and multiple frames are spaced between two adjacent video frames in the M1 video frames; the reader analyzes the M1 video frames through an ML model to determine whether there is a security incident in the closed road section, and the ML model is deployed in the reader.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an abnormality analysis method and system for intelligent monitoring videos. Background Art

[0002] With the advent of the information age, the Internet of Things (IoT) has become an important direction of scientific and technological development. On this basis, the lightweight network form of Environmental Internet of Things (A-IoT) came into being, bringing many conveniences to our lives.

[0003] Ambient Internet of Things (A-IoT) is a new form of Internet of Things. It connects various devices in the environment (such as sensors, actuators, etc.) to form an intelligent network that can sense, process and transmit environmental information. In the A-IoT network, the communication between A-IoT devices and readers (such as base stations or terminals) is the key. This communication can be achieved wirelessly. For example, inventory scenario: In warehouses, logistics and other places, A-IoT devices are used as tags, and readers can communicate with A-IoT devices to achieve real-time monitoring and management of inventory, improving the efficiency and accuracy of inventory. For example, command scenario: In scenarios such as smart homes and industrial automation, A-IoT devices can perform corresponding operations such as reading, writing, and deactivation according to the commands received from the reader. Monitoring scenario: In scenarios such as security monitoring and environmental monitoring, A-IoT devices can collect and transmit environmental / monitoring information in real time and return it to the reader to achieve lightweight monitoring. In scenarios where real-time monitoring is not required, such as accident identification and rescue identification of closed road sections, it has a wide range of application value.

[0004] However, in monitoring scenarios, how A-IoT devices interact with readers to achieve intelligent monitoring video anomaly analysis is a current research issue. Summary of the invention

[0005] The embodiments of the present invention provide an intelligent monitoring video anomaly analysis method and system, which are used to realize intelligent monitoring video anomaly analysis through lightweight data interaction between an A-IoT device and a reader.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] In a first aspect, a method for analyzing anomalies in intelligent surveillance videos is provided, which is applied to a reader. The method includes: the reader sends a first R2D message to an A-IoT device, the A-IoT device is deployed in a closed road segment, the first R2D message includes a first read command, and the first read command indicates reading data in a first storage area of ​​the A-IoT device; the reader receives M1 first D2R messages returned by the A-IoT device according to the first R2D message, each of the M1 first D2R messages includes a corresponding video frame, and there are M1 video frames in total, each of the M1 video frames includes a picture of the closed road segment, and multiple frames are spaced between two adjacent video frames in the M1 video frames; the reader analyzes the M1 video frames through an ML model to determine whether there is a security incident in the closed road segment, and the ML model is deployed in the reader.

[0008] Optionally, before the reader sends the first R2D message to the A-IoT device, the method also includes: during the process of the A-IoT device randomly accessing the reader, the reader obtains the EPC of the A-IoT device reported by the A-IoT device; the reader obtains device parameter information of the A-IoT device based on the EPC of the A-IoT device; and the reader determines a first storage area from the storage area of ​​the A-IoT device based on the device parameter information.

[0009] Optionally, the reader obtains device parameter information of the A-IoT device according to the EPC of the A-IoT device, including: the reader sends the EPC of the A-IoT device to a data storage network element in the network, and the network is the network where the reader is located; the reader receives the device parameter information of the A-IoT device returned by the data storage network element according to the EPC of the A-IoT device, the device parameter information of the A-IoT device includes at least the device type of the A-IoT device, or the hardware and software information of the A-IoT device, the device type of the A-IoT device indicates that the A-IoT device is a monitoring device, and the hardware and software information of the A-IoT device includes at least the monitoring resolution of the A-IoT device, or the size of the storage area of ​​the A-IoT device; the reader determines a first storage area from the storage area of ​​the A-IoT device according to the device parameter information, including: in response to the A-IoT device being a monitoring device, the reader determines the size of a video frame collected by the A-IoT device according to the monitoring resolution of the A-IoT device; the reader determines the first storage area from the storage area of ​​the A-IoT device according to the size of a video frame collected by the A-IoT device and the size of the storage area of ​​the A-IoT device, and the size of the first storage area is the same as the size of a video frame collected by the A-IoT device.

[0010] Optionally, the size of the storage area of ​​the A-IoT device is L1, and the size of a video frame collected by the A-IoT device is L2, L1 / L2=K, indicating that the storage area of ​​the A-IoT device can store K complete video frames, K is an integer greater than 1, and also indicates that the storage area of ​​the A-IoT device can be divided into K storage areas, each of the K storage areas is used to store a corresponding video frame, the first storage area is the Q1th storage area among the K storage areas, Q1=P1ModK, P1 is the decimal value corresponding to the bit string consisting of the x1th to y1th bits in the EPC of the A-IoT device, Mod represents modulo operation, and x1 and y1 are positive integers with different values.

[0011] Optionally, the reader allocates wireless transmission resources of the A-IoT device according to the size of a video frame collected by the A-IoT device, and the size of the wireless transmission resources of the A-IoT device matches the size of any one of the M1 first D2R messages.

[0012] Optionally, M1 first D2R messages correspond to M1 transmission opportunities, t is an integer traversing from 1 to M1-1, and the A-IoT device is configured as follows: at the tth transmission opportunity, the A-IoT device stores the data #t collected at the tth transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area into the first D2R message #t and sends it; after the first D2R message #t is sent, the A-IoT device sends the A-IoT device The data #t stored in the storage area is released; thereafter, at the t+1th transmission timing, the A-IoT device stores the data #t+1 collected at the t+1th transmission timing in the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t+1 in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area to the first D2R message #t+1 and sends it; after the first D2R message #t+1 is sent, the A-IoT device continues to release the data #t+1 stored in the storage area of ​​the A-IoT device.

[0013] Optionally, the value of M1 is the upper limit of the number of video frames that the ML model can jointly analyze, and the reader analyzes M1 video frames through the ML model to determine whether there is a security incident in the closed road segment, including: the reader combines any two of the M1 video frames to obtain C(M1,2) combinations; the reader inputs the C(M1,2) combinations into the ML model for analysis to obtain C(M1,2) analysis results. If N of the C(M1,2) analysis results indicate that a security incident has occurred on the same target object, N is a positive integer less than or equal to M1, and the value of N is greater than a preset threshold, then it is determined that a security incident has occurred on the target object; otherwise, it is determined that no security incident has occurred on the object, and the target object is an object in the closed road segment.

[0014] Optionally, if a security incident occurs in the target object, the method also includes: the reader sends a second R2D message to the A-IoT device, the second R2D message includes a second read command, and the second read command indicates reading data in the second storage area and the third storage area of ​​the A-IoT device; the reader receives M2 second D2R messages returned by the A-IoT device based on the second R2D message, each of the M2 second D2R messages includes two corresponding video frames, for a total of 2*M2 video frames, each of the 2*M2 video frames includes a picture of a closed road segment, and there are multiple frames between two adjacent video frames in the 2*M2 video frames; the reader analyzes the pictures of the target object in the M2 video frames and the M1 video frames through the ML model to track the target object in the M2 video frames.

[0015] Optionally, the data in the second storage area corresponds to a video frame, and the data in the third storage area also corresponds to a video frame. The second storage area is the Q2th storage area among the K storage areas, Q2=P2ModK, P2 is the decimal value corresponding to the bit string consisting of the x2th to y2th bits in the EPC of the A-IoT device, x2 and y2 are positive integers with different values, and the third storage area is the Q3th storage area among the K storage areas, Q3=P3ModK, P3 is the decimal value corresponding to the bit string consisting of the x3th to y3th bits in the EPC of the A-IoT device, and x3 and y3 are positive integers with different values.

[0016] Optionally, M2 second D2R messages correspond to M2 transmission opportunities, u is an integer traversing from 1 to M2-1, and the A-IoT device is configured as follows: at the u-th transmission opportunity, the A-IoT device stores the data #u collected at the u-th transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #u in the second storage area and the data #u in the third storage area from the storage area of ​​the A-IoT device according to the second read command, and carries the data #u in the second storage area and the data #u in the third storage area into the second D2R message #u and sends it; after the second D2R message #u is sent, the A-IoT device stores the data #u collected at the u-th transmission opportunity into the storage area of ​​the A-IoT device; The data #u stored in the storage area is released; thereafter, at the u+1th transmission opportunity, the A-IoT device stores the data #u+1 collected at the u+1th transmission opportunity in the storage area of ​​the A-IoT device; according to the second read command, the A-IoT device obtains the data #u+1 in the second storage area and the data #u+1 in the third storage area from the storage area of ​​the A-IoT device, and carries the data #u+1 in the second storage area and the data #u+1 in the third storage area into the second D2R message #u+1 and sends it; after the second D2R message #u+1 is sent, the A-IoT device releases the data #u+1 stored in the storage area of ​​the A-IoT device.

[0017] Optionally, the reader allocates wireless transmission resources of the A-IoT device according to the size of two video frames collected by the A-IoT device, and the size of the wireless transmission resources of the A-IoT device matches the size of any one of the M2 second D2R messages.

[0018] In a second aspect, a smart surveillance video anomaly analysis system is provided, the system includes a reader, and the system is configured as follows: the reader sends a first R2D message to an A-IoT device, the A-IoT device is deployed in a closed road segment, the first R2D message includes a first read command, and the first read command indicates reading data in a first storage area of ​​the A-IoT device; the reader receives M first D2R messages returned by the A-IoT device according to the first R2D message, each of the M first D2R messages includes a corresponding video frame, a total of M video frames, each of the M video frames includes a picture of the closed road segment, and multiple frames are spaced between two adjacent video frames in the M video frames; the reader analyzes the M video frames through an ML model to determine whether there is a security incident in the closed road segment, and the ML model is deployed in the reader.

[0019] The reader sends a first R2D message to the A-IoT device, and the A-IoT device is deployed in a closed road section. The first R2D message includes a first read command, and the first read command indicates to read data in a first storage area of ​​the A-IoT device; the reader receives M1 first D2R messages returned by the A-IoT device according to the first R2D message, and each of the M1 first D2R messages includes a corresponding video frame, and there are M1 video frames in total. Each video frame in the M1 video frames includes a picture of the closed road section, and there are multiple frames between two adjacent video frames in the M1 video frames; the reader analyzes the M1 video frames through the ML model to determine whether there is a security incident in the closed road section, and the ML model is deployed in the reader.

[0020] Optionally, before the reader sends the first R2D message to the A-IoT device, the system is further configured as follows: during the process of the A-IoT device randomly accessing the reader, the reader obtains the EPC of the A-IoT device reported by the A-IoT device; the reader obtains device parameter information of the A-IoT device based on the EPC of the A-IoT device; the reader determines a first storage area from the storage area of ​​the A-IoT device based on the device parameter information.

[0021] Optionally, the reader obtains device parameter information of the A-IoT device according to the EPC of the A-IoT device, including: the reader sends the EPC of the A-IoT device to a data storage network element in the network, and the network is the network where the reader is located; the reader receives the device parameter information of the A-IoT device returned by the data storage network element according to the EPC of the A-IoT device, the device parameter information of the A-IoT device includes at least the device type of the A-IoT device, or the hardware and software information of the A-IoT device, the device type of the A-IoT device indicates that the A-IoT device is a monitoring device, and the hardware and software information of the A-IoT device includes at least the monitoring resolution of the A-IoT device, or the size of the storage area of ​​the A-IoT device; the reader determines a first storage area from the storage area of ​​the A-IoT device according to the device parameter information, including: in response to the A-IoT device being a monitoring device, the reader determines the size of a video frame collected by the A-IoT device according to the monitoring resolution of the A-IoT device; the reader determines the first storage area from the storage area of ​​the A-IoT device according to the size of a video frame collected by the A-IoT device and the size of the storage area of ​​the A-IoT device, and the size of the first storage area is the same as the size of a video frame collected by the A-IoT device.

[0022] Optionally, the size of the storage area of ​​the A-IoT device is L1, and the size of a video frame collected by the A-IoT device is L2, L1 / L2=K, indicating that the storage area of ​​the A-IoT device can store K complete video frames, K is an integer greater than 1, and also indicates that the storage area of ​​the A-IoT device can be divided into K storage areas, each of the K storage areas is used to store a corresponding video frame, the first storage area is the Q1th storage area among the K storage areas, Q1=P1ModK, P1 is the decimal value corresponding to the bit string consisting of the x1th to y1th bits in the EPC of the A-IoT device, Mod represents modulo operation, and x1 and y1 are positive integers with different values.

[0023] Optionally, the reader allocates wireless transmission resources of the A-IoT device according to the size of a video frame collected by the A-IoT device, and the size of the wireless transmission resources of the A-IoT device matches the size of any one of the M1 first D2R messages.

[0024] Optionally, M1 first D2R messages correspond to M1 transmission opportunities, t is an integer traversing from 1 to M1-1, and the A-IoT device is configured as follows: at the tth transmission opportunity, the A-IoT device stores the data #t collected at the tth transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area into the first D2R message #t and sends it; after the first D2R message #t is sent, the A-IoT device sends the A-IoT device The data #t stored in the storage area is released; thereafter, at the t+1th transmission timing, the A-IoT device stores the data #t+1 collected at the t+1th transmission timing in the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t+1 in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area to the first D2R message #t+1 and sends it; after the first D2R message #t+1 is sent, the A-IoT device continues to release the data #t+1 stored in the storage area of ​​the A-IoT device.

[0025] Optionally, the value of M1 is the upper limit of the number of video frames that the ML model can jointly analyze, and the reader analyzes M1 video frames through the ML model to determine whether there is a security incident in the closed road segment, including: the reader combines any two of the M1 video frames to obtain C(M1,2) combinations; the reader inputs the C(M1,2) combinations into the ML model for analysis to obtain C(M1,2) analysis results. If N of the C(M1,2) analysis results indicate that a security incident has occurred on the same target object, N is a positive integer less than or equal to M1, and the value of N is greater than a preset threshold, then it is determined that a security incident has occurred on the target object; otherwise, it is determined that no security incident has occurred on the object, and the target object is an object in the closed road segment.

[0026] Optionally, if a security incident occurs in the target object, the system is further configured as follows: the reader sends a second R2D message to the A-IoT device, the second R2D message includes a second read command, and the second read command indicates reading data in the second storage area and the third storage area of ​​the A-IoT device; the reader receives M2 second D2R messages returned by the A-IoT device based on the second R2D message, each of the M2 second D2R messages includes two corresponding video frames, for a total of 2*M2 video frames, each of the 2*M2 video frames includes a picture of a closed road segment, and there are multiple frames between two adjacent video frames in the 2*M2 video frames; the reader analyzes the pictures of the target object in the M2 video frames and the M1 video frames through the ML model to track the target object in the M2 video frames.

[0027] Optionally, the data in the second storage area corresponds to a video frame, and the data in the third storage area also corresponds to a video frame. The second storage area is the Q2th storage area among the K storage areas, Q2=P2ModK, P2 is the decimal value corresponding to the bit string consisting of the x2th to y2th bits in the EPC of the A-IoT device, x2 and y2 are positive integers with different values, and the third storage area is the Q3th storage area among the K storage areas, Q3=P3ModK, P3 is the decimal value corresponding to the bit string consisting of the x3th to y3th bits in the EPC of the A-IoT device, and x3 and y3 are positive integers with different values.

[0028] Optionally, M2 second D2R messages correspond to M2 transmission opportunities, u is an integer traversing from 1 to M2-1, and the A-IoT device is configured as follows: at the u-th transmission opportunity, the A-IoT device stores the data #u collected at the u-th transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #u in the second storage area and the data #u in the third storage area from the storage area of ​​the A-IoT device according to the second read command, and carries the data #u in the second storage area and the data #u in the third storage area into the second D2R message #u and sends it; after the second D2R message #u is sent, the A-IoT device stores the data #u collected at the u-th transmission opportunity into the storage area of ​​the A-IoT device; The data #u stored in the storage area is released; thereafter, at the u+1th transmission opportunity, the A-IoT device stores the data #u+1 collected at the u+1th transmission opportunity in the storage area of ​​the A-IoT device; according to the second read command, the A-IoT device obtains the data #u+1 in the second storage area and the data #u+1 in the third storage area from the storage area of ​​the A-IoT device, and carries the data #u+1 in the second storage area and the data #u+1 in the third storage area into the second D2R message #u+1 and sends it; after the second D2R message #u+1 is sent, the A-IoT device releases the data #u+1 stored in the storage area of ​​the A-IoT device.

[0029] Optionally, the reader allocates wireless transmission resources of the A-IoT device according to the size of two video frames collected by the A-IoT device, and the size of the wireless transmission resources of the A-IoT device matches the size of any one of the M2 second D2R messages.

[0030] In a third aspect, an electronic device is provided, comprising: a processor and a memory; the memory is used to store a computer program, and when the processor executes the computer program, the electronic device executes the method described in the first aspect.

[0031] In a possible design solution, the electronic device described in the third aspect may further include a transceiver. The transceiver may be a transceiver circuit or an interface circuit. The transceiver may be used for the electronic device described in the third aspect to communicate with other electronic devices.

[0032] In an embodiment of the present invention, the electronic device described in the third aspect may be a terminal, or a chip (system) or other parts or components that may be arranged in the terminal, or a system including the terminal.

[0033] In a fourth aspect, a computer-readable storage medium is provided, comprising: a computer program or instructions; when the computer program or instructions are executed on a computer, the computer is caused to execute the method described in the first aspect.

[0034] In summary, the above method and system have the following technical effects:

[0035] When the reader sends the first read command to the A-IoT device, the A-IoT device can continuously return M1 first D2R messages according to the first read command. Each of the M1 first D2R messages carries a corresponding video frame, that is, the A-IoT device continuously reports M1 video frames to the reader. There are multiple frames between two adjacent video frames in the M1 video frames, that is, the A-IoT device reports the surveillance video frame by frame. Therefore, the reader can determine whether there is a security incident in the closed road section by analyzing the M1 video frames through the ML model, that is, through the lightweight data interaction between the A-IoT device and the reader, intelligent surveillance video anomaly analysis is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of the architecture of an abnormality analysis system for intelligent surveillance video provided by an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of a process flow of an abnormality analysis method for intelligent surveillance video provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention will be described below in conjunction with the accompanying drawings.

[0040] In the embodiment of the present invention, "indication" may include direct indication and indirect indication, and may also include explicit indication and implicit indication. The information indicated by a certain information is called information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, the information to be indicated can be directly indicated, such as the information to be indicated itself or the index of the information to be indicated. The information to be indicated can also be indirectly indicated by indicating other information, wherein there is an association relationship between the other information and the information to be indicated. It is also possible to indicate only a part of the information to be indicated, while the other parts of the information to be indicated are known or agreed in advance. For example, the indication of specific information can also be achieved by means of the arrangement order of each information agreed in advance (for example, specified by the protocol), thereby reducing the indication overhead to a certain extent. At the same time, the common parts of each information can also be identified and indicated uniformly to reduce the indication overhead caused by indicating the same information separately.

[0041] In addition, the specific indication method may also be various existing indication methods, such as but not limited to the above-mentioned indication methods and various combinations thereof. The specific details of the various indication methods can refer to the prior art and will not be repeated herein. As can be seen from the above, for example, when it is necessary to indicate multiple information of the same type, different indication methods may be used for different information. In the specific implementation process, the desired indication method can be selected according to specific needs. The embodiment of the present invention does not limit the selected indication method. In this way, the indication method involved in the embodiment of the present invention should be understood to cover various methods that can enable the party to be indicated to obtain the information to be indicated.

[0042] It should be understood that the information to be indicated can be sent as a whole or divided into multiple sub-information and sent separately, and the sending period and / or sending time of these sub-information can be the same or different. The specific sending method is not limited in the embodiment of the present invention. Among them, the sending period and / or sending time of these sub-information can be pre-defined, for example, pre-defined according to a protocol, or can be configured by the sending end device by sending configuration information to the receiving end device.

[0043] "Pre-definition" or "pre-configuration" can be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in the device, and the embodiments of the present invention do not limit the specific implementation method. Among them, "saving" can mean saving in one or more memories. The one or more memories can be set separately or integrated in an encoder or decoder, a processor, or an electronic device. The one or more memories can also be partially set separately and partially integrated in a decoder, a processor, or an electronic device. The type of memory can be any form of storage medium, which is not limited by the embodiments of the present invention.

[0044] The "protocol" involved in the embodiments of the present invention may refer to a protocol family in the communication field, a standard protocol with a similar protocol family frame structure, or a related protocol applied in future systems, and the embodiments of the present invention do not specifically limit this.

[0045] In the embodiments of the present invention, descriptions such as "when...", "in the case of...", "if" and "if" all mean that the device will perform corresponding processing under certain objective circumstances, but do not limit the time, nor do they require the device to perform judgment actions when implementing, nor do they mean the existence of other limitations.

[0046] In the description of the embodiments of the present invention, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the embodiments of the present invention is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, in the description of the embodiments of the present invention, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical solution of the embodiments of the present invention, in the embodiments of the present invention, the words "first" and "second" are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art will appreciate that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference. At the same time, in the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0047] The network architecture and business scenarios described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present invention are equally applicable to similar technical problems.

[0048] To facilitate understanding of the embodiments of the present invention, first Figure 1 Taking the abnormal analysis system of the intelligent monitoring video as an example, for example, Figure 1 A schematic diagram of the architecture of an intelligent surveillance video anomaly analysis system applicable to the intelligent surveillance video anomaly analysis method provided in an embodiment of the present invention.

[0049] like Figure 1 As shown, the abnormal analysis system of the intelligent monitoring video may include: a reader and an A-IoT device.

[0050] The abnormal analysis system of the intelligent surveillance video can be applied to the ambient IoT (A-IoT) scenario.

[0051] In the A-IoT scenario, A-IoT devices can be passive. They can obtain energy through solar energy, radio frequency, wind energy, hydropower or tidal energy, and there is no restriction on the way of obtaining energy. These nodes are not equipped with or rely on power devices such as batteries, but obtain energy from the environment to support data perception, transmission and distributed computing. The nodes can also store the obtained energy. Specifically, it can also be user equipment (UE), access terminal, subscriber unit (subscriber unit), user station, mobile station (MS), mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device. The terminal in the embodiments of the present application can be a mobile phone, a cellular phone, a smart phone, a tablet computer, a wireless data card, a personal digital assistant (PDA), a wireless modem, a handset, a laptop computer, a machine type communication (MTC) terminal, a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a vehicle-mounted terminal, a road side unit (RSU) with terminal function, etc. The terminal of the present application may also be a vehicle-mounted module, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit built into a vehicle as one or more components or units. Alternatively, the terminal may also be customer-premises equipment (CPE).

[0052] The reader may be an access network device, for example, the access network device may also include 5G, such as a gNB in ​​a new radio (NR) system, or one or a group of antenna panels (including multiple antenna panels) of a base station in 5G, or a network node constituting a gNB, a transmission and reception point (TRP or transmission point, TP) or a transmission measurement function (TMF), such as a central unit (CU), a distributed unit (DU), a CU-control plane (CP), a CU-user plane (UP), or a radio unit (RU), an RSU with base station function, or a wired access gateway, or a core network element of 5G, etc. Alternatively, the network device may also include: an access point (AP) in a WiFi system, a wireless relay node, a wireless backhaul node, various forms of macro base stations, micro base stations (also known as small stations), relay stations, access points, wearable devices, vehicle-mounted devices, etc.

[0053] The reader can also be a terminal device, such as a mobile phone, IoT device, handheld reader, etc. The reader uses wireless radio frequency to perform non-contact two-way data communication, and uses wireless radio frequency to read and write A-IoT devices, so as to achieve the purpose of identifying targets and exchanging data. For example, A-IoT devices can store part of the electrical energy through solar energy and other means, so that they can actively send signals of a certain frequency (also called semi-passive or semi-active A-IoT devices). After the reader receives and decodes the information, it sends it to the central information system for relevant data processing.

[0054] Figure 2 The flowchart of the method provided by the embodiment of the present invention is as follows. The abnormal analysis method of the intelligent monitoring video is applicable to the abnormal analysis system of the intelligent monitoring video, and involves the interaction between the reader and the A-IoT device. The specific process is as follows:

[0055] S201: The reader sends a first R2D message to the A-IoT device.

[0056] A-IoT devices are deployed on closed road sections to conduct video surveillance of the closed road sections.

[0057] The first R2D message includes a first read command, which may instruct to read data in a first storage area of ​​the A-IoT device. The first storage area may be an area where the A-IoT device caches the captured surveillance video, such as a cache or memory.

[0058] For example, before the reader sends the first R2D message to the A-IoT device, the method further includes:

[0059] First, when the A-IoT device is randomly connected to the reader, the reader obtains the electronic product code (EPC) of the A-IoT device reported by the A-IoT device.

[0060] Specifically, the reader can first send a paging message, carrying a mask for paging the A-IoT device. After that, the reader sends a query message or a query repetition message to indicate the random access opportunity of the A-IoT device. The query message or the query repetition message can be sent repeatedly. Each time the A-IoT device receives a query message or a query repetition message, the counter value is reduced by 1 until it is reduced to 0. The A-IoT device responds and sends RN16, which is a 16-bit random number. The reader returns ACK based on RN16, indicating that the A-IoT device can be randomly accessed, and then the A-IoT device sends its own EPC.

[0061] Secondly, the reader can obtain the device parameter information of the A-IoT device based on the EPC of the A-IoT device.

[0062] Specifically, the reader can send the EPC of the A-IoT device to a data storage network element (such as a data management function (UDM)) in the network, which is the network where the reader is located, such as the core network (CN) of a mobile communication network. The reader can correspondingly receive the device parameter information of the A-IoT device returned by the data storage network element according to the EPC of the A-IoT device. The device parameter information of the A-IoT device may be in the contract information of the A-IoT device. The device parameter information of the A-IoT device includes at least the device type of the A-IoT device, or the hardware and software information of the A-IoT device. Among them, the device type of the A-IoT device indicates that the A-IoT device is a monitoring device, and the hardware and software information of the A-IoT device includes at least the monitoring resolution of the A-IoT device, or the size of the storage area of ​​the A-IoT device.

[0063] Finally, the reader can determine the first storage area from the storage area of ​​the A-IoT device according to the device parameter information of the A-IoT device. For example, in response to the A-IoT device being a monitoring device, the reader can determine the size of a video frame collected by the A-IoT device according to the monitoring resolution of the A-IoT device, such as pre-configuring the correspondence between different monitoring resolutions and different sizes of a video frame, and then traversing the correspondence according to the monitoring resolution of the A-IoT device to determine the size of a corresponding video frame. It can be understood that for the A-IoT device, the size of a video frame it collects is also the size corresponding to its monitoring resolution, which is consistent on the network side and the device side. The reader can determine the first storage area from the storage area of ​​the A-IoT device according to the size of a video frame collected by the A-IoT device and the size of the storage area of ​​the A-IoT device, and the size of the first storage area is the same as the size of a video frame collected by the A-IoT device. For example, the size of the storage area of ​​the A-IoT device is L1, and the size of a video frame collected by the A-IoT device is L2. L1 / L2=K, which means that the storage area of ​​the A-IoT device can store K complete video frames. K is an integer greater than 1, which also means that the storage area of ​​the A-IoT device can be divided into K storage areas, and each storage area in the K storage areas is used to store a corresponding video frame. For example, the size of the storage area is 20MB, and the size of a video frame is 1.2MB. 20MB / 1.2MB=16.67, that is, the last space cannot store the complete 17th video frame, so 16.67 is rounded up to get K=16. The first storage area is the Q1th storage area among the K storage areas, Q1=P1ModK, P1 is the decimal value corresponding to the bit string composed of the x1th to y1th bits in the EPC of the A-IoT device, Mod represents the modulo operation, and x1 and y1 are positive integers with different values. For example, the bit string formed by the x1th bit to the y1th bit is 110110, and its corresponding decimal value is 54, 54mod16=6, which is the sixth storage area. It can be understood that the bit difference between the x1th bit and the y1th bit must be large enough to ensure that its corresponding decimal value is greater than K. The x1th bit to the y1th bit can be determined by the reader itself, such as randomly or according to a rule.

[0064] Optionally, the reader may also allocate wireless transmission resources to the A-IoT device based on the size of a video frame collected by the A-IoT device, and the size of the wireless transmission resources of the A-IoT device matches the size of any of the M1 first D2R messages (or is the same, or the size of the wireless transmission resources of the A-IoT device is slightly larger than the size of any of the M1 first D2R messages). The wireless transmission resources may be time-frequency resources, such as one or more resource blocks, one or more resource elements, etc. The A-IoT device may configure the wireless transmission resources of the A-IoT device to the A-IoT device, such as sending configuration information to indicate the index of a resource block or resource element in the wireless transmission resource.

[0065] S202, the reader receives M1 first D2R messages returned by the A-IoT device according to the first R2D message.

[0066] Each of the M1 first D2R messages includes a corresponding video frame, and there are M1 video frames in total. Each of the M1 video frames includes a picture of a closed road segment. There are multiple frames between two adjacent video frames in the M1 video frames. M1 is an integer greater than 1.

[0067] For example, M1 first D2R messages correspond to M1 transmission opportunities, t is an integer traversing from 1 to M1-1, and the A-IoT device is configured as follows: at the tth transmission opportunity, the A-IoT device stores the data #t collected at the tth transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area to the first D2R message #t for sending; after the first D2R message #t is sent, the A-IoT device releases the data #t stored in the storage area of ​​the A-IoT device. Taking the above K=16, and the first storage area being the sixth storage area as an example, at the first transmission opportunity (such as a certain moment), the A-IoT device stores the 16 video frames collected continuously after the start of the first transmission opportunity one by one into the 16 storage areas, and then selects a video frame in the sixth storage area, carries it into the first first D2R message and reports it to the reader.

[0068] Afterwards, at the t+1th transmission opportunity, the A-IoT device stores the data #t+1 collected at the t+1th transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t+1 in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area to the first D2R message #t+1 for sending; after the first D2R message #t+1 is sent, the A-IoT device continues to release the data #t+1 stored in the storage area of ​​the A-IoT device. Still taking the above K=16 and the first storage area being the sixth storage area as an example, at the second transmission opportunity (such as a certain moment), the A-IoT device stores the 16 video frames collected continuously after the start of the second transmission opportunity one by one into the 16 storage areas, and then continues to select a video frame in the sixth storage area, carries it into the second first D2R message and reports it to the reader, and then uses this type.

[0069] It can be understood that before the A-IoT device receives instructions from the reader, such as a new read command, or an instruction to terminate the current read command, the A-IoT device will continue to send the first D2R message. That is, the above-mentioned M1 first D2R messages are only an example. The A-IoT device continues to send after sending the M1th first D2R message, such as sending the M1+1th first D2R message.

[0070] S203, the reader analyzes M1 video frames through the ML model to determine whether there is a safety incident in the closed road section. The ML model is deployed in the reader.

[0071] It can be understood that the value of M1 is the upper limit of the number of video frames that the machine learning (ML) model can jointly analyze.

[0072] The ML model can be an existing convolutional neural network or a deep convolutional neural network, and there is no restriction on the specific model type. The ML model can realize security event recognition by analyzing the differences of the same object between two video frames, such as position differences. On this basis, the reader can combine any two video frames in M1 video frames to obtain C(M1,2) combinations.

[0073] The reader can input C(M1,2) combinations into the ML model for analysis to obtain C(M1,2) analysis results. If N analysis results among the C(M1,2) analysis results indicate that a security incident has occurred on the same target object, N is a positive integer less than or equal to M1, and the value of N is greater than a preset threshold, then it is determined that a security incident has occurred on the target object. Otherwise, it is determined that no security incident has occurred on the target object, and the target object is an object in a closed road segment. The preset threshold can be a proportional value, such as indicating 2 / 3 of C(M1,2), that is, if more than 2 / 3 of the analysis results among the C(M1,2) analysis results indicate that a security incident has occurred on the same target object, then it is determined that a security incident has occurred on the target object. Otherwise, it is determined that no security incident has occurred on the target object. The security incident can be a traffic accident, a vehicle failure, etc. In these cases, the target object (such as an accident / faulty vehicle) will stop moving, resulting in no difference in the position of the same target object in any two video frames. The ML model can determine that a security incident has occurred on the target object based on this.

[0074] For target objects where security incidents occur, the ML model can also extract images of the target objects from video frames.

[0075] Optionally, if a security incident occurs in the target object, the method further includes:

[0076] S1: The reader may send a second R2D message to the A-IoT device. The second R2D message includes a second read command, and the second read command indicates to read data in the second storage area and the third storage area of ​​the A-IoT device.

[0077] Among them, similar to the first storage area mentioned above, the data in the second storage area is a corresponding video frame, and the data in the third storage area is also a corresponding video frame. The second storage area is the Q2 storage area among the K storage areas, Q2=P2ModK, P2 is the decimal value corresponding to the bit string composed of the x2th to y2th bits in the EPC of the A-IoT device, x2 and y2 are positive integers with different values, and the third storage area is the Q3 storage area among the K storage areas, Q3=P3ModK, P3 is the decimal value corresponding to the bit string composed of the x3th to y3th bits in the EPC of the A-IoT device, x3 and y3 are positive integers with different values. For details, please refer to the relevant introduction of the first storage area mentioned above, which will not be repeated. It can be understood that the value of the bit string composed of the x3th to y3th bits needs to ensure that the calculated Q2 and Q3 are different. If they are the same, the reader can re-determine the bit string composed of the x3th to y3th bits by itself.

[0078] Similarly, at the uth transmission opportunity, the A-IoT device stores the data #u collected at the uth transmission opportunity in the storage area of ​​the A-IoT device; the A-IoT device obtains the data #u in the second storage area and the data #u in the third storage area from the storage area of ​​the A-IoT device according to the second read command, and carries the data #u in the second storage area and the data #u in the third storage area into the second D2R message #u and sends it; after the second D2R message #u is sent, the A-IoT device releases the data #u stored in the storage area of ​​the A-IoT device. Taking the above K=16, the second storage area is the 6th storage area as an example, and the third storage area is the 9th storage area as an example, the A-IoT device will store the 16 video frames continuously collected after the start of the uth transmission opportunity one by one in the 16 storage areas at the uth transmission opportunity (such as a certain moment), and then select a video frame in the 6th storage area, recorded as the 6th frame, and select a video frame in the 9th storage area, recorded as the 9th frame, and carry the 6th frame and the 9th frame to the uth second D2R message to report to the reader.

[0079] Afterwards, at the u+1th transmission opportunity, the A-IoT device stores the data #u+1 collected at the u+1th transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #u+1 in the second storage area and the data #u+1 in the third storage area from the storage area of ​​the A-IoT device according to the second read command, and carries the data #u+1 in the second storage area and the data #u+1 in the third storage area into the second D2R message #u+1 and sends it; after the second D2R message #u+1 is sent, the A-IoT device releases the data #u+1 stored in the storage area of ​​the A-IoT device. Taking the above K=16, the second storage area is the 6th storage area as an example, and the third storage area is the 9th storage area as an example, the A-IoT device will store the 16 video frames continuously collected after the start of the u+1th transmission opportunity one by one in the 16 storage areas at the uth transmission opportunity (such as a certain moment), and then select a video frame in the 6th storage area, recorded as the 6th frame, and select a video frame in the 9th storage area, recorded as the 9th frame, and carry the 6th frame and the 9th frame to the u+1th second D2R message to report to the reader.

[0080] S2: The reader receives M2 second D2R messages returned by the A-IoT device based on the second R2D message. Each of the M2 second D2R messages contains two corresponding video frames, for a total of 2*M2 video frames. Each of the 2*M2 video frames contains a picture of a closed road segment. There are multiple frames between two adjacent video frames in the 2*M2 video frames.

[0081] Optionally, since the data volume of the second D2R message is different from that of the first D2R message, the reader may also allocate wireless transmission resources of the A-IoT device according to the size of the two video frames collected by the A-IoT device, and configure them to the A-IoT device in a reconfiguration manner to refine its originally configured wireless transmission resources. The size of the wireless transmission resources of the A-IoT device matches the size of any one of the M2 second D2R messages.

[0082] S3: The reader analyzes the images of the target object in the M2 video frames and the M1 video frames through the ML model to track the target object in the M2 video frames.

[0083] In summary, when the reader sends the first read command to the A-IoT device, the A-IoT device can continuously return M1 first D2R messages according to the first read command, and each of the M1 first D2R messages carries a corresponding video frame, that is, the A-IoT device continuously reports M1 video frames to the reader, and there are multiple frames between two adjacent video frames in the M1 video frames, that is, the A-IoT device reports the surveillance video frame by frame. Therefore, the reader can determine whether there is a security incident in the closed road section by analyzing the M1 video frames through the ML model, that is, through the lightweight data interaction between the A-IoT device and the reader, intelligent surveillance video anomaly analysis is realized.

[0084] Combination of the above Figure 2 The method provided by the embodiment of the present invention is described in detail. The following is a detailed description of an abnormal analysis system of an intelligent monitoring video for executing the method provided by the embodiment of the present invention.

[0085] The system includes a reader, which is configured as follows: the reader sends a first R2D message to an A-IoT device, the A-IoT device is deployed in a closed road segment, the first R2D message includes a first read command, and the first read command indicates reading data in a first storage area of ​​the A-IoT device; the reader receives M first D2R messages returned by the A-IoT device according to the first R2D message, each of the M first D2R messages includes a corresponding video frame, and there are M video frames in total, each of the M video frames includes a picture of the closed road segment, and multiple frames are spaced between two adjacent video frames in the M video frames; the reader analyzes the M video frames through an ML model to determine whether there is a security incident in the closed road segment, and the ML model is deployed in the reader.

[0086] The reader sends a first R2D message to the A-IoT device, and the A-IoT device is deployed in a closed road section. The first R2D message includes a first read command, and the first read command indicates to read data in a first storage area of ​​the A-IoT device; the reader receives M1 first D2R messages returned by the A-IoT device according to the first R2D message, and each of the M1 first D2R messages includes a corresponding video frame, and there are M1 video frames in total. Each video frame in the M1 video frames includes a picture of the closed road section, and there are multiple frames between two adjacent video frames in the M1 video frames; the reader analyzes the M1 video frames through the ML model to determine whether there is a security incident in the closed road section, and the ML model is deployed in the reader.

[0087] Optionally, before the reader sends the first R2D message to the A-IoT device, the system is further configured as follows: during the process of the A-IoT device randomly accessing the reader, the reader obtains the EPC of the A-IoT device reported by the A-IoT device; the reader obtains device parameter information of the A-IoT device based on the EPC of the A-IoT device; the reader determines a first storage area from the storage area of ​​the A-IoT device based on the device parameter information.

[0088] Optionally, the reader obtains device parameter information of the A-IoT device according to the EPC of the A-IoT device, including: the reader sends the EPC of the A-IoT device to a data storage network element in the network, and the network is the network where the reader is located; the reader receives the device parameter information of the A-IoT device returned by the data storage network element according to the EPC of the A-IoT device, the device parameter information of the A-IoT device includes at least the device type of the A-IoT device, or the hardware and software information of the A-IoT device, the device type of the A-IoT device indicates that the A-IoT device is a monitoring device, and the hardware and software information of the A-IoT device includes at least the monitoring resolution of the A-IoT device, or the size of the storage area of ​​the A-IoT device; the reader determines a first storage area from the storage area of ​​the A-IoT device according to the device parameter information, including: in response to the A-IoT device being a monitoring device, the reader determines the size of a video frame collected by the A-IoT device according to the monitoring resolution of the A-IoT device; the reader determines the first storage area from the storage area of ​​the A-IoT device according to the size of a video frame collected by the A-IoT device and the size of the storage area of ​​the A-IoT device, and the size of the first storage area is the same as the size of a video frame collected by the A-IoT device.

[0089] Optionally, the size of the storage area of ​​the A-IoT device is L1, and the size of a video frame collected by the A-IoT device is L2, L1 / L2=K, indicating that the storage area of ​​the A-IoT device can store K complete video frames, K is an integer greater than 1, and also indicates that the storage area of ​​the A-IoT device can be divided into K storage areas, each of the K storage areas is used to store a corresponding video frame, the first storage area is the Q1th storage area among the K storage areas, Q1=P1ModK, P1 is the decimal value corresponding to the bit string consisting of the x1th to y1th bits in the EPC of the A-IoT device, Mod represents modulo operation, and x1 and y1 are positive integers with different values.

[0090] Optionally, the reader allocates wireless transmission resources of the A-IoT device according to the size of a video frame collected by the A-IoT device, and the size of the wireless transmission resources of the A-IoT device matches the size of any one of the M1 first D2R messages.

[0091] Optionally, M1 first D2R messages correspond to M1 transmission opportunities, t is an integer traversing from 1 to M1-1, and the A-IoT device is configured as follows: at the tth transmission opportunity, the A-IoT device stores the data #t collected at the tth transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area into the first D2R message #t and sends it; after the first D2R message #t is sent, the A-IoT device sends the A-IoT device The data #t stored in the storage area is released; thereafter, at the t+1th transmission timing, the A-IoT device stores the data #t+1 collected at the t+1th transmission timing in the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t+1 in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area to the first D2R message #t+1 and sends it; after the first D2R message #t+1 is sent, the A-IoT device continues to release the data #t+1 stored in the storage area of ​​the A-IoT device.

[0092] Optionally, the value of M1 is the upper limit of the number of video frames that the ML model can jointly analyze, and the reader analyzes M1 video frames through the ML model to determine whether there is a security incident in the closed road segment, including: the reader combines any two of the M1 video frames to obtain C(M1,2) combinations; the reader inputs the C(M1,2) combinations into the ML model for analysis to obtain C(M1,2) analysis results. If N of the C(M1,2) analysis results indicate that a security incident has occurred on the same target object, N is a positive integer less than or equal to M1, and the value of N is greater than a preset threshold, then it is determined that a security incident has occurred on the target object; otherwise, it is determined that no security incident has occurred on the object, and the target object is an object in the closed road segment.

[0093] Optionally, if a security incident occurs in the target object, the system is further configured as follows: the reader sends a second R2D message to the A-IoT device, the first R2D message includes a second read command, and the second read command indicates reading data in the second storage area and the third storage area of ​​the A-IoT device; the reader receives M2 second D2R messages returned by the A-IoT device based on the second R2D message, each of the M2 second D2R messages includes two corresponding video frames, for a total of 2*M2 video frames, each of the 2*M2 video frames includes a picture of a closed road segment, and there are multiple frames between two adjacent video frames in the 2*M2 video frames; the reader analyzes the pictures of the target object in the M2 video frames and the M1 video frames through the ML model to track the target object in the M2 video frames.

[0094] Optionally, the data in the second storage area corresponds to a video frame, and the data in the third storage area also corresponds to a video frame. The second storage area is the Q2th storage area among the K storage areas, Q2=P2ModK, P2 is the decimal value corresponding to the bit string consisting of the x2th to y2th bits in the EPC of the A-IoT device, x2 and y2 are positive integers with different values, and the third storage area is the Q3th storage area among the K storage areas, Q3=P3ModK, P3 is the decimal value corresponding to the bit string consisting of the x3th to y3th bits in the EPC of the A-IoT device, and x3 and y3 are positive integers with different values.

[0095] Optionally, M2 second D2R messages correspond to M2 transmission opportunities, u is an integer traversing from 1 to M2-1, and the A-IoT device is configured as follows: at the u-th transmission opportunity, the A-IoT device stores the data #u collected at the u-th transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #u in the second storage area and the data #u in the third storage area from the storage area of ​​the A-IoT device according to the second read command, and carries the data #u in the second storage area and the data #u in the third storage area into the second D2R message #u and sends it; after the second D2R message #u is sent, the A-IoT device stores the data #u collected at the u-th transmission opportunity into the storage area of ​​the A-IoT device; The data #u stored in the storage area is released; thereafter, at the u+1th transmission opportunity, the A-IoT device stores the data #u+1 collected at the u+1th transmission opportunity in the storage area of ​​the A-IoT device; according to the second read command, the A-IoT device obtains the data #u+1 in the second storage area and the data #u+1 in the third storage area from the storage area of ​​the A-IoT device, and carries the data #u+1 in the second storage area and the data #u+1 in the third storage area into the second D2R message #u+1 and sends it; after the second D2R message #u+1 is sent, the A-IoT device releases the data #u+1 stored in the storage area of ​​the A-IoT device.

[0096] Optionally, the reader allocates wireless transmission resources of the A-IoT device according to the size of two video frames collected by the A-IoT device, and the size of the wireless transmission resources of the A-IoT device matches the size of any one of the M2 second D2R messages.

[0097] Figure 3 The schematic diagram of the structure of the electronic device provided by the embodiment of the present invention. For example, the electronic device may be a network device, or a chip (system) or other components or assemblies that can be set in the network device. Figure 3 As shown, the electronic device 400 may include a processor 401. Optionally, the electronic device 400 may further include a memory 402 and / or a transceiver 403. The processor 401 is coupled with the memory 402 and the transceiver 403, such as being connected via a communication bus.

[0098] Combine the following Figure 3 The components of the electronic device 400 are described in detail:

[0099] The processor 401 is the control center of the electronic device 400, and may be a processor or a general term for multiple processing elements. For example, the processor 401 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0100] Optionally, the processor 401 can execute various functions of the electronic device 400 by running or executing the software program stored in the memory 402 and calling the data stored in the memory 402, such as executing the above Figure 2 The abnormality analysis method of the intelligent surveillance video is shown.

[0101] In a specific implementation, as an embodiment, the processor 401 may include one or more CPUs, such as Figure 3 CPU0 and CPU1 are shown in FIG.

[0102] In a specific implementation, as an embodiment, the electronic device 400 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0103] The memory 402 is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor 401. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0104] Optionally, the memory 402 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 402 may be integrated with the processor 401, or may exist independently and accessed through the interface circuit ( Figure 3 The processor 401 is coupled to the processor 401 (not shown), which is not specifically limited in this embodiment of the present invention.

[0105] The transceiver 403 is used for communication with other electronic devices. For example, if the electronic device 400 is a terminal, the transceiver 403 can be used to communicate with a network device, or with another terminal device. For another example, if the electronic device 400 is a network device, the transceiver 403 can be used to communicate with a terminal, or with another network device.

[0106] Optionally, the transceiver 403 may include a receiver and a transmitter ( Figure 3 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0107] Optionally, the transceiver 403 may be integrated with the processor 401, or may exist independently and communicate with the electronic device 400 through an interface circuit ( Figure 3 The processor 401 is coupled to the processor 401 (not shown), which is not specifically limited in this embodiment of the present invention.

[0108] Understandably, Figure 3 The structure of the electronic device 400 shown in the figure does not constitute a limitation on the electronic device, and the actual electronic device may include more or fewer components than those shown in the figure, or combine certain components, or arrange the components differently.

[0109] In addition, the technical effects of the electronic device 400 can refer to the technical effects of the methods described in the above method embodiments, which will not be repeated here.

[0110] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0111] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0112] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0113] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0114] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0115] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0118] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0119] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0120] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0121] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0122] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for abnormal analysis of intelligent monitoring video, characterized in that: Applied to a reader, the method comprises: The reader sends a first R2D message to the A-IoT device, where the A-IoT device is deployed in a closed road segment, and the first R2D message includes a first read command, where the first read command indicates reading data in a first storage area of ​​the A-IoT device; The reader receives M1 first D2R messages returned by the A-IoT device according to the first R2D message, each of the M1 first D2R messages includes a corresponding video frame, and there are M1 video frames in total, each of the M1 video frames includes a picture of the closed road segment, and there are multiple frames between two adjacent video frames in the M1 video frames; The reader analyzes the M1 video frames by using an ML model to determine whether there is a safety incident in the closed road segment, and the ML model is deployed in the reader; Before the reader sends the first R2D message to the A-IoT device, the method further includes: In the process of the A-IoT device randomly accessing the reader, the reader obtains the EPC of the A-IoT device reported by the A-IoT device; The reader obtains device parameter information of the A-IoT device according to the EPC of the A-IoT device; The reader determines the first storage area from the storage area of ​​the A-IoT device according to the device parameter information; The reader obtains device parameter information of the A-IoT device according to the EPC of the A-IoT device, including: The reader sends the EPC of the A-IoT device to a data storage network element in a network, where the reader is located; The reader receives the device parameter information of the A-IoT device returned by the data storage network element according to the EPC of the A-IoT device, where the device parameter information of the A-IoT device includes at least the device type of the A-IoT device or the hardware and software information of the A-IoT device, where the device type of the A-IoT device indicates that the A-IoT device is a monitoring device, and the hardware and software information of the A-IoT device includes at least the monitoring resolution of the A-IoT device or the size of the storage area of ​​the A-IoT device; The reader determines, according to the device parameter information, the first storage area from the storage area of ​​the A-IoT device, including: In response to the A-IoT device being a monitoring device, the reader determines the size of a video frame collected by the A-IoT device according to a monitoring resolution of the A-IoT device; The reader determines the first storage area from the storage area of ​​the A-IoT device based on the size of a video frame collected by the A-IoT device and the size of the storage area of ​​the A-IoT device, and the size of the first storage area is the same as the size of a video frame collected by the A-IoT device.

2. The method according to claim 1, characterized in that The size of the storage area of ​​the A-IoT device is L1, and the size of a video frame collected by the A-IoT device is L2. L1 / L2=K, indicating that the storage area of ​​the A-IoT device can store K complete video frames, K is an integer greater than 1, and also indicates that the storage area of ​​the A-IoT device can be divided into K storage areas, each of the K storage areas is used to store a corresponding video frame, the first storage area is the Q1th storage area among the K storage areas, Q1=P1ModK, P1 is the decimal value corresponding to the bit string consisting of the x1th to y1th bits in the EPC of the A-IoT device, Mod represents modulo operation, and x1 and y1 are positive integers with different values.

3. The method according to claim 2, characterized in that The M1 first D2R messages correspond to M1 transmission opportunities, t is an integer ranging from 1 to M1-1, and the A-IoT device is configured as follows: At the tth transmission opportunity, the A-IoT device stores the data #t collected at the tth transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area into the first D2R message #t for sending; after the first D2R message #t is sent, the A-IoT device releases the data #t stored in the storage area of ​​the A-IoT device; Afterwards, at the t+1th transmission timing, the A-IoT device stores the data #t+1 collected at the t+1th transmission timing into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #t+1 in the first storage area from the storage area of ​​the A-IoT device according to the first read command, and carries the data #t in the first storage area into the first D2R message #t+1 to send; after the first D2R message #t+1 is sent, the A-IoT device continues to release the data #t+1 stored in the storage area of ​​the A-IoT device.

4. The method according to claim 3, characterized in that The value of M1 is the upper limit of the number of video frames that the ML model can jointly analyze. The reader analyzes the M1 video frames through the ML model to determine whether there is a safety incident in the closed road segment, including: The reader combines any two video frames among the M1 video frames to obtain C (M1, 2) combinations; The reader inputs the C(M1,2) combinations into the ML model for analysis, respectively, to obtain C(M1,2) analysis results. If N of the C(M1,2) analysis results indicate that a security incident has occurred on the same target object, N is a positive integer less than or equal to M1, and the value of N is greater than a preset threshold, then it is determined that a security incident has occurred on the target object; otherwise, it is determined that no security incident has occurred on the object, and the target object is an object in the closed road segment.

5. The method according to claim 4, characterized in that If a security incident occurs to the target object, the method further includes: The reader sends a second R2D message to the A-IoT device, where the second R2D message includes a second read command, where the second read command indicates reading data in a second storage area and a third storage area of ​​the A-IoT device; The reader receives M2 second D2R messages returned by the A-IoT device according to the second R2D message, each of the M2 second D2R messages includes two corresponding video frames, a total of 2*M2 video frames, each of the 2*M2 video frames includes a picture of the closed road segment, and a plurality of frames are spaced between two adjacent video frames in the 2*M2 video frames; The reader analyzes the images of the target object in the 2*M2 video frames and the M1 video frames through the ML model to track the target object in the 2*M2 video frames.

6. The method according to claim 5, characterized in that The data in the second storage area corresponds to a video frame, and the data in the third storage area also corresponds to a video frame. The second storage area is the Q2th storage area among the K storage areas, Q2=P2ModK, P2 is the decimal value corresponding to the bit string composed of the x2th to y2th bits in the EPC of the A-IoT device, x2 and y2 are positive integers with different values, and the third storage area is the Q3th storage area among the K storage areas, Q3=P3ModK, P3 is the decimal value corresponding to the bit string composed of the x3th to y3th bits in the EPC of the A-IoT device, and x3 and y3 are positive integers with different values.

7. The method according to claim 6, characterized in that The M2 second D2R messages correspond to M2 transmission opportunities, u is an integer ranging from 1 to M2-1, and the A-IoT device is configured as follows: At the uth transmission opportunity, the A-IoT device stores the data #u collected at the uth transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #u in the second storage area and the data #u in the third storage area from the storage area of ​​the A-IoT device according to the second read command, and carries the data #u in the second storage area and the data #u in the third storage area into the second D2R message #u and sends it; after the second D2R message #u is sent, the A-IoT device releases the data #u stored in the storage area of ​​the A-IoT device; Afterwards, at the u+1th transmission opportunity, the A-IoT device stores the data #u+1 collected at the u+1th transmission opportunity into the storage area of ​​the A-IoT device; the A-IoT device obtains the data #u+1 in the second storage area and the data #u+1 in the third storage area from the storage area of ​​the A-IoT device according to the second read command, and carries the data #u+1 in the second storage area and the data #u+1 in the third storage area into the second D2R message #u+1 and sends it; after the second D2R message #u+1 is sent, the A-IoT device releases the data #u+1 stored in the storage area of ​​the A-IoT device.

8. An abnormality analysis system for intelligent surveillance video, characterized in that: The system comprises a reader, and the system is configured to: The reader sends a first R2D message to the A-IoT device, where the A-IoT device is deployed in a closed road segment, and the first R2D message includes a first read command, where the first read command indicates reading data in a first storage area of ​​the A-IoT device; The reader receives M first D2R messages returned by the A-IoT device according to the first R2D message, each of the M first D2R messages includes a corresponding video frame, and there are M video frames in total, each of the M video frames includes a picture of the closed road segment, and there are multiple frames between two adjacent video frames in the M video frames; The reader analyzes the M video frames by using an ML model to determine whether there is a safety event in the closed road segment, and the ML model is deployed in the reader; Before the reader sends the first R2D message to the A-IoT device, during the process of the A-IoT device randomly accessing the reader, the reader obtains the EPC of the A-IoT device reported by the A-IoT device; The reader obtains device parameter information of the A-IoT device according to the EPC of the A-IoT device; The reader determines the first storage area from the storage area of ​​the A-IoT device according to the device parameter information; The reader obtains device parameter information of the A-IoT device according to the EPC of the A-IoT device, including: The reader sends the EPC of the A-IoT device to a data storage network element in a network, where the reader is located; The reader receives the device parameter information of the A-IoT device returned by the data storage network element according to the EPC of the A-IoT device, where the device parameter information of the A-IoT device includes at least the device type of the A-IoT device or the hardware and software information of the A-IoT device, where the device type of the A-IoT device indicates that the A-IoT device is a monitoring device, and the hardware and software information of the A-IoT device includes at least the monitoring resolution of the A-IoT device or the size of the storage area of ​​the A-IoT device; The reader determines, according to the device parameter information, the first storage area from the storage area of ​​the A-IoT device, including: In response to the A-IoT device being a monitoring device, the reader determines the size of a video frame collected by the A-IoT device according to a monitoring resolution of the A-IoT device; The reader determines the first storage area from the storage area of ​​the A-IoT device based on the size of a video frame collected by the A-IoT device and the size of the storage area of ​​the A-IoT device, and the size of the first storage area is the same as the size of a video frame collected by the A-IoT device.

Images