State information acquisition system for audio and video processing

Abstract

The invention relates to a state information acquisition system for audio and video processing. The state information acquisition system comprises an audio and video processing system, a shared memory and an audio and video monitoring system, the audio and video processing system obtains topological data and operation data of each audio and video processing node; storing the topological data to the topological region, and storing the operation data of each audio and video processing node to a corresponding parameter region; the audio and video monitoring system obtains the topological data and the storage address corresponding to each parameter area from the topological area; according to the storage address, obtaining operation data of the corresponding audio and video processing node from each parameter area; and the audio and video monitoring system determines the operation state of each audio and video processing node according to the topological data and the operation data of each audio and video processing node, so that the topological structure of the audio and video processing system and the operation state of each audio and video processing node can be automatically and quickly determined.

Description

Technical Field

[0001] The present invention relates to the technical field of audio and video processing, and in particular to a state information acquisition system for audio and video processing. Background Art

[0002] With the development of audio and video services, the number of audio and video usage scenarios is increasing, which makes audio and video processing more complex. The chain audio and video processing system is formed by multiple audio and video processing nodes connected to each other. It is easy to expand and has clear and intuitive logic. It is more and more widely used in audio and video processing. For example, the avfilter module of the open source library ffmpeg, Microsoft's directshow, and the hardware pipeline of the graphics card.

[0003] In the process of realizing the present invention, the inventors found that: with the further development of the chain audio and video processing system and its higher complexity, the monitoring of the current topology structure of the audio and video processing system and the operating status of each node can no longer be determined only by its output. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a state information acquisition system for audio and video processing, which has the advantages of automatically and quickly determining the topological structure of the audio and video processing system and the operating status of each audio and video processing node.

[0005] According to a first aspect of an embodiment of the present application, a state information acquisition system for audio and video processing is provided, including an audio and video processing system, a shared memory, and an audio and video monitoring system; the audio and video processing system includes a plurality of audio and video processing nodes, the shared memory includes a topology area and a plurality of parameter areas, the topology area stores a storage address of each parameter area in the shared memory, and each parameter area corresponds to an audio and video processing node;

[0006] The audio and video processing system obtains topology data and operation data of each audio and video processing node; stores the topology data in the topology area, and stores the operation data of each audio and video processing node in the corresponding parameter area; wherein the topology data includes node information and connection information of each audio and video processing node;

[0007] The audio and video monitoring system obtains topology data and the storage address corresponding to each parameter area from the topology area; according to the storage address, the operating data of the corresponding audio and video processing node is obtained from each parameter area;

[0008] The audio and video monitoring system determines the operating status of each audio and video processing node according to the topology data and the operating data of each audio and video processing node.

[0009] The embodiment of the present application obtains topological data and the operating data of each audio and video processing node through the audio and video processing system; stores the topological data in the topological area, and stores the operating data of each audio and video processing node in the corresponding parameter area; wherein the topological data includes the node information and connection information of each audio and video processing node; the audio and video monitoring system obtains the topological data and the storage address corresponding to each parameter area from the topological area; obtains the operating data of the corresponding audio and video processing node from each parameter area according to the storage address; the audio and video monitoring system determines the operating state of each audio and video processing node according to the topological data and the operating data of each audio and video processing node. The present application stores the topological data and the operating data of each audio and video processing node in the shared memory through the audio and video processing system, and obtains the flutter data and the operating data of each audio and video processing node from the shared memory through the audio and video monitoring system, so that the topological structure of the audio and video processing system and the operating state of each audio and video processing node can be automatically and quickly determined. In addition, the audio and video processing system is used for audio and video processing, and the audio and video monitoring system is used for audio and video monitoring, so that the audio and video processing is separated from the audio and video monitoring, which can maximize the performance of the audio and video processing system and the audio and video monitoring system.

[0010] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0011] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic block diagram of the structure of a system for acquiring status information of audio and video processing provided in one embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0014] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0015] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0016] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0017] In addition, in the description of this application, unless otherwise specified, "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0018] The inventors found in the process of implementing the present invention that as the chain audio and video processing system further develops and becomes more complex, the monitoring of the current topology of the audio and video processing system and the operating status of each node can no longer be determined solely by its output.

[0019] To this end, the present application stores the topology data and the operating data of each audio and video processing node in a shared memory through an audio and video processing system, and obtains the data and the operating data of each audio and video processing node from the shared memory through an audio and video monitoring system, so as to automatically and quickly determine the topology structure of the audio and video processing system and the operating status of each audio and video processing node.

[0020] See also Figure 1, which is a schematic block diagram of the structure of a system for obtaining status information of audio and video processing provided by an embodiment of the present application. The present application embodiment provides a system 1 for obtaining status information of audio and video processing, including an audio and video processing system 11, a shared memory 12, and an audio and video monitoring system 13; the audio and video processing system 11 includes a plurality of audio and video processing nodes, the shared memory 12 includes a topology area and a plurality of parameter areas, the topology area stores the storage address of each parameter area in the shared memory 12, and each parameter area corresponds to an audio and video processing node;

[0021] The audio and video processing system 11 obtains topology data and operation data of each audio and video processing node; stores the topology data in the topology area, and stores the operation data of each audio and video processing node in the corresponding parameter area; wherein the topology data includes node information and connection information of each audio and video processing node;

[0022] The audio and video monitoring system 13 obtains the topology data and the storage address corresponding to each parameter area from the topology area; according to the storage address, obtains the operation data of the corresponding audio and video processing node from each parameter area;

[0023] The audio and video monitoring system 13 determines the operating status of each audio and video processing node according to the topology data and the operating data of each audio and video processing node.

[0024] The audio and video processing system 11 is a chain audio and video processing system for processing audio and video signals, including but not limited to collecting, encoding, transmitting, decoding and rendering audio and video signals. Each processing process of the audio and video signal can be performed in an audio and video processing node, and each audio and video processing node is connected to each other to form a chain audio and video processing system.

[0025] The shared memory 12 is used to store the topology data of the audio and video processing system 11 and the operation data of each audio and video processing node in the audio and video processing system 11 . The shared memory 12 can be accessed by the audio and video processing system 11 and the audio and video monitoring system 13 .

[0026] The audio and video monitoring system 13 is used to monitor the topological data of the audio and video processing system 11 and the operation data of each audio and video processing node in the audio and video processing system 11 to determine the operation status of each audio and video processing node.

[0027] Among them, the topological data is used to indicate the topological structure formed by each audio and video processing node of the audio and video processing system 11. Among them, the node information of each audio and video processing node includes the node identification, node type and node parameter table of the audio and video processing node. The node identification is used to uniquely identify the audio and video processing node, which can be a letter, number or other number. The node type refers to the type to which the audio and video processing node belongs. For example, the audio and video processing node includes an acquisition node, an encoding node and a decoding node. The node parameter table refers to the operating parameter list of the audio and video processing node, and several operating parameters are arranged in order to form an operating parameter list. The operating parameters include but are not limited to stream type, stream format, frame rate, jitter number, jitter duration, packet loss rate, network delay, bit rate, stream duration, processing time, number of channels, sampling rate, sampling depth, volume, resolution, GOP, average frame interval, maximum frame interval and minimum frame interval.

[0028] The connection information of each audio and video processing node includes the connection relationship between the audio and video processing nodes, the signal transmission type and the signal transmission mode between the audio and video processing nodes.

[0029] The topology data also includes the total number of nodes and the total number of connections of the audio and video processing nodes.

[0030] For example, the audio and video processing system 11 includes a microphone acquisition node, an AAC encoding node and an MP3 multiplexer node connected in sequence. There is queue cache transmission pcm between the microphone acquisition node and the AAC encoding node, and there is no queue cache transmission aac between the AAC encoding node and the MP3 multiplexer node. The total number of nodes in the audio and video processing system 11 is 3, and the total number of connections is 2. The signal transmission type between the microphone acquisition node and the AAC encoding node is pcm transmission, and the signal transmission mode is asynchronous transmission. The signal transmission type between the AAC encoding node and the MP3 multiplexer node is aac transmission, and the signal transmission mode is synchronous transmission.

[0031] The operation data of each audio and video processing node refers to the specific values ​​of the operation parameters of each audio and video processing node. For example, the sampling rate of the microphone acquisition node is 8000, the volume is 0.5, and the bit rate of the AAC encoding node is 100 kbps.

[0032] In the embodiment of the present application, the shared memory 12 is configured with corresponding storage addresses for the topology area and each parameter area, and the audio and video processing system 11 can pre-store the storage address corresponding to the topology area and the storage address corresponding to each parameter area, and write the topology data into the topology area according to the storage address corresponding to the topology area. According to the storage address corresponding to each parameter area, the operation data of each audio and video processing node is written into the corresponding parameter area.

[0033] Optionally, the topology area stores the storage address corresponding to each parameter area. The audio and video processing system 11 can obtain the storage address corresponding to each parameter area from the topology area, and write the operating data of each audio and video processing node into the corresponding parameter area according to the storage address corresponding to each parameter area.

[0034] In the embodiment of the present application, the audio and video monitoring system 13 pre-stores the storage address corresponding to the topology area, and obtains the topology data from the topology area according to the storage address corresponding to the topology area. The topology area stores the storage address corresponding to each parameter area, and the audio and video monitoring system 13 obtains the storage address corresponding to each parameter area from the topology area, and obtains the operation data of the corresponding audio and video processing node from each parameter area according to the storage address corresponding to each parameter area.

[0035] When a new audio and video processing node is added or an existing audio and video processing node is deleted in the audio and video processing system 11, the topological data of the audio and video processing system 11 will change. During the operation of the audio and video processing system 11, the operation data of each audio and video processing node also changes in real time.

[0036] The operation status of each audio and video processing node includes but is not limited to whether the function of the audio and video processing node is normal and whether the audio and video processing node has a performance bottleneck.

[0037] In the embodiment of the present application, the audio and video monitoring system 13 can determine whether each audio and video processing node is abnormal based on the topology data and the operation data of each audio and video processing node.

[0038] By using the embodiment of the present application, the topology data and the operation data of each audio and video processing node are obtained through the audio and video processing system 11; the topology data is stored in the topology area, and the operation data of each audio and video processing node is stored in the corresponding parameter area; wherein the topology data includes the node information and connection information of each audio and video processing node; the audio and video monitoring system 13 obtains the topology data and the storage address corresponding to each parameter area from the topology area; according to the storage address, the operation data of the corresponding audio and video processing node is obtained from each parameter area; the audio and video monitoring system 13 determines the operation status of each audio and video processing node according to the topology data and the operation data of each audio and video processing node. The present application stores the topology data and the operation data of each audio and video processing node in the shared memory 12 through the audio and video processing system 11, and obtains the flutter data and the operation data of each audio and video processing node from the shared memory 12 through the audio and video monitoring system 13, so as to automatically and quickly determine the topology structure of the audio and video processing system 11 and the operation status of each audio and video processing node. Furthermore, the audio and video processing system 11 is used for audio and video processing, and the audio and video monitoring system 13 is used for audio and video monitoring, so that the audio and video processing is separated from the audio and video monitoring, which can maximize the performance of the audio and video processing system 11 and the audio and video monitoring system 13.

[0039] In an optional embodiment, the audio and video processing system 11 further includes a control module, a first acquisition control node, and a plurality of second acquisition control nodes; the control module is connected to the first acquisition control node and the plurality of audio and video processing nodes, respectively, and each second acquisition control node corresponds to an audio and video processing node. The audio and video processing system 11 obtains topological data and operating data of each audio and video processing node; the steps of storing the topological data in the topological area and storing the operating data of each audio and video processing node in the corresponding parameter area include:

[0040] S101: A control module acquires topology data, and sends the topology data to a first acquisition and control node, and the first acquisition and control node writes the topology data into a topology area.

[0041] The control module is used to record the node information and connection information of all the audio and video processing nodes in the audio and video processing system 11 .

[0042] In the embodiment of the present application, the first acquisition and control node stores the storage address of the topology area in the shared memory 12, and writes the topology data into the topology area according to the storage address of the topology area in the shared memory 12.

[0043] Optionally, the first acquisition and control node may obtain a storage address corresponding to the topology area from the shared memory 12, and write the topology data into the topology area according to the storage address corresponding to the topology area.

[0044] S102: Each second acquisition and control node obtains the operating data of the corresponding audio and video processing node, and writes the operating data of the corresponding audio and video processing node into a corresponding parameter area.

[0045] In the embodiment of the present application, each second acquisition and control node stores a storage address of a parameter area in the shared memory 12, and writes the operating data of the corresponding audio and video processing node into the parameter according to the storage address of the parameter area in the shared memory 12.

[0046] By setting a first acquisition and control node and a plurality of second acquisition and control nodes, the topology data and the operation data corresponding to each audio and video processing node can be automatically and quickly written into the shared memory 12.

[0047] In an optional embodiment, the shared memory 12 includes a synchronization area, and the synchronization area stores a topology data change state identifier. The step of storing the topology data in the topology area includes:

[0048] S11: The audio and video processing system obtains a topology data change state identifier. If the topology data change state identifier is a first topology data change state identifier, the first topology data change state identifier is changed to obtain a second topology data change state identifier, and the topology data is written into the topology area.

[0049] The topology data change state identifier is used to identify the data state of the topology area, and the data state of the topology area includes the unchanged state, the writing state, the changed state, and the reading state. The topology data change state identifier includes the first topology data change state identifier, the second topology data change state identifier, the third topology data change state identifier, and the fourth topology data change state identifier.

[0050] Specifically, the first topology data change state identifier indicates that the data state of the topology area is in the unchanged state, which means that the topology data of the topology area is empty or the topology data of the topology area has been read by the audio and video monitoring system 13. The second topology data change state identifier indicates that the data state of the topology area is in the writing state, which means that the topology data of the topology area is being written by the audio and video processing system 11. The third topology data change state identifier indicates that the data state of the topology area is in the changed state, which means that the topology data of the topology area has been written by the audio and video processing system 11. The fourth topology data change state identifier indicates that the data state of the topology area is in the reading state, which means that the topology data of the topology area is being read by the audio and video monitoring system 13.

[0051] Among them, the change of the data state of the topology area will be restored to the initial data state after one writing and one reading, so that the next writing can start. In addition, the change of the data state of the topology area is continuous, and the no change state, the writing state, the changed state and the reading state are cycled in sequence. The topological data of the topological area can only be written by the audio and video processing system 11 when the data state of the topological area is the no change state. The topological data of the topological area can only be read by the audio and video monitoring system 13 when the data state of the topological area is the changed state.

[0052] The topology data change state identifier is represented by two binary bits 0 and 1. Specifically, the first topology data change state identifier is 00, the second topology data change state identifier is 01, the third topology data change state identifier is 10, and the fourth topology data change state identifier is 11.

[0053] In the embodiment of the present application, if the audio and video processing system 11 detects that the topology data change state flag of the synchronization area is 00, the audio and video processing system 11 first changes the topology data change state flag of the synchronization area from 00 to 01, and then writes the topology data into the topology area. If the audio and video processing system 11 detects that the topology data change state flag of the synchronization area is not 00, the audio and video processing system 11 will not write the topology data into the topology area.

[0054] S12: After writing the topology data to the topology area, the audio and video processing system changes the second topology data change status identifier and obtains the third topology data change status identifier; wherein, the first topology data change status identifier indicates that the topology data in the topology area is empty or the topology data in the topology area has been read by the audio and video monitoring system, the second topology data change status identifier indicates that the topology data in the topology area is being written by the audio and video processing system, and the third topology data change status identifier indicates that the topology data in the topology area has been written by the audio and video processing system.

[0055] In the embodiment of the present application, after writing the topology data into the topology area, the audio and video processing system 11 changes the topology data change state flag of the synchronization area from 01 to 10.

[0056] By writing the topology data to the topology area of ​​the shared memory 12 when the data status in the topology area is unchanged, conflicts between writing and reading the topology data in the shared memory 12 can be avoided, and orderly data transmission between the audio and video processing system 11 and the audio and video monitoring system 13 can be achieved.

[0057] In an optional embodiment, the synchronization area stores the operating data change status identifier of each audio and video processing node. The step of storing the operating data of each audio and video processing node in the corresponding parameter area includes:

[0058] S13: The audio and video processing system obtains the operation data change status identifier of each audio and video processing node, traverses the operation data change status identifier of each audio and video processing node, and if the operation data change status identifier of the current audio and video processing node is the first operation data change status identifier, changes the first operation data change status identifier, obtains the second operation data change status identifier, and writes the operation data of the current audio and video processing node into the corresponding parameter area; wherein the current audio and video processing node is one of the audio and video processing nodes.

[0059] The operation data change state identifier of each audio and video processing node is used to identify the data state of the parameter area corresponding to each audio and video processing node, and the data state of the parameter area includes no change state, writing state, changed state and reading state. The operation data change state identifier includes a first operation data change state identifier, a second operation data change state identifier, a third operation data change state identifier and a fourth operation data change state identifier.

[0060] Specifically, the first operation data change state identifier indicates that the data state of the parameter area is in the unchanged state, which means that the operation data of the parameter area corresponding to the current audio and video processing node is empty or the operation data of the parameter area corresponding to the current audio and video processing node has been read by the audio and video monitoring system 13. The second operation data change state identifier indicates that the data state of the parameter area is in the writing state, which means that the operation data of the parameter area corresponding to the current audio and video processing node is being written by the audio and video processing system 11. The third operation data change state identifier indicates that the data state of the parameter area is in the changed state, which means that the operation data of the parameter area corresponding to the current audio and video processing node has been written by the audio and video processing system 11. The fourth operation data change state identifier indicates that the data state of the parameter area is in the reading state, which means that the operation data of the parameter area corresponding to the current audio and video processing node is being read by the audio and video monitoring system 13.

[0061] The change of the data state of the parameter area will be restored to the initial data state after one writing and one reading, so that the next writing can start. In addition, the change of the data state of the parameter area is continuous, and the no-change state, the writing state, the changed state and the reading state are cycled in sequence. The operating data of the parameter area can only be written by the audio and video processing system 11 when the data state of the parameter area is the no-change state. The operating data of the parameter area can only be read by the audio and video monitoring system 13 when the data state of the parameter area is the changed state.

[0062] Among them, two binary bits 0 and 1 are used to represent the operation data change state identifier. Specifically, the first operation data change state identifier is 00, the second operation data change state identifier is 01, the third operation data change state identifier is 10, and the fourth operation data change state identifier is 11. Considering that there are multiple audio and video processing nodes, there are also multiple corresponding parameter areas, and the data state of each parameter area is identified by 2 binary bits. For example, 32 binary bits are used to represent the data state of the parameter areas corresponding to 16 audio and video processing nodes.

[0063] In the embodiment of the present application, when the audio and video processing system 11 writes the operation data of each audio and video processing node into the corresponding parameter area, it determines whether the operation data change state identifier of each audio and video processing node is the first operation data change state identifier. If the operation data change state identifier of the audio and video processing node is the first operation data change state identifier, the first operation data change state identifier of the audio and video processing node is changed to the second operation data change state identifier, and the operation data of the audio and video processing node is written into the corresponding parameter area. If the operation data change state identifier of the audio and video processing node is not the first operation data change state identifier, the audio and video processing node is skipped, and it is determined whether the operation data of the next audio and video processing node can be written into the corresponding parameter area.

[0064] Specifically, the audio and video processing system 11 obtains the operation data change status identifier of each audio and video processing node in the synchronization area, and traverses the operation data change status identifier of each audio and video processing node. If the operation data change status identifier of the current audio and video processing node in the synchronization area is 00, the audio and video processing system 11 first changes the operation data change status identifier of the current audio and video processing node in the synchronization area from 00 to 01, and then writes the operation data of the current audio and video processing node into the corresponding parameter area.

[0065] If the operation data change status identifier of the current audio and video processing node in the synchronization area is not 00, the audio and video processing system 11 will not write the operation data of the current audio and video processing node into the corresponding parameter area. The audio and video processing system 11 determines whether the operation data change status identifier of the next audio and video processing node of the current audio and video processing node is 00. If so, the audio and video processing system 11 changes the operation data change status identifier of the next audio and video processing node of the current audio and video processing node from 00 to 01, and then writes the operation data of the next audio and video processing node of the current audio and video processing node into the corresponding parameter area.

[0066] S14: After writing the operating data of the current audio and video processing node into the corresponding parameter area, the audio and video processing system changes the second operating data change status identifier and obtains the third operating data change status identifier; wherein, the first operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node is empty or the operating data of the parameter area corresponding to the current audio and video processing node has been read by the audio and video monitoring system, the second operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node is being written by the audio and video processing system, and the third operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node has been written by the audio and video processing system.

[0067] In the embodiment of the present application, after the audio and video processing system 11 writes the operating data of the current audio and video processing node into the corresponding parameter area, the second operating data change state identifier is changed to a third operating data change state identifier.

[0068] Specifically, after the audio and video processing system 11 writes the operating data of the current audio and video processing node into the corresponding parameter area, the audio and video processing system 11 changes the operating data change state flag of the current audio and video processing node in the synchronization area from 01 to 10. Similarly, after the audio and video processing system 11 writes the operating data of the next audio and video processing node of the current audio and video processing node into the corresponding parameter area, the audio and video processing system 11 changes the operating data change state flag of the next audio and video processing node of the current audio and video processing node in the synchronization area from 01 to 10.

[0069] By writing the operating data of the audio and video processing node into the corresponding parameter area when the data status in the parameter area is unchanged, conflicts between writing and reading the operating data of the audio and video processing node in the shared memory 12 can be avoided, and orderly data transmission between the audio and video processing system 11 and the audio and video monitoring system 13 can be achieved.

[0070] In an optional embodiment, the shared memory 12 includes a synchronization area, and the synchronization area stores a topology data change state identifier. The step of obtaining topology data from the topology area includes:

[0071] S201: Obtain a topology data change state identifier. If the topology data change state identifier is a third topology data change state identifier, change the third topology data change state identifier, obtain a fourth topology data change state identifier, and read topology data from the topology area.

[0072] In the embodiment of the present application, if the topology data change state flag of the synchronization area is detected to be 10, the audio and video monitoring system 13 first changes the topology data change state flag of the synchronization area from 10 to 11, and then reads the topology data from the topology area. If the topology data change state flag of the synchronization area is not detected to be 10, the audio and video monitoring system 13 will not read the topology data from the topology area.

[0073] S202: After reading the topology data from the topology area, the audio and video monitoring system changes the fourth topology data change status identifier and obtains the first topology data change status identifier; wherein, the third topology data change status identifier indicates that the topology data in the topology area has been written by the audio and video processing system, the fourth topology data change status identifier indicates that the topology data in the topology area is being read by the audio and video monitoring system, and the first topology data change status identifier indicates that the topology data in the topology area is empty or the topology data in the topology area has been read by the audio and video monitoring system.

[0074] In the embodiment of the present application, after reading the topology data from the topology area, the audio and video monitoring system 13 changes the topology data change state flag of the synchronization area from 11 to 00.

[0075] By reading topology data from the topology area when the data state in the topology area is changed, conflicts between reading and writing topology data in the shared memory 12 can be avoided, and orderly data transmission between the audio and video processing system 11 and the audio and video monitoring system 13 can be achieved.

[0076] In an optional embodiment, the shared memory 12 includes a synchronization area, and the synchronization area stores the operation data change status identifier of each audio and video processing node. According to the storage address, the step of obtaining the operation data of the corresponding audio and video processing node from each parameter area includes:

[0077] S203: The audio and video monitoring system obtains the operation data change status identifier of each audio and video processing node, traverses the operation data change status identifier of each audio and video processing node, if the operation data change status identifier of the current audio and video processing node is the third operation data change status identifier, changes the third operation data change status identifier, obtains the fourth operation data change status identifier, and reads the operation data of the current audio and video processing node from the corresponding parameter area of ​​the current audio and video processing node according to the storage address.

[0078] In the embodiment of the present application, when the audio and video monitoring system 13 reads the operation data of each audio and video processing node from the parameter area corresponding to each audio and video processing node, it determines whether the operation data change state identifier of each audio and video processing node is the third operation data change state identifier. If the operation data change state identifier of the audio and video processing node is the third operation data change state identifier, the third operation data change state identifier of the audio and video processing node is changed to the fourth operation data change state identifier, and the operation data of the audio and video processing node is read from the parameter area corresponding to the audio and video processing node. If the operation data change state identifier of the audio and video processing node is not the third operation data change state identifier, the audio and video processing node is skipped, and it is determined whether the operation data of the next audio and video processing node can be read from the parameter area corresponding to the next audio and video processing node.

[0079] Specifically, the audio and video monitoring system 13 obtains the operation data change status identifier of each audio and video processing node in the synchronization area, and traverses the operation data change status identifier of each audio and video processing node. If the operation data change status identifier of the current audio and video processing node in the synchronization area is 10, the audio and video monitoring system 13 first changes the operation data change status identifier of the current audio and video processing node in the synchronization area from 10 to 11, and then reads the operation data of the current audio and video processing node from the corresponding parameter area of ​​the current audio and video processing node.

[0080] If the operation data change status identifier of the current audio and video processing node in the synchronization area is not 10, the audio and video monitoring system 13 will not read the operation data of the current audio and video processing node from the corresponding parameter area of ​​the current audio and video processing node. The audio and video monitoring system 13 determines whether the operation data change status identifier of the next audio and video processing node of the current audio and video processing node is 10. If so, the audio and video monitoring system 13 first changes the operation data change status identifier of the next audio and video processing node of the current audio and video processing node from 10 to 11, and then reads the operation data of the next audio and video processing node of the current audio and video processing node from the corresponding parameter area of ​​the next audio and video processing node of the current audio and video processing node.

[0081] S204: After reading the operating data of the current audio and video processing node from the parameter area corresponding to the current audio and video processing node, the audio and video monitoring system changes the fourth operating data change status identifier and obtains the first operating data change status identifier; wherein, the third operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node has been written by the audio and video processing system, the fourth operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node is being read by the audio and video monitoring system, and the first operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node is empty or the operating data of the parameter area corresponding to the current audio and video processing node has been read by the audio and video monitoring system.

[0082] In an embodiment of the present application, after reading the operating data of the current audio and video processing node from the corresponding parameter area of ​​the current audio and video processing node, the audio and video monitoring system 13 changes the third operating data change state identifier of the current audio and video processing node to the fourth operating data change state identifier.

[0083] Specifically, after reading the operation data of the current audio and video processing node from the corresponding parameter area of ​​the current audio and video processing node, the audio and video monitoring system 13 changes the operation data change status flag of the current audio and video processing node in the synchronization area from 11 to 00. Similarly, after reading the operation data of the next audio and video processing node of the current audio and video processing node from the corresponding parameter area of ​​the next audio and video processing node of the current audio and video processing node, the audio and video monitoring system 13 changes the operation data change status flag of the next audio and video processing node of the current audio and video processing node from 11 to 00.

[0084] By reading the operating data of the audio and video processing node from the corresponding parameter area of ​​the audio and video processing node when the data status of the parameter area is changed, conflicts between reading and writing the operating data of the audio and video processing node in the shared memory 12 can be avoided, and orderly data transmission between the audio and video processing system 11 and the audio and video monitoring system 13 can be achieved.

[0085] In an optional embodiment, the operation data includes audio and video stream indicator data, and the audio and video monitoring system 13 determines the operation status of each audio and video processing node according to the topology data and the operation data of each audio and video processing node, including:

[0086] S301: traverse each audio and video processing node according to the topology data, compare the audio and video stream index data of each audio and video processing node with a preset index threshold, and determine an abnormal audio and video processing node from each audio and video processing node.

[0087] Among them, audio and video stream indicator data refers to the specific values ​​of audio and video stream indicators, which include general indicators applicable to audio and video streams, indicators applicable only to audio streams, and indicators applicable only to video streams. General indicators applicable to audio and video streams include stream type, stream format, frame rate, jitter number, jitter duration, packet loss rate, network delay, bit rate, stream duration, and processing time. Indicators applicable only to audio streams include number of channels, sampling rate, sampling depth, and volume. Indicators applicable only to video streams include resolution, GOP, average frame interval, maximum frame interval, and minimum frame interval.

[0088] In one embodiment, the preset indicator threshold can be manually set according to actual needs.

[0089] In another embodiment, the audio and video processing system 11 sets an expected output value for each audio and video indicator, and the audio and video monitoring system 13 can obtain the preset indicator threshold corresponding to each audio and video stream indicator according to the expected output value. Specifically, the expected output value corresponding to each audio and video indicator can be directly used as the preset indicator threshold corresponding to each audio and video stream indicator, or a value fluctuation range can be determined according to the expected output value, and the value fluctuation range is used as the preset indicator threshold. For example, a value is added to and a value is reduced from the expected output value corresponding to each audio and video indicator to obtain the preset threshold range corresponding to each audio and video indicator, and the preset threshold range corresponding to each audio and video indicator is used as the preset indicator threshold corresponding to each audio and video stream indicator.

[0090] In an embodiment of the present application, the audio and video monitoring system 13 traverses the previous audio and video processing node of the current audio and video processing node, the current audio and video processing node, and the next audio and video processing node of the current audio and video processing node in turn according to the topological data, and obtains a number of audio and video stream indicator data of each audio and video processing node, and compares each audio and video stream indicator data of each audio and video processing node with the corresponding preset indicator threshold value to determine whether each audio and video processing node is abnormal. For example, for the frame rate, the expected output value set is 25 frames per second, then the preset indicator threshold is [25*9 / 10, 25*11 / 10], and the error is allowed to be within 1 / 10 of the expected output value. When the frame rate output by the audio and video processing node fluctuates between 24.98 frames per second and 25.02 frames per second, the audio and video processing node is determined to be normal. When the frame rate output by the audio and video processing node fluctuates between 15.2 frames per second and 30.22 frames per second, the audio and video processing node is determined to be abnormal.

[0091] By comparing the audio and video stream indicator data of each audio and video processing node with the preset indicator threshold, abnormal audio and video processing nodes can be automatically and quickly identified.

[0092] In an optional embodiment, the audio and video monitoring system 13 includes a display module, which displays the topological data of the audio and video processing system 11 and the operation data of each audio and video processing node through a visual interface.

[0093] The display module may be a display screen, and the visualization interface may be a graphical user interface.

[0094] In an embodiment of the present application, after the audio and video monitoring system 13 obtains the topological data of the audio and video processing system 11 and the operating data of each audio and video processing node, the topological data of the audio and video processing system 11 and the operating data of each audio and video processing node can be presented in a visual interface, so that the user can intuitively understand the status information of the audio and video processing system 11.

[0095] In an optional embodiment, the audio and video monitoring system 13 includes a log module, which records the operation data of the abnormal audio and video processing nodes into a log.

[0096] In an embodiment of the present application, the log module records the operating data of abnormal audio and video processing nodes detected with low performance, low quality, processing failure, etc. into the log, thereby facilitating the troubleshooting of abnormal audio and video processing nodes.

[0097] In an optional embodiment, the audio and video monitoring system 13 includes a cloud reporting module, which uploads the topological data of the audio and video processing system 11 and the operating data of each audio and video processing node to the cloud server for processing, analysis and storage.

[0098] In the embodiment of the present application, the cloud reporting module uploads the topological data of the audio and video processing system 11 and the operating data of each audio and video processing node to the cloud server, and the cloud server processes, analyzes and stores the topological data and the operating data, which can reduce the data processing burden of the audio and video monitoring system 13. At the same time, the cloud server can publish the topological data of the audio and video processing system 11 and the operating data of each audio and video processing node to a web page for display, so that users can view the status information of the audio and video processing system 11.

[0099] Optionally, the audio and video monitoring system 13 also includes a database and a feedback module. The database is used to store the topological data of the audio and video processing system 11 and the operating data of each audio and video processing node. The feedback module is used to send a feedback instruction to the audio and video processing system 11 after determining the operating status of each audio and video processing node, so as to adjust the topological structure of the audio and video processing system 11 and the operating parameters of the audio and video processing node, so as to improve the performance of the audio and video processing node and the output quality of the audio and video stream.

[0100] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0101] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A system for acquiring status information of audio and video processing, characterized in that: It includes an audio and video processing system, a shared memory and an audio and video monitoring system; the audio and video processing system includes a plurality of audio and video processing nodes, the shared memory includes a topology area and a plurality of parameter areas, the topology area stores the storage address of each parameter area in the shared memory, and each parameter area corresponds to an audio and video processing node; The audio and video processing system acquires topology data and operation data of each of the audio and video processing nodes; The topology data is stored in the topology area, and the operation data of each audio and video processing node is stored in the corresponding parameter area; wherein the topology data includes node information and connection information of each audio and video processing node; The audio and video monitoring system obtains the topology data and the storage address corresponding to each of the parameter areas from the topology area; and obtains the operation data of the corresponding audio and video processing node from each of the parameter areas according to the storage address; The audio and video monitoring system determines the operating status of each audio and video processing node according to the topology data and the operating data of each audio and video processing node.

2. The system for acquiring state information of audio and video processing according to claim 1, characterized in that: The shared memory includes a synchronization area, and the synchronization area stores a topology data change state identifier; The step of storing the topological data in the topological area comprises: The audio and video processing system obtains the topology data change state identifier, and if the topology data change state identifier is the first topology data change state identifier, changes the first topology data change state identifier, obtains the second topology data change state identifier, and writes the topology data into the topology area; After writing the topology data to the topology area, the audio and video processing system changes the second topology data change status identifier and obtains a third topology data change status identifier; wherein, the first topology data change status identifier indicates that the topology data of the topology area is empty or the topology data of the topology area has been read by the audio and video monitoring system, the second topology data change status identifier indicates that the topology data of the topology area is being written by the audio and video processing system, and the third topology data change status identifier indicates that the topology data of the topology area has been written by the audio and video processing system.

3. The state information acquisition system for audio and video processing according to claim 1, characterized in that: The synchronization area stores the running data change status identifier of each audio and video processing node; The step of storing the operating data of each of the audio and video processing nodes in the corresponding parameter area includes: The audio and video processing system obtains the operation data change state identifier of each audio and video processing node, traverses the operation data change state identifier of each audio and video processing node, and if the operation data change state identifier of the current audio and video processing node is the first operation data change state identifier, changes the first operation data change state identifier, obtains the second operation data change state identifier, and writes the operation data of the current audio and video processing node into the corresponding parameter area; wherein the current audio and video processing node is one of the audio and video processing nodes; After writing the operating data of the current audio and video processing node into the corresponding parameter area, the audio and video processing system changes the second operating data change status identifier and obtains the third operating data change status identifier; wherein, the first operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node is empty or the operating data of the parameter area corresponding to the current audio and video processing node has been read by the audio and video monitoring system, the second operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node is being written by the audio and video processing system, and the third operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node has been written by the audio and video processing system.

4. The state information acquisition system for audio and video processing according to claim 1, characterized in that: The shared memory includes a synchronization area, and the synchronization area stores a topology data change state identifier; The step of the audio and video monitoring system acquiring the topology data from the topology area includes: The audio and video monitoring system obtains the topology data change state identifier, and if the topology data change state identifier is the third topology data change state identifier, changes the third topology data change state identifier, obtains the fourth topology data change state identifier, and reads the topology data from the topology area; After reading the topology data from the topology area, the audio and video monitoring system changes the fourth topology data change status identifier to obtain the first topology data change status identifier; wherein, the third topology data change status identifier indicates that the topology data of the topology area has been written by the audio and video processing system, the fourth topology data change status identifier indicates that the topology data of the topology area is being read by the audio and video monitoring system, and the first topology data change status identifier indicates that the topology data of the topology area is empty or the topology data of the topology area has been read by the audio and video monitoring system.

5. The state information acquisition system for audio and video processing according to claim 1, characterized in that: The synchronization area stores the running data change status identifier of each audio and video processing node; The step of acquiring the operation data of the corresponding audio and video processing node from each parameter area according to the storage address comprises: The audio and video monitoring system obtains the operation data change state identifier of each of the audio and video processing nodes, traverses the operation data change state identifier of each of the audio and video processing nodes, and if the operation data change state identifier of the current audio and video processing node is the third operation data change state identifier, changes the third operation data change state identifier to obtain a fourth operation data change state identifier, and reads the operation data of the current audio and video processing node from the parameter area corresponding to the current audio and video processing node according to the storage address; After reading the operating data of the current audio and video processing node from the parameter area corresponding to the current audio and video processing node, the audio and video monitoring system changes the fourth operating data change status identifier to obtain the first operating data change status identifier; wherein, the third operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node has been written by the audio and video processing system, the fourth operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node is being read by the audio and video monitoring system, and the first operating data change status identifier indicates that the operating data of the parameter area corresponding to the current audio and video processing node is empty or the operating data of the parameter area corresponding to the current audio and video processing node has been read by the audio and video monitoring system.

6. The system for acquiring state information of audio and video processing according to any one of claims 1 to 5, characterized in that: The operation data includes audio and video stream indicator data; The step of the audio and video monitoring system determining the operating state of each of the audio and video processing nodes according to the topology data and the operating data of each of the audio and video processing nodes comprises: The audio and video monitoring system traverses each of the audio and video processing nodes according to the topology data, compares the audio and video stream indicator data of each of the audio and video processing nodes with a preset indicator threshold, and determines abnormal audio and video processing nodes from each of the audio and video processing nodes.

7. The system for acquiring state information of audio and video processing according to any one of claims 1 to 5, characterized in that: The audio and video processing system also includes a control module, a first acquisition and control node, and a plurality of second acquisition and control nodes; The control module is respectively connected to the first acquisition and control node and a plurality of the audio and video processing nodes, and each of the second acquisition and control nodes corresponds to one of the audio and video processing nodes; The audio and video processing system acquires topology data and operation data of each of the audio and video processing nodes; The step of storing the topology data in the topology area and storing the operation data of each audio and video processing node in the corresponding parameter area includes: The control module acquires the topology data, and sends the topology data to the first acquisition and regulation node, and the first acquisition and regulation node writes the topology data into the topology area; Each of the second acquisition and control nodes obtains the operating data of the corresponding audio and video processing node, and writes the operating data of the corresponding audio and video processing node into the corresponding parameter area.

8. The system for acquiring state information of audio and video processing according to any one of claims 1 to 5, characterized in that: The audio and video monitoring system comprises a display module, and the display module displays the topological data of the audio and video processing system and the operation data of each audio and video processing node through a visual interface.

9. The system for acquiring state information of audio and video processing according to any one of claims 1 to 5, characterized in that: The audio and video monitoring system includes a log module, and the log module records the operation data of the abnormal audio and video processing node in a log.

10. The system for acquiring state information of audio and video processing according to any one of claims 1 to 5, characterized in that: The audio and video monitoring system includes a cloud reporting module, which uploads the topological data of the audio and video processing system and the operating data of each audio and video processing node to a cloud server for processing, analysis and storage.

Images