A low-delay, low-coupling multi-channel analog video signal processing system and method

Abstract

The present application relates to a low-latency, low-coupling multi-channel analog video signal processing system, method, electronic device and storage medium provided by the embodiments of the present application. Aiming at the limitations of the existing multi-channel video processing technology, the system adopts analog signal receiving board, controller, video processing development board, computing power development board, serial port receiver and other hardware and Python, Gstreamer and other software technologies, and solves the problems through signal reception, frequency point screening, video processing and transmission, intelligent recognition, custom layout and other steps. The system achieves the positive effects of centralized computing power processing, accelerated calculation, fast frequency scanning and low-latency display, user-customizable layout, convenient access to intelligent recognition module, etc., and is suitable for a variety of practical scenarios under analog video signals.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a low-latency, low-coupling multi-channel analog video signal processing system, method, computer device and computer-readable storage medium. Background Art

[0002] In today's era of rapid technological development, video processing technology has become a vital component in many industries. Whether in the construction of smart cities, video processing technology is used to achieve real-time monitoring and intelligent management of every corner of the city; or in the field of security monitoring, it plays an indispensable role as an important means to ensure social security; or in the emerging low-altitude economic field, video processing technology is also widely used, such as monitoring the flight status of low-altitude aircraft. In these industries, we can clearly see the presence of video processing technology.

[0003] In practical applications, efficient processing and low-latency transmission of video data are undoubtedly the key factors that determine the application effect. They are directly related to the timeliness and accuracy of information, and are of great significance for ensuring the smooth progress of various tasks. However, the existing multi-channel video processing technology still has many limitations in practical applications. Among them, the high latency problem seriously affects the real-time transmission of information, which may lead to the inability to respond in time in some emergency situations. At the same time, the poor layout flexibility also makes it difficult for the technology to be flexibly adjusted and optimized according to actual needs when facing complex and changeable application scenarios, making it difficult to fully meet the diverse needs of complex application scenarios. Summary of the invention

[0004] The embodiments of the present application provide a low-latency, low-coupling multi-channel analog video signal processing system, method, computer device and computer-readable storage medium to at least solve the problem of high latency in the actual application of multi-channel video processing technology in the related art.

[0005] In a first aspect, an embodiment of the present application provides a low-latency, low-coupling multi-channel analog video signal processing system, the system comprising: a signal receiving board, a signal controller, a computing power development board, and a video processing development board, wherein:

[0006] The signal controller is used to obtain the frequency information and RSSI value of the initial analog signal from the signal receiving board, and send it to the computing power development board for multi-threaded processing to obtain a compliant analog signal, and configure the signal frequency according to the compliant analog signal;

[0007] The signal receiving board is used to collect valid analog signals according to the signal frequency points configured by the compliant analog signals, and transmit the valid analog signals to the video processing development board;

[0008] The video processing development board is used to convert the valid analog signal into a valid digital signal, and send the valid digital signal to the computing power development board to instruct the computing power development board to use the message bus characteristics of the Gstreamer playback module to perform pipeline processing on the valid digital signal and output decoded video data.

[0009] In some of the embodiments, the computing power development board is also used to: before processing the video stream, ping the IP addresses of the signal controller, the signal receiving board and the video processing development board, and determine to remove the playback module or rebuild the playback module based on the result of the ping detection.

[0010] In some embodiments, the video processing development board is used to convert the valid analog signal into a video stream of a valid digital signal, and transmit the video stream to the computing power development board via a network;

[0011] The computing power development board is used to perform pipeline processing on the effective digital signal, decode and play the video stream through a pre-created Gstreamer playback module.

[0012] In some embodiments, the Gstreamer playback module includes a preprocessing module, a configuration module, a transmission module and a decoding module, wherein:

[0013] The preprocessing module is used to select a target decoder and a sink plug-in according to the property information of the system, and to capture the video stream using a transmission control framework established based on RTSP;

[0014] The configuration module is used to set pipeline elements of each pipeline of the Gstreamer playback module, wherein the pipeline elements are used to constrain the data transmission efficiency to be no higher than a preset delay threshold;

[0015] The transmission module is used to establish a transmission channel with the network address of the video stream sender through the UDP protocol under the transmission control framework, and receive the video stream through the transmission channel under the preset delay threshold constraint;

[0016] The decoding module is used to decode the video stream received by the transmission module through the target decoder, and transmit the decoded video data to the sink plug-in.

[0017] In some embodiments, the computing power development board further includes an image processing module, wherein:

[0018] The image processing module is used to obtain the video data from the sink plug-in, and call a pre-trained processing model based on user needs through the OpenCV framework, and, through the processing model, adopt asynchronous processing and multi-threaded processing technology to accelerate the recognition of the video data.

[0019] In some embodiments, the computing power development board further includes a rendering module, wherein:

[0020] The rendering module is used to create a class using the QT5 framework to integrate the Frame class of the QT5 framework, and configure the window attribute information of the Frame class according to the width and height values ​​of the current display device;

[0021] and, obtaining a plurality of basic layout grid templates by calculating through a layout grid algorithm, and storing the layout grid templates in a layout array;

[0022] A video window is created according to the configuration information of the video data, and the recognition results of each video are configured to the corresponding position of the display window by traversing the layout array.

[0023] In some embodiments, the computing power development board further includes a user interaction module, wherein:

[0024] The user interaction module is used to respond to the user's interaction instruction, trigger the corresponding action signal, recalculate the layout information according to the action signal, and adjust the layout format of each video window according to the recalculated layout information.

[0025] In a second aspect, an embodiment of the present application further provides a low-latency, low-coupling multi-channel analog video signal processing method, which is implemented by a signal receiving board, a signal controller, a computing power development board, and a video processing development board, including:

[0026] The signal controller obtains the frequency information and RSSI value of the initial analog signal from the signal receiving board, and sends it to the computing power development board for multi-threaded processing to obtain a compliant analog signal, and configures the signal frequency according to the compliant analog signal;

[0027] The signal receiving board is used to collect valid analog signals according to the signal frequency points configured by the compliant analog signals, and transmit the valid analog signals to the video processing development board;

[0028] The effective analog signal is converted into an effective digital signal through the video processing development board, and the effective digital signal is sent to the computing power development board to instruct the computing power development board to use the message bus characteristics of the Gstreamer playback module to perform pipeline processing on the effective digital signal and output the decoded video data.

[0029] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the second aspect above when executing the computer program.

[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the second aspect above.

[0031] Compared with the related art, the low-latency and low-coupling multi-channel analog video signal processing system provided in the embodiment of the present application adopts hardware such as analog signal receiving board, controller, video processing development board, computing power development board, serial port receiver and software technologies such as Python and Gstreamer to solve the problems through steps such as signal reception, frequency point screening, video processing and transmission, intelligent recognition, and custom layout. It achieves the positive effects of centralized computing power processing to accelerate calculation, quickly scan frequency points and display pictures with low latency, user-customizable layout, and convenient access to intelligent recognition modules, and is suitable for a variety of practical scenarios under analog video signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0033] Figure 1 is a structural block diagram of a low-latency, low-coupling multi-channel analog video signal processing system according to an embodiment of the present application;

[0034] Figure 2 It is a schematic diagram of a signal flow relationship between a signal controller, a computing power development board, and a signal receiving board according to an embodiment of the present application;

[0035] Figure 3 is a schematic diagram of a ping detection according to an embodiment of the present application;

[0036] Figure 4 It is a schematic diagram of the signal flow relationship between a computing power development board and a video processing development board according to an embodiment of the present application;

[0037] Figure 5 is a flow chart of receiving and identifying video data according to an embodiment of the present application;

[0038] Figure 6 is a flow chart of a user-defined layout interface according to an embodiment of the present application;

[0039] Figure 7 is a flow chart of a low-latency, low-coupling multi-channel analog video signal processing method according to an embodiment of the present application;

[0040] Figure 8 is a flow chart of another low-latency, low-coupling multi-channel analog video signal processing method according to an embodiment of the present application;

[0041] Fig. 9 It is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0043] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0044] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0045] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0046] In this document, it should be understood that the terms involved may be technical means or other summary technical terms used to implement a part of the present invention. For example, the terms may include:

[0047] RTSP: A control protocol for real-time streaming media transmission and playback, which can be used for on-demand and live streaming scenarios.

[0048] Gstreamer: an open source multimedia framework for building streaming media applications; a decoder is used to decode a video stream into a playable video file; a sink is an element used by Gstreamer to receive the final video data;

[0049] OpenCV - a cross-platform computer vision and machine learning software library.

[0050] QT5 - computer visualization graphics library;

[0051] UDP: A protocol for sending messages between computer programs, characterized by fast message transmission;

[0052] Frequency: The frequency of analog signal reception; it is the center frequency of the modulated signal and is the number given to the fixed frequency;

[0053] RSSI: signal strength value of analog signal;

[0054] GB28181 protocol: a national standard protocol that meets the requirements for video surveillance, networked information transmission and exchange, etc. Basic equipment such as video recorders need to implement this protocol to connect to the GB28181 upper platform.

[0055] Video processing technology has now become an indispensable part of many industries. It can be seen in many industries such as smart cities, security monitoring, and low-altitude economic fields. Efficient processing and low-latency transmission of video data are important factors that determine the application effect. However, the existing multi-channel video processing technology still has the following limitations in practical applications, making it difficult to meet the needs of complex application scenarios.

[0056] 1. High latency: For analog signals, they need to be converted into digital signals before they can be displayed on the computer system. The industry will adopt mature solutions, special video processing boards to convert analog signals into digital signals, and then transmit video information through the network. Existing multi-channel video processing technologies usually rely on hardware decoding and processing modules. Since most of the domestic communications and video transmissions are carried out through the GB28181 protocol or third-party platforms, streaming media servers will also be built to specifically handle the conversion of video streams. Due to the complexity of data transmission and decoding, the high latency problem of video streams is relatively common. In real-time application scenarios (such as drone monitoring and security monitoring), these delays will directly affect the real-time and accuracy of operations, and cannot respond quickly to emergencies. Although some technologies attempt to reduce latency by optimizing transmission protocols, they still cannot completely solve the latency problem when processing multiple videos at the same time.

[0057] 2. Inflexible layout: Because the layout of the images transmitted by the video processing board is calculated in the chip, the layout operation is rigid and lacks variability and flexibility. It is difficult for users to flexibly define the position, size and number of video windows according to specific application scenarios. This is very unfavorable for scenes with complex monitoring requirements and intensive information (such as traffic command centers and large security sites), affecting the user's operating experience and work efficiency.

[0058] 3. High coupling: For traditional analog video processing solutions, the software and hardware are highly coupled, which makes it difficult for users to flexibly adjust the system structure according to their needs. This design limits the scalability and adaptability of the system and cannot meet diverse application scenarios. For example, some systems can only process a fixed number of video inputs and cannot be adjusted according to actual needs.

[0059] 4. Difficulty in integrating intelligent modules: From traditional security monitoring to emerging drone object recognition and tracking, the market has demonstrated the need for rapid integration and deployment of recognition functions in different scenarios for video processing. However, many of them are implemented by the manufacturer's built-in hardware recognition, which limits the scalability of the system and the realization of intelligent applications.

[0060] 5. Analog signal processing: Analog signals need to be filtered after capture to obtain the correct video information. In most cases, developers will run the filtering algorithm on a single-chip microcomputer. However, when faced with a large amount of data, the microcontroller of the single-chip microcomputer cannot process the data more efficiently, which will cause obvious lag in user operations.

[0061] In view of this, in order to solve the above problems, the present application provides a low-latency and low-coupling multi-channel analog video signal processing system. Figure 1 is a structural block diagram of a low-latency, low-coupling multi-channel analog video signal processing system according to an embodiment of the present application, such as Figure 1 As shown, the system includes a signal receiving board 10, a signal controller 11, a computing power development board 12 and a video processing development board 13, wherein:

[0062] The signal controller 11 is used to obtain the frequency information and RSSI value of the initial analog signal from the signal receiving board 10, and send it to the computing power development board 12 for multi-threaded processing to obtain a valid analog signal, and configure the signal frequency according to the valid analog signal;

[0063] Specifically, the signal receiving board 10 serves as the receiving source of the analog signal and sends the acquired analog signal to the analog signal controller 11. This link is the beginning of the entire analog signal processing flow. The main function of the signal receiving board 10 is to receive external analog signals, such as signals from video sources of different channels.

[0064] Figure 2 is a schematic diagram of a signal flow relationship between a signal controller, a computing power development board, and a signal receiving board according to an embodiment of the present application, such as Figure 2 As shown, the analog signal controller 11 obtains the analog signal from the signal receiving board 10 and extracts the frequency point and RSSI value data therefrom;

[0065] Among them, frequency refers to the specific frequency point used in wireless communication. In wireless communication systems, different communication devices and networks use different frequencies to avoid interference with each other and ensure smooth communication. RSSI (Received Signal Strength Indicator) is the received signal strength indicator, which indicates the signal strength received by the receiving end; RSSI value is usually negative, and the closer it is to zero, the higher the signal strength. RSSI value can be used to measure the connection quality and signal strength of wireless communication systems, and to determine whether there are interference and other problems.

[0066] The signal controller 11 sends the frequency point and RSSI value data to the computing power development board 12 through the serial port protocol. In this process, the serial port protocol plays a reliable data transmission role here, ensuring that the data is accurately transmitted from the controller to the computing power development board 12.

[0067] Furthermore, after receiving the data, the computing power development board 12 creates multiple threads to specifically process the data. The processing process mainly includes screening the analog signal data. Optionally, the specific screening condition is that the RSSI value is greater than a certain threshold, thereby excluding analog signals with weak signal strength, possible interference or instability; at the same time, the frequency points with high probability of the same image are eliminated to avoid interference from repeated or similar signals, so as to obtain more valuable and unique analog signal data; and it must comply with specific rules, which may be based on the system's pre-set standards or algorithms to ensure that the screened analog signal data meets the overall system operation requirements. After such processing, the analog signal data that meets the conditions is returned to the analog signal controller 11.

[0068] The signal receiving board 10 is used to collect effective analog signals according to the signal frequency, and transmit the effective analog signals to the video processing development board 13 .

[0069] Among them, after receiving the valid analog signal data returned by the computing power development board 12, the analog signal controller 11 sets the corresponding analog signal frequency according to these data. It can be understood that through the system action of the computing power development board 12 and the signal controller 11, the system can focus on the valid signal, provide an accurate signal source basis for subsequent video signal processing, and ensure that the video processing process can be operated based on high-quality and accurate analog signals, thereby improving the performance and accuracy of the entire multi-channel video vision processing system.

[0070] Specifically, the analog signal controller 11 accurately sets the corresponding analog signal frequency according to the data returned after processing by the computing power development board 12. This step is a key operation after screening and optimizing the analog signal, ensuring that the analog signal subsequently processed by the system is in a suitable frequency range to obtain a more stable and effective signal.

[0071] At the same time, the signal receiving board 10 establishes connections with each analog channel of the video processing board, and simultaneously transmits the video signal containing valid information to the video processing development board 13. Among them, the signal receiving board 10 plays the role of signal switching, smoothly directing the received analog signal to the video processing development board 13, and providing original signal materials for subsequent video processing work.

[0072] In addition, in this system, a convenient frequency operation button is provided for the user in the graphical interface of the program. This design greatly enhances the user's control over the system frequency setting.

[0073] Specifically, the automatic frequency scanning function allows users to quickly scan and find the appropriate frequency point, reducing the tedious process of manually testing the frequency points one by one and improving the frequency point search efficiency. In addition to automatic frequency scanning, users can also set specific frequency bands according to their own needs. When the user performs a specific frequency band setting operation, the system will send these specific operation information to the analog signal controller 11. After receiving the operation information, the analog signal controller 11 will perform the corresponding operation according to the user's instructions, thereby realizing the user's personalized and precise control of the analog signal frequency point.

[0074] Through the synergy of the signal receiving board 10, signal controller 11 and computing power development board 12, centralized computing power processing is adopted, and parts that need to be calculated, such as frequency detection, are handed over to the computing power development board 12 with more powerful performance. Other hardware only needs to provide communication methods to speed up the calculation process of such special algorithms. There is no need to set up tests for each frequency point, and the correct frequency point can be quickly scanned and the screen can be displayed with extremely low latency.

[0075] The video processing development board 13 is used to convert the effective analog signal into an effective digital signal, and utilize the message bus characteristics of the Gstreamer playback module to perform pipeline processing on the effective digital signal and output the decoded video data.

[0076] It should be noted that before processing the video stream, the computing power development board 12 is also used to: ping the IP addresses of the signal controller 11, the signal receiving board 10 and the video processing development board 13, and determine whether to remove the playback module or rebuild the playback module based on the results of the ping detection.

[0077] Figure 3 is a schematic diagram of a ping detection according to an embodiment of the present application, such as Figure 3As shown in the figure, if the ping detection finds an error, it indicates that there is a problem with the device IP, which may cause the video stream transmission to fail or be abnormal. At this time, the system will safely remove the playback module to avoid bus congestion and thread occupation caused by the playback module running in the wrong IP environment.

[0078] Those skilled in the art understand that bus congestion will affect the data transmission efficiency of the entire system, and thread occupation may lead to system resource exhaustion, inability to process other tasks normally, and even cause system crashes. By removing the playback module in a timely manner, these problems can be effectively prevented and the stability of the system can be maintained.

[0079] In this embodiment, the system will continue to detect the IP within a reasonable time. This continuous detection mechanism ensures that the system can capture changes in the IP status in a timely manner so that it can respond quickly when the device IP returns to normal. When the IP is detected successfully, the system will rebuild the playback module. This is because the playback module was previously removed due to an IP error. In order to restore the normal playback of the video stream, an available playback module needs to be rebuilt. After the playback module is rebuilt, the correct video data will be connected to ensure that the video stream can enter the processing flow accurately. Preventing program crashes due to build errors further ensures the stability and reliability of the entire video processing system, allowing the video processing process to proceed smoothly and avoiding program exceptions caused by build errors from causing damage to the system.

[0080] After completing the above ping detection, the video processing development board 14 converts the received valid analog signal into a digital signal and, specifically, after receiving the analog signal (transmitted by the signal receiving board 10), the video processing development board 13 synchronously converts it into a digital signal. This conversion process is a key step for the video to be subsequently processed and transmitted in a digital system, converting the continuously changing analog signal into a discrete digital signal for easy recognition and processing by the computer system. The converted digital video signal is transmitted to the computing power development board 12 via the network, providing a data basis for subsequent playback and decoding operations.

[0081] Figure 4 is a schematic diagram of the signal flow relationship between a computing power development board and a video processing development board according to an embodiment of the present application, such as Figure 4 As shown, the computing power development board 12 uses the message bus characteristics of the Gstreamer playback module to process the effective digital signal in a pipeline manner and output the decoded video data. Specifically, the Gstreamer playback module includes a preprocessing module, a configuration module, a transmission module and a decoding module, wherein:

[0082] The preprocessing module is used to select the target decoder and sink plug-in according to the system attribute information, and to capture the video stream using the transmission control framework established based on RTSP;

[0083] The configuration module is used to set pipeline elements of each pipeline of the Gstreamer playback module, wherein the pipeline element is used to constrain the data transmission efficiency to be no higher than a preset delay threshold;

[0084] Among them, the setting of pipeline elements plays a key role in the process of Gstreamer processing videos in the form of pipelines. By setting pipeline elements, the behavior and parameters of the video stream at each processing stage can be accurately defined. For example, setting the delay attribute value of the pipeline element to a specific value directly affects the transmission speed and processing efficiency of the video stream in the pipeline, and reduces the residence time of the video data in the pipeline.

[0085] The transmission module is used to establish a transmission channel with the network address of the video stream sender through the UDP protocol under the transmission control framework, and receive the video stream through the transmission channel under the constraint of a preset delay threshold;

[0086] The decoding module is used to decode the video stream received by the transmission module through the target decoder and transmit the decoded video data to the sink plug-in.

[0087] In addition, it should be noted that under the technical framework of this system, Gstreamer is used for video stream playback processing in the multi-channel video visual processing technology, which can be replaced by ffmpeg. Among them, ffmpeg is a widely used multimedia processing tool, which has powerful audio and video encoding and decoding capabilities and rich format support. If ffmpeg is used instead of Gstreamer, the system can use the characteristics of ffmpeg to decode and convert video streams, and can also achieve effective processing of video streams to meet the needs of video playback and processing. It is just that the specific interface calls, processing procedures and function implementation details are different from those of using Gstreamer, but both can achieve the purpose of processing video streams.

[0088] Figure 5 is a flow chart of receiving and identifying video data according to an embodiment of the present application, such as Figure 5 As shown, the computing power development board 12 receives the video stream from the video processing board through network communication means. After receiving the video stream, the computing power development board 12 creates a Gstreamer playback module and uses the Gstreamer streaming media processing capability to process the video. And according to the differences in system platforms, the decoder and sink plug-in adapted thereto are selected to ensure the compatibility and effectiveness of video decoding and playback.

[0089] Furthermore, Gstreamer processes the video in a pipeline form. First, it captures the RTSP video stream in the network. The RTSP protocol provides a control basis for real-time streaming transmission. Then, the delay attribute value of the pipeline element is set to a specific value. This operation reduces the delay in the video stream transmission process and improves real-time performance. In addition, the video stream network address is connected through the UDP protocol. The UDP protocol, with its fast message transmission characteristics, helps to further reduce the video transmission delay. Finally, the video stream is decoded using the h264 format decoder. H264 is a widely used and efficient video encoding format that can effectively compress the data volume while ensuring video quality. The decoded video data is passed to the sink plug-in for subsequent use.

[0090] It can be understood that RTSP and UDP work together in the process of video stream processing. The RTSP protocol establishes a control framework for video stream transmission, which is responsible for processing the interaction of control instructions such as the start, pause, and end of the video stream, ensuring the orderliness and controllability of video stream transmission. The UDP protocol is responsible for quickly and efficiently transmitting the video data itself under this framework.

[0091] In this embodiment, by directly parsing the RTSP video stream and combining it with the UDP transmission protocol, the system can maintain the quality and fluency of the video while ensuring low latency. Low latency ensures the real-time nature of video playback, which is crucial for real-time application scenarios such as drone monitoring and security monitoring, allowing operators to obtain accurate video information in a timely manner; while the guarantee of video quality and fluency provides a clear and coherent visual experience, meeting the user's requirements for video processing effects, and enabling the entire multi-channel video visual processing technology to perform well in practical applications.

[0092] Furthermore, the computing power development board 12 includes an image processing module and a rendering module, wherein: the image processing module is used to obtain video data from a sink plug-in, and call a pre-trained processing model based on user needs through the OpenCV framework, and, through the processing model, use asynchronous processing and multi-threaded processing technology to accelerate the recognition of video data.

[0093] Specifically, OpenCV, as the basic library framework for image processing, provides the system with a wealth of image processing functions and algorithms. First, the system calls pre-trained model files, which contain recognition knowledge for specific objects or scenes, such as face recognition models, object recognition models, etc. According to different actual needs, different intelligent recognition modules are written based on OpenCV. These modules can use the knowledge in the model files to intelligently recognize the content in the video frame.

[0094] In addition, in order to speed up and optimize the recognition process, asynchronous and multi-threading technologies are used. Asynchronous operations allow the system to continue to perform other tasks while waiting for certain operations to complete (such as waiting for data loading or model calculation), improving the overall efficiency of the system. Multi-threading technology makes full use of the advantages of multi-core processors to distribute recognition tasks to multiple threads for parallel execution, further shortening the recognition time and enabling intelligent recognition to respond more quickly to information in the video stream.

[0095] It should also be noted that, since intelligent recognition is a hot trend at present, it is necessary to be able to quickly access or add an applicable recognition module, such as a face recognition module that needs to identify people, an object recognition module that needs to identify dangerous goods, or area tracking, etc. Through the image processing module provided in this embodiment, users can quickly access other intelligent recognition modules according to their needs, thereby improving the flexibility and applicability of the solution.

[0096] Furthermore, the computing power development board 12 also includes a rendering module, wherein: the rendering module is used to use the QT5 framework to create a class that integrates the Frame class of the QT5 framework, and configure the window attribute information of the Frame class according to the width and height values ​​of the current display device; and calculate multiple basic layout grid templates through a layout grid algorithm, and store the layout grid templates in a layout array; create a video window according to the configuration information of the video data, and configure each video window to the corresponding position of the main window by traversing the layout array.

[0097] Furthermore, the user interaction module is used to respond to the user's interaction instruction, trigger the corresponding action signal, recalculate the layout information according to the action signal, and adjust the layout format of each video window according to the recalculated layout information.

[0098] Specifically, Figure 6 is a flow chart of a user-defined layout interface according to an embodiment of the present application, such as Figure 6 As shown in the figure, using the QT5 framework, first create a class and make it inherit the Frame class of QT5, set its width and height values ​​to the width and height values ​​of the current device, and ensure that the video window can adapt to the device screen size and provide the best display effect. Then, several basic layout grid results are calculated according to the layout grid algorithm, and these results are stored in the array. These basic layout grids provide a basic template for the subsequent creation of diverse video window layouts. Then, video windows are created according to the video configuration file. These video windows are the areas actually used to display video content. After the creation is completed, the video windows are stored in the video window array to prepare for the subsequent unified management and layout operations. After that, by traversing the layout array, each video window is accurately set to the corresponding position of the main window to complete the preliminary layout construction of the video window.

[0099] It should be noted that an array is a data structure that can store multiple data elements of the same type in a certain order. For example, in the layout management part, the basic layout grid results are stored in an array to facilitate the unified management and calling of multiple possible layout templates. These layout grid results may contain layout-related information such as the position and size of the window, which are stored in an array in an orderly manner, making it easier to quickly select and apply the corresponding layout template based on user needs or system configuration.

[0100] In the custom layout function, the layout grid results edited by the user (such as the layout determined by the user by dragging the zoom effect window and setting related parameters) are stored in the array to save the user-defined layout configuration. The array acts as a repository for layout configurations. Each element represents a user-defined layout scheme. Different layouts can be distinguished and indexed by serial numbers and other methods.

[0101] In addition, override the click method inherited by the new class, and trigger the corresponding operation by accurately determining the state of click and release and the time taken for the operation. "Override the click method inherited by the new class" means redefining (overriding) the click operation-related methods inherited from the parent class in the subclass (a newly created class that inherits from other classes).

[0102] For example, when a user clicks on a video window, functions such as pausing, playing the video, or popping up a related operation menu can be realized; the time-consuming determination of the operation process can be used to distinguish different types of click operations, such as short clicks and long clicks may trigger different responses. When the user drags the window to adjust it, the program will reassign the layout coordinates to the dragged window based on the coordinate information of the user's last leaving the screen, so that the window can be moved to the user's desired position. The layout information is then recalculated to ensure that all video windows are reasonably arranged and displayed under the new layout. After the calculation is completed, the new layout information is stored in the memory so that the user-defined layout state can be quickly restored the next time it is called, providing a good user operation experience and layout memory function.

[0103] In addition, after opening the custom layout interface of the program, users can flexibly set the number, size, and position of video windows according to their own needs. Users can arrange video windows through intuitive drag and drop, and create special layouts that meet the needs of specific scenarios. In the custom layout interface, you can also create effect windows, which are only used to display layout effects and do not actually play videos. Users edit effect windows by dragging and dropping, setting the effect window sequence number, and clicking Save. The system will store the current layout grid results in the array. When the user clicks to switch the layout, the program will read the layout results stored in the array, and traverse all video windows again, and re-layout each video window based on the stored layout information, realizing the flexible switching and management of user-defined layouts, meeting the diverse needs of different users for video window layouts in different scenarios, and improving the flexibility and adaptability of the system.

[0104] In addition, under the technical framework of the present application, QT5, as a computer visualization graphics library, can be replaced by any visualization graphics library, or replaced by a kiosk-style embedded web application (developed using any desktop web application development framework such as react native, tauri, etc.). The replacement of the visualization graphics library is mainly based on its ability to provide the function of creating a graphical user interface. As long as the new graphics library can meet the system's requirements for video window creation, layout management, user interaction response (such as clicks, drags, etc.), QT5 can be replaced. When using a kiosk-style embedded web application as a replacement, the video is obtained and converted from the video processing development board by deploying a webrtc streaming media server, and the web application uses the webrtc protocol to pull the video stream. This method can also achieve the effect of playing low-latency video. It takes advantage of the cross-platform nature and flexibility of web technology, and while ensuring low-latency video playback, it provides another implementation method for the system's interface presentation and user interaction, expanding the deployment possibilities of the system on different platforms and application scenarios.

[0105] Through the above system, the video stream is played through Gstreamer, and its message bus can be used to obtain and manage the playback status in real time, set the network communication method (such as using the UDP protocol) and element attributes (such as specific delay values), effectively reduce the video transmission and processing delay, ensure the fast presentation of the video, and meet the application scenarios with high real-time requirements. In addition, all modules support custom installation and uninstallation without affecting the stability of the program, and the physical device is plug-and-play, which enhances the flexibility and scalability of the system, facilitates system maintenance, upgrades, and flexible configuration according to needs.

[0106] Furthermore, the system simplifies the operation process, automatically handles unnecessary settings, and provides intuitive interactive design for touch and keyboard and mouse. The layout algorithm enables easy and free custom layout. Users can set the number, size and position of video windows according to their needs. It is suitable for a variety of practical scenarios and improves the convenience and efficiency of operation. At the same time, the intelligent recognition module does not affect the smoothness of the video during processing, ensuring the viewing experience. The module is simple and safe to enable and disable, which allows users to flexibly control it according to the needs of the scene. At the same time, it is friendly to third-party development access, easy to integrate new recognition functions, adapt to different intelligent recognition application requirements, and improve the intelligence level of the system.

[0107] On the other hand, the embodiment of the present application also provides a low-latency and low-coupling multi-channel analog video signal processing method. Figure 7 is a flow chart of a low-latency, low-coupling multi-channel analog video signal processing method according to an embodiment of the present application, such as Figure 7 As shown, the method is implemented by a signal receiving board, a signal controller, a computing power development board and a video processing development board, including:

[0108] S701, through the signal controller, obtain the frequency information and RSSI value of the initial analog signal from the signal receiving board, and send it to the computing power development board for multi-threaded processing to obtain a compliant analog signal, and configure the signal frequency according to the compliant analog signal;

[0109] S702, using the signal receiving board to collect valid analog signals according to the signal frequency, and transmitting the valid analog signals to the video processing development board;

[0110] S703, converting the effective analog signal into an effective digital signal through the video processing development board, and sending the effective digital signal to the computing power development board to instruct the computing power development board to use the message bus characteristics of the Gstreamer playback module to perform pipeline processing on the effective digital signal and output the decoded video data.

[0111] Through the above steps S701 to S703, in view of the limitations of existing multi-channel video processing technology, analog signal receiving board, controller, video processing development board, computing power development board, serial port receiver and other hardware and Python, Gstreamer and other software technologies are used to solve the problems through signal reception, frequency point screening, video processing and transmission, intelligent recognition, custom layout and other steps. The positive effects of centralized computing power processing, accelerated calculation, fast frequency scanning and low-latency display of the screen, user-customizable layout, and convenient access to intelligent recognition modules are achieved, which is suitable for a variety of actual scenarios under analog video signals.

[0112] also, Figure 8 This is a flowchart of another low-latency and low-coupling multi-channel analog video signal processing method according to an embodiment of the present application.

[0113] In one embodiment, Fig. 9 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Fig. 9 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in Fig. 9 As shown. The electronic device includes a processor, a network interface, an internal memory and a non-volatile memory connected through an internal bus, wherein the non-volatile memory stores an operating system, a computer program and a database. The processor is used to provide computing and control capabilities, the network interface is used to communicate with an external terminal through a network connection, the internal memory is used to provide an environment for the operation of the operating system, the computer program is executed by the processor to implement a low-latency, low-coupling multi-channel analog video signal processing method, and the database is used to store data.

[0114] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by hardware related to computer program instructions, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A low-latency, low-coupling multi-channel analog video signal processing system, characterized in that: The system includes: a signal receiving board, a signal controller, a computing power development board and a video processing development board, wherein: The signal controller is used to obtain the frequency information and RSSI value of the initial analog signal from the signal receiving board, and send it to the computing power development board for multi-threaded data screening and processing to obtain a compliant analog signal, and configure the signal frequency according to the compliant analog signal; The signal receiving board is used to collect valid analog signals according to the signal frequency points configured by the compliant analog signals, and transmit the valid analog signals to the video processing development board; The video processing development board is used to convert the valid analog signal into a valid digital signal, and send the valid digital signal to the computing power development board to instruct the computing power development board to use the message bus characteristics of the Gstreamer playback module to perform pipeline processing on the valid digital signal and output decoded video data, wherein the delay attribute value of the pipeline element is set to a specific value, which affects the transmission speed and processing efficiency of the video stream in the pipeline and reduces the residence time of the video data in the pipeline.

2. The system according to claim 1, characterized in that The computing power development board is also used for: Before processing the video stream, a ping test is performed on the IP addresses of the signal controller, the signal receiving board and the video processing development board, and the removal of the playback module or the reconstruction of the playback module is determined according to the result of the ping test.

3. The system according to claim 1, characterized in that The video processing development board is used to convert the valid analog signal into a video stream of a valid digital signal, and transmit the video stream to the computing power development board through a network; The computing power development board is used to perform pipeline processing on the effective digital signal, decode and play the video stream through a pre-created Gstreamer playback module.

4. The system according to claim 3, characterized in that The Gstreamer playback module includes a preprocessing module, a configuration module, a transmission module and a decoding module, wherein: The preprocessing module is used to select a target decoder and a sink plug-in according to the property information of the system, and to capture the video stream using a transmission control framework established based on RTSP; The configuration module is used to set pipeline elements of each pipeline of the Gstreamer playback module, wherein the pipeline elements are used to constrain data transmission efficiency to be no higher than a preset delay threshold; The transmission module is used to establish a transmission channel with the network address of the video stream sender through the UDP protocol under the transmission control framework, and receive the video stream through the transmission channel under the preset delay threshold constraint; The decoding module is used to decode the video stream received by the transmission module through the target decoder, and transmit the decoded video data to the sink plug-in.

5. The system according to claim 4, characterized in that The computing power development board also includes an image processing module, wherein: The image processing module is used to obtain the video data from the sink plug-in, and call a pre-trained processing model based on user needs through the OpenCV framework, and, through the processing model, adopt asynchronous processing and multi-threaded processing technology to accelerate the recognition of the video data.

6. The system according to claim 5, characterized in that The computing power development board also includes a rendering module, wherein: The rendering module is used to create a class using the QT5 framework to integrate the Frame class of the QT5 framework, and configure the window attribute information of the Frame class according to the width and height values ​​of the current display device; and, obtaining a plurality of basic layout grid templates by calculating through a layout grid algorithm, and storing the layout grid templates in a layout array; A video window is created according to the configuration information of the video data, and the recognition results of each video are configured to the corresponding position of the display window by traversing the layout array.

7. The system according to claim 6, characterized in that The computing power development board also includes a user interaction module, wherein: The user interaction module is used to respond to the user's interaction instruction, trigger the corresponding action signal, recalculate the layout information according to the action signal, and adjust the layout format of each video window according to the recalculated layout information.

8. A low-delay, low-coupling multi-channel analog video signal processing method, characterized in that: The method is implemented by a signal receiving board, a signal controller, a computing power development board and a video processing development board, and includes: The signal controller obtains the frequency information and RSSI value of the initial analog signal from the signal receiving board, and sends it to the computing power development board for multi-threaded data screening and processing to obtain a compliant analog signal, and configures the signal frequency according to the compliant analog signal; The signal receiving board is used to collect valid analog signals according to the signal frequency points configured by the compliant analog signals, and transmit the valid analog signals to the video processing development board; The effective analog signal is converted into an effective digital signal through the video processing development board, and the effective digital signal is sent to the computing power development board to instruct the computing power development board to use the message bus characteristics of the Gstreamer playback module to perform pipeline processing on the effective digital signal and output decoded video data, wherein the delay attribute value of the pipeline element is set to a specific value, which affects the transmission speed and processing efficiency of the video stream in the pipeline and reduces the residence time of the video data in the pipeline.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to claim 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to claim 8 is implemented.

Images