Radio intelligent live-action monitoring and direction-finding system based on atomization protocol

Abstract

The invention relates to a radio intelligent live-action monitoring and direction-finding system based on an atomization protocol, which integrates an augmented reality technology and an atomization protocol, and aims to solve the problems of incapability of sensing an electromagnetic wave propagation environment, insufficient data processing capability, single information presentation mode and the like in the traditional radio monitoring technology. The system comprises radio monitoring equipment, high-performance video acquisition equipment, an augmented reality processing module, a data storage module and a control module. By integrating an augmented reality technology and an atomization protocol, real-time monitoring and positioning of radio signals are realized, and complex radio monitoring data are presented to a user in a visual and interactive manner; the system can perform fusion processing on radio monitoring data and a live-action video to generate an augmented reality video, thereby providing more visual and vivid data support for radio monitoring work; and meanwhile, the system also supports multiple monitoring modes and remote control functions, so that the flexibility and practicability of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of radio monitoring technology, and in particular to a radio intelligent real-scene monitoring and direction-finding system based on an atomization protocol. Background Art

[0002] Radio monitoring is a crucial technology in the field of radio communications, which involves multiple links such as the reception, analysis, identification and processing of radio signals. With the rapid development and widespread application of radio communication technology, radio monitoring technology is also facing more and more challenges and opportunities.

[0003] Traditional radio monitoring technology mainly relies on a single radio monitoring device to obtain relevant information by receiving and analyzing radio signals. However, this technology has problems such as limited monitoring range, insufficient data processing capabilities, and a single information presentation method, which makes it difficult to meet the complex needs of modern radio communication monitoring.

[0004] In order to solve these problems, a variety of improvements and innovations in radio monitoring technologies have emerged in recent years. Among them, radio monitoring technology based on atomic protocols has attracted much attention. Atomic protocols are a technology that decomposes complex communication protocols into a series of simple, independent, and reusable atomic operations. It can improve the flexibility and scalability of communication protocols and reduce the complexity of system development and maintenance.

[0005] In the field of radio monitoring, the application of atomic protocols can realize data exchange and sharing between radio monitoring equipment, and improve the accuracy and real-time nature of monitoring data. At the same time, combined with augmented reality technology, radio monitoring data can be fused with real-life video to generate augmented reality video, allowing users to view the distribution and changes of radio signals in the monitoring area in a more intuitive and interactive way.

[0006] However, the existing radio monitoring technology based on atomic protocols still has some problems, such as low data processing efficiency, poor system stability, poor user experience, etc. Therefore, a new type of radio monitoring system is needed to solve these problems, improve the efficiency and accuracy of radio monitoring, and meet the complex needs of modern radio communication monitoring.

[0007] In summary, the present invention proposes a radio intelligent real-scene monitoring and direction finding system based on an atomization protocol, which aims to realize real-time monitoring and direction finding of radio signals by integrating augmented reality technology and an atomization protocol, and present the monitoring data to users in an intuitive and interactive manner, thereby improving the efficiency and accuracy of radio monitoring and promoting the innovation and development of radio monitoring technology. Summary of the invention

[0008] In order to solve the problems of the prior art, the present invention provides a radio intelligent real-scene monitoring and direction finding system based on an atomization protocol.

[0009] In order to solve the above technical problems, the present invention is implemented by the following technical solutions: a radio intelligent real-scene monitoring and direction finding system based on an atomization protocol, characterized in that it includes:

[0010] Radio monitoring equipment, deployed with an atomic protocol interface, is used to obtain monitoring data of radio signals in real time;

[0011] High-performance video acquisition equipment, used to collect real-scene video of the monitored area;

[0012] Augmented reality processing module, used to integrate radio monitoring data into real-life video to achieve real-time visualization of radio signals;

[0013] A data storage module, used to store fused real-scene video and radio monitoring data;

[0014] The control module is used to configure the parameters of the radio monitoring equipment, manage the direction finding tasks, and control the workflow of the video acquisition equipment and the augmented reality processing module.

[0015] In this implementation, the predetermined goals were achieved by designing the system architecture from the hardware layer, data layer, application layer, network layer and key technology aspects; by integrating augmented reality technology and atomization protocol, real-time monitoring and positioning of radio signals were achieved, and complex radio monitoring data were presented to users in an intuitive and interactive way, greatly improving the efficiency and accuracy of monitoring work; the system can integrate radio monitoring data with real-life videos to generate augmented reality videos, so that users can intuitively see the distribution and changes of radio signals in the monitoring area, providing more intuitive and vivid data support for radio monitoring work; in radio interference troubleshooting and illegal identification and positioning scenarios, the system can quickly and intuitively display real-life videos of the transmission area of ​​interference sources or illegal stations, providing timely and accurate information support for emergency response, helping to quickly solve problems and ensure the standardization and safety of radio spectrum use; the system supports multiple monitoring modes such as single signal monitoring, multi-signal monitoring and spectrum scanning, which can meet the needs of different application scenarios and provide more comprehensive and flexible technical support for radio monitoring work.

[0016] In a specific implementation, the radio monitoring device communicates with the data processing module through an atomic protocol interface to implement a direction finding function of the radio signal and sends the direction finding data to the augmented reality processing module.

[0017] In a specific implementation, an augmented reality processing module is developed using C++ language to process video stream data and radio monitoring data, and to overlay key information such as frequency, field strength, and direction of radio signals onto real-life video in an intuitive and interactive manner.

[0018] In a specific implementation, the data storage module includes two modes: local storage and remote storage, which are used to store monitoring data and real-scene videos for a long time for subsequent analysis and backtracking.

[0019] In a specific implementation, the control module provides a user interaction interface for configuring basic information such as the IP address, port number, and device ID of the radio monitoring equipment, managing the creation, modification, viewing, and deletion of single-frequency direction finding tasks, and controlling the working status of the video acquisition device and the augmented reality processing module in real time.

[0020] In a specific implementation, the system also includes an on-board artificial intelligence edge control module for realizing real-time processing and analysis of radio monitoring data on a mobile monitoring vehicle, thereby improving the response speed and accuracy of the system.

[0021] In a specific implementation, the system supports multiple monitoring modes, including single signal monitoring, multi-signal monitoring, and spectrum scanning, etc., to meet the needs of different application scenarios.

[0022] In a specific implementation manner, the system implements data communication through TCP / IP protocol and adopts multiplexing technology to improve data transmission efficiency and ensure the real-time and accuracy of radio monitoring data.

[0023] In a specific implementation, the system client is also included, including: a PTZ monitoring real-time playback and control module, which is mainly connected to the camera development tool to realize the function of real-time playback of the video stream of the monitoring camera and to realize the control of the PTZ;

[0024] Camera login module: used to implement the camera login function, which is the basis for the PTZ control and the playback and control of the video collected by the camera;

[0025] Screenshot and screen recording viewing module: used to view and manage screenshots and screen recordings during video streaming;

[0026] Radio monitoring vehicle configuration module: configure the basic information of the radio monitoring vehicle's atomic protocol, including the intranet IP address, port number, and basic device ID information;

[0027] Single-frequency direction finding task module: used to manage single-frequency direction finding tasks, including creating single-frequency direction finding tasks, modifying single-frequency direction finding tasks, obtaining single-frequency direction finding data, parsing single-frequency direction finding data, etc.

[0028] Database management module: encapsulates the related functions of database management, which realizes the operation of tables in the database, mainly realizing the functions of adding, deleting, modifying and checking data;

[0029] Radio atomization protocol docking module: used to dock with the atomization protocol and realize docking with the atomization service, to perform corresponding radio monitoring tasks, obtain corresponding data, and parse the data into the data format required by the cost system;

[0030] The video stream processing module has two functions. The first function is to obtain the video stream captured by the camera through the camera development tool, and parse and process the data to realize the real-time playback function of the video stream. The second function is to use augmented reality technology to merge the video stream and single-frequency direction finding technology to form a new video stream.

[0031] File service module: used for file server function, saving the captured and modified video stream to the file server;

[0032] Streaming service module: Use FFMPEG function library to implement video streaming function, which is used to provide video streaming API service function to third parties;

[0033] Network service module: used to implement network services, connect to network interfaces, and provide basic services for the connection of atomic protocols;

[0034] API module: used to provide API interface services to third parties. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the system hardware architecture of the present invention.

[0036] Figure 2 It is a schematic diagram of the system data flow architecture of the present invention.

[0037] Figure 3 It is a schematic diagram of the client functional module of the present invention.

[0038] Figure 4 It is a schematic diagram of the application layer architecture of the present invention.

[0039] Figure 5 It is a schematic diagram of the system function modules of the present invention.

[0040] Figure 6 It is a schematic diagram of the pan-tilt monitoring, playback and control module of the present invention.

[0041] Figure 7 It is a schematic diagram of the Hikvision real-time preview module flow of the present invention.

[0042] Figure 8It is a schematic diagram of the flow chart of the pan / tilt control module of the present invention.

[0043] Fig. 9 It is a basic flow chart of the camera development tool of the present invention.

[0044] Fig.10 It is a functional schematic diagram of the configuration module of the radio monitoring vehicle of the present invention.

[0045] Fig.11 It is a schematic diagram of a single-frequency direction finding task module of the present invention.

[0046] Fig.12 It is a schematic diagram of the functional details of the database management module of the present invention.

[0047] Fig.13 It is a schematic diagram of the video stream processing functional module of the present invention.

[0048] Fig.14 It is a schematic diagram of the file service module of the present invention.

[0049] Fig.15 It is a flow chart of the monitoring video streaming function of the present invention.

[0050] Fig.16 It is a schematic diagram of the network service module of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0052] Example: Figure 1 As shown: The main hardware includes a mobile radio monitoring vehicle, radio monitoring equipment with atomic protocol, wireless vehicle-mounted PTZ camera, vehicle-mounted keyboard display terminal, vehicle-mounted artificial intelligence edge control module, and vehicle-mounted display. In this hardware architecture, the radio monitoring equipment and PTZ surveillance camera are connected to the artificial intelligence edge module controller through a gateway device. The program in the artificial intelligence edge module controller controls the radio monitoring equipment and PTZ surveillance camera through the network protocol, and obtains the corresponding data corresponding to the corresponding network protocol SDK. The vehicle-mounted keyboard display terminal is connected through a data cable. After the connection is successful, the vehicle-mounted keyboard display terminal can display the image data of the surveillance camera and control the PTZ camera through the keyboard.

[0053] like Figure 2As shown, in this system, the main data includes camera monitoring data stream, pan / tilt control data stream, radio monitoring equipment data stream, data stream storage, and vehicle-mounted keyboard display terminal data stream.

[0054] First of all, for the data source of the pan-tilt surveillance camera, the corresponding data is the video data collected by the camera. The format of the video data is H.264 data. The video data is transmitted to the video stream processing system, which will process the video stream data and convert it into video data that is easy to process and save, and send it to the file server and the streaming server for video file storage and video stream streaming services; and through the streaming server, an interface API is provided to the corresponding algorithm of the external theoretical research part.

[0055] For the vehicle-mounted keyboard display terminal, the corresponding data includes video stream data, which is transmitted by the pan-tilt monitoring camera to the display terminal for real-time display of the video; on the other hand, the data includes vehicle-mounted keyboard control instructions, which are transmitted to the pan-tilt monitoring camera through the vehicle-mounted keyboard, and then control the pan-tilt or monitoring, including focusing, focusing, bit stream and other controls, as well as pan-tilt rotation and other controls.

[0056] Finally, for the radio monitoring equipment, the main data of the equipment is the radio monitoring direction finding data, which is acquired through the atomic protocol, and then transmitted to the radio direction finding data processing algorithm to process the direction finding data, mainly parsing it according to the corresponding protocol standards, parsing it into the data format required by the system, and then transmitting it to the database for storage.

[0057] In addition, the system's video stream data and radio monitoring direction-finding data are both fed into the augmented reality processing algorithm for processing after being processed by the corresponding algorithms, and then the data is played in real time to assist radio regulators in completing radio monitoring tasks.

[0058] On the other hand, for the data stored in the database, when the supervisor performs radio data backtracking, the data is transmitted to the data display part through the corresponding backtracking algorithm for corresponding data display.

[0059] like Figure 3 As shown, for the application layer, the system mainly refers to the program used for the user end, and the user end program is mainly divided into several parts: PTZ surveillance camera real-time playback and control module, system configuration module, camera login module, screenshot viewing or screen recording module, radio detection vehicle configuration module, and radio single-frequency direction finding task management module.

[0060] In the current system version, and in version v1.0, the main client mode is implemented, that is, the corresponding module development is carried out in the client. The corresponding configuration and functions of each module are as follows.

[0061] The design of these modules is based on the v1.0 version. In the subsequent version iteration process, the C / S architecture will be implemented, that is, the client focuses on the display of integrated data, while the server realizes data processing and storage.

[0062] The specific descriptions of the above corresponding modules are as follows:

[0063] PTZ monitoring real-time playback and control module: This module is mainly connected to the camera development tool to realize the real-time playback function of the surveillance camera's video stream and to realize the control of the PTZ.

[0064] Camera login module: This module mainly implements the camera login function, which is the basis for the playback and control of the pan / tilt control and the video captured by the camera.

[0065] Screenshot and screen recording viewing module: This module is mainly used to view and manage screenshots and screen recordings during video streaming.

[0066] Radio monitoring vehicle configuration module: This module is mainly used to configure the basic information of the radio monitoring vehicle's atomic protocol, including the intranet IP address, port number, device ID and other basic information.

[0067] Single-frequency direction finding task module: This module is mainly used to manage single-frequency direction finding tasks, including creating single-frequency direction finding tasks, modifying single-frequency direction finding tasks, obtaining single-frequency direction finding data, parsing single-frequency direction finding data, etc.

[0068] Database management module: This module encapsulates the relevant functions of database management, which implements operations on tables in the database, mainly realizing the functions of adding, deleting, modifying and checking data.

[0069] Radio atomic protocol docking module: This module is one of the core modules of the system. This module is mainly used to dock with the atomic protocol and realize the docking with the atomic service. It is used to perform corresponding radio monitoring tasks, obtain corresponding data, and parse the data into the data format required by the cost system.

[0070] Video stream processing module: This module is also one of the core functional modules of the system. This module has two main functions. Function one is to obtain the video stream captured by the camera through the camera development tool, and parse and process the data to realize the real-time playback function of the video stream; Function two is to use augmented reality technology to merge the video stream and single-frequency direction finding technology to form a new video stream.

[0071] File service module: This module is mainly used for file server functions, saving the captured and modified video stream to the file server. Push streaming service module: This module mainly uses the FFMPEG function library to implement the push streaming function of the video stream, and is used to provide video stream API service functions to third parties. Network service module: This module mainly implements network services, which are used to connect to the network interface and provide basic services for the connection of the atomic protocol. API module: This module is mainly used to provide API interface services to third parties.

[0072] like Figure 4 As shown, in the subsequent development of v2.0, the system will be modified to a C / S architecture, with the client used for data display and export, and the server used for data acquisition, processing and storage.

[0073] This version has upgraded the overall architecture of the V1.0 version, which is also the final overall system architecture of the system. Compared with the system architecture of v1.0, the system architecture mode is C / S mode, and the main function of the system lies in the server part. This part designs the module of the v1.0 version as a microservice module on the server side, designs the system as a microservice architecture, and adds nginx to be responsible for load balancing of balancing services for requests. In addition, the current limiting and fuse function of Sentinel is added, and the service registration center and configuration management center are added. In terms of database, in addition to mysql and MongoDB, in addition to adding redis as a cache service, the big data architecture Hadoop is added, Hive is used for data storage, and the MapReduce framework is used for radio monitoring data processing and development work.

[0074] like Figure 5 As shown in the introduction of the system application layer architecture, we divide the system into two stages. The first stage is an independent client mode, where all system functions are integrated into the client, and other interfaces such as video API are provided from the client to other modules. The second stage divides the system into client and server modes, namely C / S mode, which implements system function services on the server side and implements data display and customer interaction functions on the client side. Although the subject is divided into two stages, the actual corresponding functions are one-to-one and have not changed significantly; the reason for the change in architecture is the result of continuous iteration as the functions are used.

[0075] System function design is to carry out detailed design of system functions based on the above system function module design. The following is a description diagram of the system function module:

[0076] In the application layer architecture, it is also introduced that the system is divided into PTZ monitoring playback and control module, camera login module, screenshot and screen recording function module, radio monitoring vehicle configuration module, single-frequency direction finding task module, database management module, radio atomization protocol docking module, video stream processing module, file service module, streaming service module, network service module and API module. The following will be a detailed design of each module of the system.

[0077] like Figure 6 As shown, the PTZ monitoring playback and control module is mainly responsible for the control of the camera and PTZ. The specific functional design is described as follows:

[0078] Real-time playback of surveillance video streams: By connecting to the playback control component of Hikvision's official SDK, the real-time playback function of the video stream is realized. The corresponding playback control method is: CHCNetSDK.NET_DVR_RealPlay_V40(m_lUserID,ref lpPreviewInfo,null / *RealData* / ,pUser);

[0079] Surveillance video stream playback control: This function is mainly used to control the playback of the video stream, start previewing the surveillance video stream and stop playing the surveillance video stream.

[0080] Video stream code setting: In the preview function of monitoring video stream, you can switch the code stream of the video stream: main code stream and sub code stream. This function is mainly to realize the playback control of the two code streams.

[0081] Video stream resolution setting: In the preview of the monitoring video stream, you can also modify the resolution of the video stream to adjust the video playback effect.

[0082] Focus control of surveillance cameras: Control the focus of surveillance cameras by connecting to Hikvision's control SDK.

[0083] Surveillance camera focus control: Control the focus of the surveillance camera by connecting to Hikvision's control SDK.

[0084] Surveillance camera aperture control: By connecting to Hikvision's control SDK, you can control the aperture of the surveillance camera and the amount of light passing through the lens, thereby affecting the exposure and depth of field of the photo. The aperture controls the amount of light entering the camera by changing its opening. The larger the aperture (i.e., the smaller the F number), the more light enters, and the photo will be brighter; conversely, the smaller the aperture (i.e., the larger the F number), the less light enters, and the photo will be darker.

[0085] Surveillance camera wiper control: Since the surveillance PTZ of the system is vehicle-mounted, that is, it needs to be installed on a radio monitoring vehicle, it needs to adapt to complex external environments. When it rains or snows, raindrops or snowflakes are likely to gather on the surveillance lens, so the wipers on the surveillance camera have a corresponding function. The system connects to the wiper control interface of the Hikvision camera to control the wipers of the surveillance camera, making the video stream clearer.

[0086] 8-direction control of the gimbal: In addition to realizing the radio real-scene intelligent monitoring and direction-finding function, the system also needs to cooperate with the theoretical research part to realize the tracking of the drone, so the direction control of the camera gimbal is also necessary; therefore, by connecting to the gimbal control SDK, the control of 8 directions such as the rotation of the gimbal can be realized.

[0087] Video stream screenshot function: By connecting to the camera development tool, you can realize the screenshot function of the video stream to retain the corresponding image.

[0088] Video stream recording function: The video stream recording function is realized by connecting to the camera development tool and saving it to the corresponding file server.

[0089] Screenshot and video viewing and management: Manage the pictures saved by screenshots and the videos saved by videos, including viewing pictures and videos, deleting, and other operations.

[0090] like Figure 7 , Figure 8 As shown, for the detailed functional design of the current module of the system, we need the real-time preview module process and the pan-tilt control module process of the camera development tool.

[0091] The modules in the dotted box are related to the preview module and can only be called after the preview is started. These modules are parallel to each other and each completes the corresponding function.

[0092] The sound control function mainly realizes the opening and closing of exclusive and shared sounds and the control of volume.

[0093] The real-time streaming data capture and recording module mainly implements the functions of real-time streaming data callback and local recording, which can be used for subsequent processing by users.

[0094] The capture function is mainly used to capture the current decoded image and save it in BMP format.

[0095] The SDK divides the PTZ control into two modes: one is to control through the handle returned by the image preview; the other is to control the PTZ without preview restrictions through the user registered ID number.

[0096] The PTZ control function is implemented under the premise of enabling preview. The handle returned by the preview interface is used as the first parameter of the PTZ control interface.

[0097] No preview is required. After registering and logging in to the device, you can perform PTZ control. The handle returned by the login interface is used as the first parameter of the PTZ control interface.

[0098] like Fig. 9 As shown, the PTZ monitoring initialization and login module

[0099] Before controlling the playback of the PTZ surveillance video stream, it is necessary to initialize and log in the PTZ surveillance camera. The purpose of initialization is to initialize the Hankang Satellite TV SDK, and the purpose of login is to log in to the built-in server of the PTZ surveillance camera through the IP, port number, account and password, obtain the authority to operate the camera, and then perform corresponding control.

[0100] At this point, all the functions of the PTZ surveillance camera have been designed in detail. The overall process of the SDK of the PTZ surveillance camera is as follows: Figure 2 As shown:

[0101] The process in the dotted box is optional and will not affect the functional use of other processes and modules. It can be divided into ten modules according to the different functions to be implemented. When implementing the functions of each module, the four processes of initializing SDK, user registration of device, device deregistration and release of SDK resources are essential.

[0102] Initialize SDK (NET_DVR_Init): Initialize the entire network SDK system, pre-allocate memory, etc.

[0103] Set the callback function for receiving abnormal messages: Since most modules in the SDK are implemented in asynchronous mode, we provide this interface to receive abnormal information from preview, alarm, playback, transparent channel, voice intercom and other modules. Users can set this callback function after initializing the SDK to receive and process abnormal messages from each module at the application layer.

[0104] Get the device's IP address from the resolution server: This interface provides a method to get the device's IP address from the resolution server when only the device name (or DDNS domain name) or serial number is known. For example, if the current device obtains a dynamic IP address through dial-up Internet access, and the server running our company's IPServer software is the resolution server or the device is registered on our company's DDNS server, we can use this interface to enter the server address, device name or serial number and other information to query the device's IP address.

[0105] User registration device (NET_DVR_Login_V40): realizes the user registration function. After successful registration, the returned user ID is used as the unique identifier for other function operations. The SDK allows a maximum of 512 registered users. CVR devices allow 16 registered user names, and a maximum of 128 users can be registered at the same time.

[0106] Channel management module: including channel addition, deletion, modification, recording plan configuration, stream status, etc. CVR channel management is divided into channel mode and stream ID mode.

[0107] Preview module: Get real-time code stream from the device, decode and display, and control playback.

[0108] PTZ control module: realizes the basic operation, preset point, cruise and track control of the PTZ. SDK divides the PTZ control into two modes: one is to control through the handle returned by the image preview; the other is to control the PTZ without preview restrictions through the user registered ID number.

[0109] Playback and download module: can remotely play back or download the device's video files by time, and can be decoded or stored later.

[0110] General parameter configuration module: Set and obtain device parameters, mainly including device parameters, network parameters, user parameters, hard disk management parameters and other parameter information.

[0111] Remote device maintenance module: implements maintenance tasks such as restarting the device, restoring default values, remote upgrading, and importing / exporting configuration files.

[0112] Alarm module: processes various alarm signals uploaded by the device. There are two types of alarms: "arming" and "monitoring". When the monitoring mode is used and the user ID is not required, the alarm module does not need to perform the "user registration" operation step.

[0113] like Fig.10 As shown, the radio monitoring vehicle configuration module

[0114] Before controlling the radio monitoring equipment in the radio monitoring vehicle, its corresponding parameters need to be configured, which is the origin of this module. The corresponding configuration of the radio monitoring equipment will be saved in the database through the database management module for subsequent use.

[0115] The main function of this module is to manage the information of the radio monitoring equipment on the radio vehicle accordingly, and then use this information when controlling the device through the docking atomic protocol.

[0116] like Fig.11 As shown, the single-frequency direction finding task module

[0117] The main functions of this module are two-fold. On the one hand, it manages single-frequency direction finding tasks, including adding new single-frequency direction finding tasks, modifying tasks, viewing single-frequency direction finding historical tasks and ongoing single-frequency direction finding tasks, and deleting tasks. On the other hand, it calls the function of the atomic protocol docking module to start the single-frequency direction finding task and obtain the corresponding data, and then displays and saves the data.

[0118] Create a scheduled task: To create a scheduled task, the user specifies the scheduled task that needs direction finding, and then creates a new scheduled task and saves it in the database; then calls the function in the atomic protocol docking module to implement the single-frequency direction finding function, and then obtains the corresponding direction finding data through callback.

[0119] View historical tasks: After the historical tasks are completed, you can view the relevant information of the historical tasks. At the same time, you can view the direction finding data obtained by the scheduled tasks and make corresponding visualization displays.

[0120] Delete scheduled tasks: Delete historical scheduled tasks.

[0121] View the ongoing scheduled tasks: In the current system, there can only be one single-frequency direction finding task running at the same time, because there is only one radio monitoring device in the radio monitoring vehicle, so only one single-frequency direction finding task can be run at the same time.

[0122] Stop the ongoing direction finding task: When viewing the running single-frequency direction finding task, you can call the atomic protocol docking module to stop the ongoing single-frequency direction finding task.

[0123] Radio Atomization Docking Module

[0124] This module is mainly connected with the atomization protocol to realize the control of radio monitoring equipment and execute the corresponding radio monitoring instructions, and then parse the acquired data through the specified radio atomization protocol to obtain the data required by the system.

[0125] like Fig.12 As shown, the database management module

[0126] In this system, the database management module mainly maintains a set of database management related interface encapsulation, users uniformly manage the database in the system, and uniformly encapsulate various CRUD operations of the database table. The system uses the windows system as the program or server-side carrier system, and the front end of the system is developed using c#, so the system uses Sql Server 2016 as the system database to save system data. In the subsequent 2.0 version development process, in addition to the Sql Server relational database, MongoDB is also required to store non-relational data. At the same time, hadoop's Hive is also required as the storage database for big data.

[0127] In this system, the main functional details of the database management module are as follows: Database parameter configuration: You need to configure the corresponding parameters for connecting to the Sql Server server, including IP, port number, account, password and database information to be connected. Database connection: Use the configured database connection parameters and the corresponding database connection driver to connect to the database. Database disconnection: Disconnect the connected database. Execute query statements: Encapsulate the method for executing query statements in the database. Execute add, delete and modify statements and return the number of affected rows: Encapsulate the method for executing add, delete and modify statements, which returns the number of affected rows.

[0128] like Fig.13 As shown, the video stream processing module

[0129] This module is mainly used for processing PTZ monitoring video streams, mainly through the camera development tool to obtain the frame data of the video stream, and then draw the graphics required by the system on the video frame, and display the direction finding data obtained by the single-frequency direction finding task in a visual form. At the same time, this module also needs to implement augmented reality technology and realize the drawing of 3D models to realize the display function of augmented reality.

[0130] In order to realize the visualization of direction finding data in version 1.0, the system draws a graphic drawing layer on the upper layer of the video stream player in version 1.0 to visualize the direction finding data. In version 2.0, the original video obtained by Hikvision surveillance cameras, that is, H.264 encoded video data, is processed and the direction finding data is drawn directly in the video stream, so that the video data can be directly saved to the file server, reducing the space occupied by the direction finding data.

[0131] like Fig.14 As shown, the file service module is mainly used for file services, which are used to store pictures and videos required by the system. The corresponding functional details of this module are shown in the figure:

[0132] The file service module mainly provides file operation related interfaces to other modules. Therefore, we divide the file service module into two management contents: Bucket management and file management. The corresponding detailed functional descriptions are as follows:

[0133] Create bucket: In the file service module, bucket can be understood as a folder, which can store file data in different categories. This function is to create a bucket for other modules to call for file management.

[0134] View bucket files: View a list of all files under the bucket.

[0135] Delete bucket: When the bucket is no longer in use, you can call this function to delete the bucket. An important condition for calling the delete bucket function is that the deleted bucket cannot be empty.

[0136] Transfer objects from the input stream and save them to a bucket: In system development, all file-related operations are based on stream processing of input and output streams. Therefore, this function saves the objects transferred in the input stream into the specified bucket in the corresponding format.

[0137] Get the input stream of the specified object from the bucket: This function is exactly the opposite of the previous function. Its function is to convert the specified object in the specified bucket into an input stream for the called module to perform stream processing.

[0138] Get the temporary URL of an object: In system development, sometimes the purpose is not to obtain the input stream of a specified object for stream processing, but just to temporarily access the object in the system; for example, to display an image object in the system, you can use this function to obtain the access URL of the object.

[0139] Delete object: If an object in a specified bucket in the file service is no longer needed and needs to be deleted, you need to call this function to perform the deletion operation on the object.

[0140] Upload Multipart File: In the system, especially when using WEB front-end development to upload files, the back-end interface obtains the corresponding Multipart File object. If you need to save the object to the file server, you need to call this function to save the file.

[0141] Download file: This function directly sends the input stream of the file object to be downloaded as the output stream of HttpServletResponse to the module that calls this function through HttpServletResponse.

[0142] like Fig.15 As shown, the streaming service module

[0143] The main function of this module is to push the video stream data obtained by the PTZ surveillance camera through FFMEPG, and expose the corresponding push stream URL to the caller for video analysis and other artificial intelligence related technical processing.

[0144] FFmpeg is the leading multimedia framework, capable of decoding, encoding, transcoding, multiplexing, demultiplexing, streaming, filtering, and playing nearly everything created by humans and machines. It is highly portable: FFmpeg runs across Linux, Mac OS X, Microsoft Windows, BSD, Solaris, and other operating systems in a variety of build environments, machine architectures, and configurations.

[0145] The corresponding functions of the push streaming service module, in conjunction with other service modules, the process of the surveillance camera video push streaming function is shown in the figure:

[0146] First of all, the video streaming service requires the cooperation of multiple modules. Initially, the initialization of the monitoring SDK is required. After the SDK is initialized, the IP, port number, login account and password of the PTZ monitoring camera are used to log in to the PTZ monitoring. After successful login, the PTZ monitoring real-time playback and control module is called to perform the playback function of the monitoring video stream to perform real-time playback of the video stream. At the same time, the video stream will send the corresponding video stream data to the callback function. In the callback function, the video stream data of the PTZ monitoring can be obtained. Then the video stream processing algorithm of the video stream processing module is called to process the video stream. After the processing is completed, on the one hand, the file service module is called to save the data to the file server. On the other hand, the video stream streaming service of the streaming service module is called to push the video stream data.

[0147] like Fig.16 As shown, the main function of the network service module is to further encapsulate the network request so that it can be called by other modules when making network data requests. The corresponding functional details are shown in the figure: First, in this module, a function with basic network data request function is encapsulated, and other functions are expanded with detailed functions based on this function, which respectively realize synchronous GET request, synchronous POST form data request, and synchronous POST XML data request.

[0148] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A radio intelligent real-time monitoring and direction finding system based on an atomic protocol, characterized in that: include: Radio monitoring equipment, deployed with an atomic protocol interface, is used to obtain radio spectrum monitoring data of all heterogeneous radio monitoring equipment in the existing radio monitoring network in real time; High-performance video acquisition equipment, used to collect real-scene video of the monitored area; Augmented reality processing module, used to integrate radio monitoring data into real-life video to achieve real-time visualization of radio signals; A data storage module, used to store fused real-scene video and radio monitoring data; The control module is used to configure the parameters of the radio monitoring equipment, manage the direction finding tasks, and control the workflow of the video acquisition equipment and the augmented reality processing module.

2. According to the atomization protocol-based radio intelligent real-scene monitoring and direction finding system according to claim 1, it is characterized by: The radio monitoring device communicates with the data processing module through an atomic protocol interface to implement monitoring and direction finding functions of the desired radio signal, and sends the monitoring and direction finding data to the augmented reality processing module.

3. The radio intelligent real-scene monitoring and direction finding system based on the atomic protocol according to claim 1 is characterized in that: The augmented reality processing module is developed in C++ language and is used to process video stream data and radio monitoring data, and superimpose key information such as frequency, field strength, and direction-finding angle of radio signals onto real-life video in an intuitive and interactive manner.

4. The radio intelligent real-scene monitoring and direction finding system based on the atomic protocol according to claim 1 is characterized in that: The data storage module includes two modes: local storage and remote storage, which are used to store monitoring data and real-scene videos for a long time for subsequent analysis and backtracking.

5. The radio intelligent real-scene monitoring and direction finding system based on the atomic protocol according to claim 1 is characterized in that: The control module provides a user interaction interface for configuring the IP address, port number, and basic device ID information of the radio monitoring equipment, managing the creation, modification, viewing, and deletion of single-frequency direction finding tasks, and controlling the working status of the video acquisition device and the augmented reality processing module in real time.

6. The radio intelligent real-scene monitoring and direction finding system based on the atomic protocol according to claim 1 is characterized by: The system also includes an on-board artificial intelligence edge control module, which is used to realize real-time processing and analysis of radio monitoring data on the mobile monitoring vehicle, thereby improving the response speed and accuracy of the system.

7. The radio intelligent real-scene monitoring and direction finding system based on the atomic protocol according to claim 1 is characterized in that: The system supports multiple monitoring modes, including single signal monitoring, multi-signal monitoring and spectrum scanning, to meet the needs of different application scenarios.

8. The radio intelligent real-scene monitoring and direction finding system based on the atomic protocol according to claim 1 is characterized by: The system realizes data communication through TCP / IP protocol and adopts multiplexing technology to improve data transmission efficiency and ensure the real-time and accuracy of radio monitoring data.

9. The radio intelligent real-scene monitoring and direction finding system based on the atomic protocol according to claim 1 is characterized by: Also includes system clients, including: The PTZ monitoring real-time playback and control module is mainly used to connect to the camera development tool to realize the real-time playback of the video stream of the monitoring camera and the control of the PTZ; Camera login module: used to implement the camera login function, which is the basis for the PTZ control and the playback and control of the video collected by the camera; Screenshot and screen recording viewing module: used to view and manage screenshots and screen recordings during video streaming; Radio monitoring vehicle configuration module: configure the basic information of the radio monitoring vehicle's atomic protocol, including the intranet IP address, port number, and basic device ID information; Single-frequency direction finding task module: used to manage single-frequency direction finding tasks, including creating single-frequency direction finding tasks, modifying single-frequency direction finding tasks, obtaining single-frequency direction finding data, and parsing single-frequency direction finding data; Database management module: encapsulates the related functions of database management, which realizes the operation of tables in the database, mainly realizing the functions of adding, deleting, modifying and checking data; Radio atomization protocol docking module: used to dock with the atomization protocol and realize docking with the atomization service, to perform corresponding radio monitoring tasks, obtain corresponding data, and parse the data into the data format required by the cost system; The video stream processing module has two functions. The first function is to obtain the video stream captured by the camera through the camera development tool, and parse and process the data to realize the real-time playback function of the video stream. The second function is to use augmented reality technology to merge the video stream and single-frequency direction finding technology to form a new video stream. File service module: used for file server function, saving the captured and modified video stream to the file server; Streaming service module: Use FFMPEG function library to implement video streaming function, which is used to provide video streaming API service function to third parties; Network service module: used to implement network services, connect to network interfaces, and provide basic services for the connection of atomic protocols; API module: used to provide API interface services to third parties.

Images