Unmanned aerial vehicle video processing method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an unmanned aerial vehicle video processing method and system, and the method comprises the steps: obtaining a video analysis instruction; configuring a video frame extraction parameter according to the video analysis instruction, wherein the video frame extraction parameter comprises a video frame extraction time interval; the method comprises the following steps: acquiring a video stream of an unmanned aerial vehicle, decoding the video stream to obtain a video frame sequence, and storing the video frame sequence into a queue; extracting a corresponding target video frame from the queue according to the video frame extraction time interval; and performing detection processing on the target video frame through an algorithm unit. According to the embodiment of the invention, a frame extraction strategy is allowed to be dynamically adjusted according to different analysis requirements, and compared with a rigid mode of fixed full-frame processing, indiscriminate analysis on all video frames is avoided, so that the number of frames input into an algorithm unit is reduced, the calculation resource requirement is reduced, and repeated calculation and resource occupation are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, and in particular to an unmanned aerial vehicle video processing method, an unmanned aerial vehicle video processing system, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Unmanned aerial vehicles are increasingly widely used in the fields of real-time monitoring, target tracking and environmental analysis, and the high-definition cameras carried by the unmanned aerial vehicles can generate continuous video streams, providing a rich data basis for subsequent intelligent analysis.

[0003] In the prior art, unmanned aerial vehicle video data is usually transmitted to a processing system, and artificial intelligence algorithms are used to perform target detection and event recognition on the video content, so as to meet the needs of security, agriculture or logistics scenarios. However, due to the high resolution and high frame rate characteristics of the unmanned aerial vehicle video stream, the processing system needs to perform real-time analysis on a large amount of video data, which puts high requirements on the performance and response speed of computing resources. SUMMARY

[0004] In view of the above problems, the present application embodiments are proposed in order to provide an unmanned aerial vehicle video processing method, an unmanned aerial vehicle video processing system, an electronic device and a computer readable storage medium which overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above problems, the present application embodiments disclose an unmanned aerial vehicle video processing method, which comprises: acquiring a video analysis instruction; configuring video frame extraction parameters according to the video analysis instruction, the video frame extraction parameters comprising a video frame extraction time interval; acquiring a video stream of an unmanned aerial vehicle, decoding the video stream to obtain a video frame sequence, and storing the video frame sequence in a queue; extracting a corresponding target video frame from the queue according to the video frame extraction time interval; and performing detection processing on the target video frame by an algorithm unit.

[0006] Optionally, the step of configuring video frame extraction parameters according to the video analysis instruction, the video frame extraction parameters comprising a video frame extraction time interval, comprises: parsing the video analysis instruction to obtain a specified algorithm type; querying a preset algorithm time consumption database to obtain the video frame extraction time interval corresponding to the algorithm type; and generating the video frame extraction parameters according to the video frame extraction time interval.

[0007] Optionally, the step of extracting a corresponding target video frame from the queue according to the video frame extraction time interval comprises: calculating a target extraction time point according to the video frame extraction time interval; and retrieving a video frame with a time stamp matching the target extraction time point from the queue as the target video frame.

[0008] Optionally, the obtaining the video analysis instruction comprises: obtaining the video analysis instruction through a dedicated data channel; wherein the video analysis instruction is forwarded through a relay service, and the relay service ensures that the data channel is isolated from an external network through an isolated addressing mechanism.

[0009] Optionally, the decoding processing the video stream to obtain a video frame sequence and storing the video frame sequence in a queue comprises: decoding the video stream through a hardware accelerated decoder to generate a continuous video frame sequence containing timestamps; and storing the video frame sequence in the queue in chronological order.

[0010] Optionally, the detecting processing the target video frame through an algorithm unit comprises: performing target detection algorithm through the algorithm unit to identify targets and / or classify events in the target video frame; and generating a detection result containing target positions and / or event types.

[0011] Optionally, after the detecting processing the target video frame through an algorithm unit, the method further comprises: performing time series analysis and / or event correlation on the detection result to generate a statistical report; and transmitting the statistical report to an instruction generation system through an encrypted channel, wherein the statistical report is used to generate updated video analysis instructions and / or adjust unmanned aerial vehicle operation parameters.

[0012] The embodiment of the application further discloses an unmanned aerial vehicle video processing system, which comprises: an analysis instruction acquisition module configured to acquire video analysis instructions; an extraction parameter configuration module configured to configure video frame extraction parameters according to the video analysis instructions, wherein the video frame extraction parameters comprise video frame extraction time intervals; a video stream decoding module configured to acquire a video stream of an unmanned aerial vehicle, decode the video stream to obtain a video frame sequence, and store the video frame sequence in a queue; a video frame extraction module configured to extract corresponding target video frames from the queue according to the video frame extraction time intervals; and a video frame detection module configured to detect the target video frames through an algorithm unit.

[0013] Optionally, the extraction parameter configuration module comprises: an algorithm type analysis module configured to analyze the video analysis instructions to obtain a specified algorithm type; a time interval query module configured to query a preset algorithm time consumption database to obtain the video frame extraction time intervals corresponding to the algorithm type; and an extraction parameter generation module configured to generate the video frame extraction parameters according to the video frame extraction time intervals.

[0014] Optionally, the video frame extraction module comprises: an extraction time point calculation module, configured to calculate a target extraction time point according to the video frame extraction time interval; and a target video frame retrieval module, configured to retrieve a video frame with a time stamp matching the target extraction time point from the queue as the target video frame.

[0015] Optionally, the analysis instruction acquisition module is configured to acquire the video analysis instruction through a dedicated data channel, wherein the video analysis instruction is forwarded through a relay service, and the relay service ensures that the data channel is isolated from an external network through an isolation addressing mechanism.

[0016] Optionally, the video stream decoding module comprises: a video frame sequence generation module, configured to decode the video stream through a hardware acceleration decoder to generate a continuous video frame sequence comprising time stamps; and a video frame sequence storage module, configured to store the video frame sequence in the queue in chronological order.

[0017] Optionally, the video frame detection module comprises: a target detection algorithm module, configured to perform a target detection algorithm through the algorithm unit to identify targets and / or classify events in the target video frame; and a detection result generation module, configured to generate a detection result comprising target positions and / or event types.

[0018] Optionally, the system further comprises: a statistical report generation module, configured to perform time series analysis and / or event correlation on the detection result after the video frame detection module performs detection processing on the target video frame through the algorithm unit, and generate a statistical report; and a statistical report transmission module, configured to transmit the statistical report to the instruction generation system through an encrypted channel, wherein the statistical report is used to generate an updated video analysis instruction and / or adjust unmanned aerial vehicle operation parameters.

[0019] The embodiment of the application further discloses an electronic device, comprising: one or more processors; and one or more machine-readable media having instructions stored thereon that, when executed by the one or more processors, cause the electronic device to perform the unmanned aerial vehicle video processing method.

[0020] The embodiment of the application further discloses a computer-readable storage medium storing a computer program that causes a processor to perform the unmanned aerial vehicle video processing method.

[0021] The embodiment of the application has the following advantages:

[0022] The unmanned aerial vehicle video processing scheme provided by the embodiment of the application comprises the following steps: obtaining a video analysis instruction; configuring a video frame extraction parameter according to the video analysis instruction, wherein the video frame extraction parameter comprises a video frame extraction time interval; obtaining a video stream of an unmanned aerial vehicle, decoding and processing the video stream to obtain a video frame sequence, and storing the video frame sequence in a queue; extracting a target video frame corresponding to the video frame extraction time interval from the queue; and detecting and processing the target video frame by using an algorithm unit.

[0023] Compared with the prior art, the embodiment of the application has the following beneficial effects:

[0024] The embodiment of the application obtains a video analysis instruction and configures a video frame extraction parameter according to the video analysis instruction, allows dynamic adjustment of a frame extraction strategy according to different analysis requirements, avoids indiscriminate analysis of all video frames relative to a rigid mode of fixed full-frame processing, thereby reducing the number of frames input to the algorithm unit, reducing the demand for computing resources, and reducing repeated calculation and resource occupation. Decoding the video stream into a sequence and storing the sequence in a queue provides a buffering mechanism, makes subsequent extraction operations independent of the real-time inflow of the video, optimizes data stream management, avoids the bottleneck and delay accumulation of direct full-flow processing. The algorithm unit only detects and processes the target video frame, ensures that the detection resources are concentrated on high-value data, reduces the invalid overhead caused by redundant frames, improves the processing efficiency and optimizes the resource utilization, not only reduces the energy consumption, but also enhances the applicability of real-time analysis. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a step flow chart of an unmanned aerial vehicle video processing method according to an embodiment of the application;

[0026] Figure 2 is a principle schematic diagram of an unmanned aerial vehicle video processing scheme based on a visual internet according to an embodiment of the application;

[0027] Figure 3 is a structural block diagram of an unmanned aerial vehicle video processing system according to an embodiment of the application. DETAILED DESCRIPTION

[0028] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the application will be further described in detail below with reference to the drawings and specific embodiments.

[0029] The embodiment of the application provides a kind of unmanned aerial vehicle video processing scheme, by obtaining video analysis instruction, according to instruction configuration containing time interval video frame extraction parameter, unmanned aerial vehicle video stream is decoded as video frame sequence and is stored into queue, according to time interval extraction target video frame, by algorithm unit, target video frame is detected and handled, high-efficiency video data processing is realized.The scheme is by dynamically configuring frame extraction parameter and selectively processing target frame, optimizes the analysis process of high frame rate video stream, significantly reduces the computing overhead, improves processing efficiency and system response speed, is applicable to real-time monitoring, target tracking and the like scene.

[0030] Reference Figure 1 A step flow chart of a kind of unmanned aerial vehicle video processing method of the embodiment of the application is shown.The unmanned aerial vehicle video processing method can be applied to unmanned aerial vehicle control system, unmanned aerial vehicle video system, video processing system and the like system (hereinafter referred to as system).The unmanned aerial vehicle video processing method specifically can include the following steps:

[0031] Step 101, obtaining video analysis instruction.

[0032] Video analysis instruction contains the relevant information of processing task, for example target detection type, event recognition demand or algorithm selection requirement, is usually generated by instruction generation system (such as control platform) and is transmitted to system by encrypted special data channel.Obtaining process first involves establishing secure communication connection, and system verifies the identity authentication (such as digital signature or key matching) of instruction source to ensure the legality and integrity of instruction.Then, system parses received instruction data packet, and extracts task parameters therein, for example specified event detection type (such as target tracking or anomaly recognition) or algorithm priority.These parameters provide explicit guidance for subsequent video frame processing, ensure that system can carry out targeted operation according to specific task demand.

[0033] Step 102, according to video analysis instruction, configure video frame extraction parameter, and video frame extraction parameter includes video frame extraction time interval.

[0034] System determines the optimal video frame extraction time interval corresponding to the algorithm type specified in the instruction by querying the preset algorithm time consumption database.For example, some high-complexity algorithms may require longer frame interval to reduce processing load, while low-complexity algorithms may allow shorter interval to improve detection accuracy.The configuration process includes integrating the time interval with other parameters (such as frame resolution or extraction priority) to generate a complete set of video frame extraction parameters, and storing them in the system configuration table for subsequent steps.In addition, system may involve real-time calibration when configuring parameters, such as dynamically adjusting time interval according to frame rate or network delay of video stream, to ensure that the extracted frame is representative and real-time.After configuration, system will verify the validity of parameters to ensure that they meet the processing capacity of algorithm unit and task target requirements.

[0035] Step 103, obtain the video stream of the UAV, decode the video stream to obtain a video frame sequence, and store the video frame sequence in a queue.

[0036] The step of obtaining the video stream involves receiving real-time video data transmitted by the UAV through a network interface to meet monitoring or analysis requirements. The system uses a hardware-accelerated decoder to decode the video stream, parse compressed data, and restore it to a continuous video frame sequence, each frame containing a timestamp and image data. The decoding process ensures the integrity and time sequence of the frame sequence to support subsequent accurate frame extraction. The decoded video frame sequence is stored in a queue data structure that uses a first-in, first-out mechanism to temporarily cache frame data to address the real-time nature of the video stream and potential network fluctuations. The storage capacity of the queue is dynamically allocated based on system memory and task requirements, typically supporting hundreds of frames of cache to ensure that critical frames are not lost due to data overflow in high frame rate scenarios. During storage, the system assigns each frame a unique timestamp identifier to facilitate subsequent retrieval of target frames based on time intervals.

[0037] Step 104, extract the corresponding target video frame from the queue according to the video frame extraction time interval.

[0038] The system first reads the video frame extraction time interval (e.g., extract 1 frame per second or 1 frame every 0.5 seconds) from the configuration table and calculates the target extraction time point based on the current system time. For example, if the time interval is 1 second, the system will determine the corresponding target frame at each whole second (e.g., t = 1s, 2s, 3s). Then, the system traverses the queue to retrieve the video frame with the closest timestamp to the target extraction time point as the target video frame. The retrieval process typically uses an efficient indexing algorithm to reduce time complexity. The extracted target video frame is marked and removed from the queue, transmitted to the subsequent algorithm unit for processing, while the unselected frames may be retained for subsequent extraction or automatically cleared based on the queue capacity. The extraction process also needs to consider real-time requirements, and the system will dynamically adjust the retrieval frequency to match the video stream input rate to prevent queue accumulation or processing delays.

[0039] Step 105, detect and process the target video frame through the algorithm unit.

[0040] The algorithm unit usually performs complex image analysis operations on the target video frame based on a deep learning model. The detection process first involves preprocessing, in which the system normalizes the target frame to adapt to the input requirements of the algorithm unit. Then, the algorithm unit performs target detection on the frame data, identifies key objects in the image (such as people, vehicles, or abnormal events), and classifies events according to task requirements (such as intrusion detection, object tracking). The detection result generates structured data containing target position coordinates, event type, and confidence level to support subsequent processing or reporting. The processing of the algorithm unit is only for the target frame extracted in step 104, not for all video frames. This selective processing mechanism greatly reduces the amount of calculation and significantly reduces resource consumption compared to the one-by-one analysis of all frames in the background technology.

[0041] The embodiments of the present application obtain video analysis instructions and configure video frame extraction parameters accordingly, allowing dynamic adjustment of frame extraction strategies according to different analysis requirements. Compared with the rigid way of fixed full-frame processing, this avoids indiscriminate analysis of all video frames, thereby reducing the number of frames input to the algorithm unit, reducing the demand for computing resources, and reducing repeated calculations and resource occupation. Decoding the video stream into a sequence and storing it in a queue provides a buffering mechanism, making the subsequent extraction operation independent of the real-time video stream, optimizing data stream management, and avoiding the bottleneck and delay accumulation of direct full-flow processing. By detecting only the target video frame through the algorithm unit, the detection resources are concentrated on high-value data, reducing the invalid overhead caused by redundant frames, improving processing efficiency and optimizing resource utilization, not only reducing energy consumption, but also enhancing the applicability of real-time analysis.

[0042] In an exemplary embodiment of the present application, one implementation of configuring video frame extraction parameters according to video analysis instructions is as follows: parsing the video analysis instruction to obtain the specified algorithm type; querying the preset algorithm time consumption database to obtain the video frame extraction time interval corresponding to the algorithm type; and generating the video frame extraction parameter according to the video frame extraction time interval.

[0043] The video analysis instruction is usually a structured data packet generated by an instruction generation system, containing key information related to the task, such as the specified algorithm type (e.g., target detection algorithm or event classification algorithm). After receiving the video analysis instruction, the system processes it through a parsing module to extract the algorithm type field. This parsing process needs to ensure data integrity to prevent data corruption during transmission. The algorithm type obtained by parsing is then used to query a pre-set algorithm time consumption database. This database is a pre-constructed storage structure containing time consumption data for various algorithm types, such as average processing time based on historical testing or simulation calculations. The query operation can be implemented through Structured Query Language (SQL) statements or key-value pair matching, with the system inputting the algorithm type as the query key and outputting the corresponding video frame extraction time interval, such as setting a longer interval (e.g., 1 second) for high-time-consumption algorithms to reduce frame processing frequency and a shorter interval (e.g., 0.2 seconds) for low-time-consumption algorithms to improve detection accuracy. After the query is completed, the system generates video frame extraction parameters based on the obtained video frame extraction time interval. This generation process includes integrating the time interval with other auxiliary parameters (such as frame resolution adjustment or extraction priority) to form a parameter set and storing it in the system's configuration table. This configuration table can be in the form of an in-memory database or a file, facilitating fast access in subsequent steps.

[0044] This embodiment realizes intelligent and adaptive parameter configuration by dynamically parsing video analysis instructions and querying a pre-set algorithm time consumption database, significantly reduces computational resource consumption, adjusts video frame extraction time intervals based on algorithm types to avoid unnecessary frequent frame processing, improves the efficiency and accuracy of video processing, ensures that the system only analyzes high-value frames, enhances the flexibility of the system to support rapid switching between different task requirements, and optimizes overall resource utilization to reduce latency in high-frame-rate video stream scenarios and support real-time applications.

[0045] In an exemplary embodiment of the present application, one implementation of extracting the corresponding target video frame from the queue according to the video frame extraction time interval is as follows: calculating the target extraction time point according to the video frame extraction time interval; retrieving the video frame with a timestamp matching the target extraction time point from the queue as the target video frame.

[0046] The video frame extraction time interval is a previously configured parameter, such as 0.5 seconds or 1 second. The system uses the current system time as a reference to periodically calculate the next target extraction time point according to the interval. The calculation needs to consider time accuracy, usually using millisecond-level timestamps to avoid error accumulation. After the calculation is completed, the system retrieves the matching video frame from the queue. The queue is a data structure that stores a sequence of video frames, using a first-in, first-out mechanism, and each video frame has a timestamp assigned at the time of decoding. Retrieving from the queue involves traversal or index search, and the system compares the timestamp of each video frame with the difference between the target extraction time point, and selects the video frame with the smallest (closest) difference as the target video frame. If there are multiple equidistant frames, the system can prefer the frame with a larger timestamp to ensure real-time performance. This retrieval process can be optimized as a binary search algorithm. The extracted target video frame is then removed or marked from the queue to prevent repeated extraction, and the unextracted frames can be automatically cleared according to the queue capacity strategy.

[0047] This implementation achieves accurate target video frame extraction by calculating the target extraction time point and retrieving the video frame with the matching timestamp, reducing the number of redundant frames processed, thereby reducing the computational load of the algorithm unit; improves the real-time performance and response speed of video processing, as only representative frames are extracted to avoid delay accumulation; enhances the robustness of the system, maintaining stable extraction when the video stream fluctuates; and optimizes memory utilization, queue management reduces unnecessary data storage, supporting efficient operation in resource-constrained environments.

[0048] In an exemplary embodiment of the present application, one implementation of obtaining video analysis instructions is to obtain video analysis instructions through a dedicated data channel; wherein the video analysis instructions are forwarded by a transit service, and the transit service ensures that the data channel is isolated from the external network through an isolation addressing mechanism.

[0049] The dedicated data channel uses a secure transmission protocol to ensure the confidentiality and integrity of the data during transmission. The video analysis instructions are first generated by an instruction generation system and then forwarded by a transit service. The transit service acts as an intermediate node responsible for routing and verifying instructions, such as checking whether the instruction format conforms to the standard and adding forwarding metadata. This forwarding process can use a message queue mechanism to ensure reliable delivery of instructions. The transit service ensures that the data channel is isolated from the external network through an isolation addressing mechanism. For example, isolation addressing can be configured to allow communication only through specific ports and IPs, blocking broadcast data and unauthorized access. When the system obtains instructions, it first establishes a connection with the transit service, verifies identity through a handshake protocol, and then receives the forwarded data packet. After receiving, the system performs preliminary verification on the video analysis instructions, such as signature verification and content integrity check, to ensure that the instructions have not been tampered with.

[0050] The embodiment obtains video analysis instructions through a dedicated data channel and an isolated addressing mechanism, improves the security and privacy protection of data transmission, reduces external interference and attack risk, improves the reliability and efficiency of instruction acquisition, and ensures stable forwarding by the relay service, enhances the isolation of the system, prevents network pollution from affecting video processing, and optimizes the robustness of the overall process to support reliable operation in a complex network environment.

[0051] In an example embodiment of the present application, one implementation of storing the video frame sequence obtained by decoding the video stream into the queue is: decoding the video stream through a hardware accelerated decoder to generate a continuous video frame sequence containing timestamps; and storing the video frame sequence in the queue in chronological order.

[0052] The video stream is compressed data transmitted in real time by a UAV. The hardware accelerated decoder can efficiently handle decoding tasks, significantly reducing CPU load compared to software decoding. The decoding process involves parsing the header information of the video stream, decompressing frame data, and generating a continuous video frame sequence, each frame including pixel data and meta information. The system assigns a timestamp to each video frame, usually based on the internal clock of the video stream or system time, ensuring the temporal continuity of the sequence. The generation process needs to handle inter-frame dependencies to avoid decoding errors. After decoding is complete, the video frame sequence is stored in the queue in chronological order. The queue uses a ring buffer or linked list structure, supports dynamic expansion, and inserts frame data in ascending order of timestamps. The storage process includes memory management, such as pre-allocated buffers to prevent overflow, and regular cleanup of expired frames. The design of the queue allows concurrent access, for example, while the decoding thread writes, the extraction thread can read.

[0053] The embodiment generates a video frame sequence with timestamps through a hardware accelerated decoder and stores it in a queue in order, speeds up the decoding process, reduces CPU resource consumption, ensures the temporal accuracy and integrity of the video frame sequence, supports accurate extraction, optimizes data management, reduces latency and memory waste through queue storage, and improves the scalability of the system. It is suitable for efficient processing in multi-video stream scenarios.

[0054] In an example embodiment of the present application, one implementation of detecting the target video frame through the algorithm unit is: executing a target detection algorithm through the algorithm unit to identify targets and / or classify events in the target video frame; and generating a detection result containing target positions and / or event types.

[0055] The algorithm unit is the core computing module of the system, usually implemented based on a deep learning framework, supporting loading of a pre-trained model. The target detection algorithm is used to analyze target video frames, first preprocessing frame data, such as scaling resolution or enhancing contrast, to optimize input quality. Then, the algorithm performs target recognition, detecting objects of interest (such as vehicles or people) in the frame, outputting bounding boxes and confidence; meanwhile or optionally performing event classification, such as identifying motion anomalies or intrusion events. The process involves multi-layer neural network computation, such as convolutional layer feature extraction, pooling layer dimension reduction, and fully connected layer classification. The generated detection results include target positions (coordinates) and event types (labels), encapsulated in a structured format such as JSON.

[0056] This embodiment generates detection results by executing a target detection algorithm through the algorithm unit, improving the accuracy and speed of target recognition and event classification; reduces invalid computation, only processing target video frames to reduce resource consumption; enhances the intelligence of video processing, supporting complex scene analysis; and optimizes the output format, facilitating subsequent statistics and application.

[0057] In an exemplary embodiment of the present application, after the target video frames are detected and processed by the algorithm unit, one embodiment is to perform time series analysis and / or event correlation on the detection results, generate a statistical report; transmit the statistical report to the instruction generation system through an encrypted channel, and the statistical report is used to generate updated video analysis instructions and / or adjust the parameters of the unmanned aerial vehicle operation.

[0058] The detection results contain target positions and / or event types, the system first performs time series analysis, such as sorting the results by timestamp, calculating event duration or frequency; then performs event correlation, such as linking the same target in consecutive frames to form a trajectory. This analysis process can use statistical models such as moving average or clustering algorithms to identify trends such as high event periods. The generated statistical report includes aggregated data such as event counts and trend charts. This generation process ensures the comprehensiveness and readability of the report, possibly integrating visualization tools such as chart embedding. After completion, the statistical report is transmitted to the instruction generation system through an encrypted channel. The report is used to generate updated video analysis instructions (such as adjusting algorithm types) or adjust the parameters of the unmanned aerial vehicle operation (such as flight path).

[0059] This embodiment generates and transmits a statistical report through time series analysis and event correlation, realizes the deep utilization of detection results, improves the decision-making ability of the system; supports closed-loop optimization, report-driven instruction update reduces manual intervention; enhances data security, encrypted transmission prevents leakage; and improves overall application efficiency, adjusts operation parameters to adapt to dynamic scenarios.

[0060] Based on the above description of an embodiment of a method for processing video of an unmanned aerial vehicle, a scheme for processing video of an unmanned aerial vehicle based on a visual internet is introduced below. Refer to Figure 2, shows a principle schematic diagram of a UAV video processing scheme based on the visual networking according to an embodiment of the present application. The scheme involves the cooperative work of multiple functional modules, including an instruction generation system (such as a flight control platform), a coordination service system, a video processing system (such as a streaming service system), a frame extraction processing unit (such as a frame extraction module), an algorithm unit (such as an AI algorithm SDK), and an event processing system (such as an event platform). Figure 2 The instruction generation system (flight control platform) is connected with the video processing system (streaming service system) through the visual networking, and the video processing system internally includes a video stream receiving module, a decoding module, a queue management module, a frame extraction processing unit (frame extraction module), and an algorithm unit (such as an AI algorithm SDK), and finally outputs the detection result to the event processing system (event platform). The processing flow is as follows:

[0061] In actual application process, the operator issues UAV operation instructions to the UAV through the instruction generation system (flight control platform), the UAV receives the instructions, takes off smoothly, and adjusts the flight path according to the in-flight instructions, while using the camera to collect video pictures in real time to generate video IP streams. The video IP streams are transmitted to the coordination service system, which is responsible for forwarding them to the video processing system (streaming service system). The coordination service system encapsulates the video IP streams into the visual networking video protocol format (video V2V stream) to ensure the compatibility of the data in the visual networking.

[0062] The instruction generation system (flight control platform) generates different intelligent analysis instructions, including task requirements and algorithm type information, such as target detection or event classification tasks. The instruction is forwarded to the video processing system (streaming service system) through the coordination service system. The coordination service system acts as an intermediate node and realizes encrypted transmission of the instruction through the special data channel of the video live streaming network. The Advanced Encryption Standard 256-bit (AES-256) protocol is used to ensure data security, and the isolation addressing mechanism is used to prevent external network interference. The isolation addressing mechanism uses private IP address space and firewall rules to ensure that the instruction transmission channel is isolated from the external network. After receiving the intelligent analysis instruction, the video processing system (streaming service system) parses the instruction content, extracts the specified algorithm type, and configures the video frame extraction parameters through the frame extraction unit (frame extraction module). The configuration process involves querying a pre-set algorithm time consumption database to determine the video frame extraction time interval (such as 0.5 seconds or 1 second) matching the algorithm type, and generating video frame extraction parameters containing the time interval and storing them in the configuration table. The configuration table uses an in-memory database structure to support fast access and dynamic updating, such as adjusting the time interval according to the task priority.

[0063] The video processing system (streaming service system) receives the video stream of the unmanned aerial vehicle through the video live streaming network. The video stream is usually in the H.264 Video Coding Standard (H.264) or High Efficiency Video Coding (H.265) format, which is high-resolution and high-frame-rate data. The system uses a hardware-accelerated decoder (such as a Graphics Processing Unit (GPU) or an Application-Specific Integrated Circuit (ASIC)) to decode the video stream, parse the compressed data, and generate a sequence of video frames containing timestamps, which are then stored in a queue in chronological order. The queue uses a First In First Out (FIFO) structure, supports dynamic capacity management, and caches hundreds of frames of data to handle the high throughput of real-time video streams, and optimizes retrieval efficiency through timestamp indexing. The frame extraction unit (frame extraction module) calculates the target extraction time point based on the configured video frame extraction time interval. The video frame with the closest timestamp to the target extraction time point is retrieved from the queue as the target video frame, and the retrieval process uses a binary search algorithm to ensure efficient extraction. The extracted target video frame is then transmitted to the algorithm unit (AI algorithm SDK).

[0064] The algorithm unit (AI algorithm SDK) performs a target detection algorithm on the target video frames to identify targets (such as people, vehicles) and classify events (such as intrusion detection). The detection process includes preprocessing (such as resolution adjustment), feature extraction, and classification, generating detection results containing target position coordinates, event type, and confidence. The detection results are reported to the event processing system (event platform) for information statistics and analysis. The event processing system (event platform) performs time series analysis (such as event frequency calculation) or event correlation (such as target trajectory generation) on the results, generating statistical reports containing event statistics and trend analysis, packaged in Extensible Markup Language (XML). The statistical reports are transmitted to the instruction generation system (flight control platform) through an encrypted channel, used to generate updated video analysis instructions (such as adjusting algorithm types) or adjust unmanned aerial vehicle operation parameters (such as flight path or camera angle).

[0065] This embodiment significantly reduces the computational overhead of high-frame-rate video streams through encrypted transmission of the video over the Internet, dynamic parameter configuration of the frame extraction processing unit (frame extraction module), and accurate detection of the algorithm unit (AI algorithm SDK), as only target video frames are processed rather than full frames; improves processing efficiency and real-time performance, with queue management and frame extraction mechanisms reducing data processing delays; enhances system security, with dedicated channels and isolated addressing preventing data leakage; and optimizes resource utilization, supporting edge device deployment and multi-task scenario applications, suitable for real-time monitoring, target tracking, and other scenarios.

[0066] It should be noted that for the method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited to the order of the described actions, because according to the embodiments of the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present application.

[0067] Referring to Figure 3 , a structural block diagram of a UAV video processing system according to an embodiment of the present application is shown. The UAV video processing system can specifically include the following modules.

[0068] The analysis instruction acquisition module 31 is configured to acquire a video analysis instruction.

[0069] The extraction parameter configuration module 32 is configured to configure video frame extraction parameters according to the video analysis instruction, wherein the video frame extraction parameters include a video frame extraction time interval.

[0070] a video stream decoding module 33, configured to acquire a video stream of the UAV, decode the video stream to obtain a video frame sequence, and store the video frame sequence in a queue;

[0071] a video frame extraction module 34, configured to extract a target video frame corresponding to the video frame extraction time interval from the queue according to the video frame extraction time interval;

[0072] a video frame detection module 35, configured to detect the target video frame by using an algorithm unit.

[0073] In an example embodiment of the present application, the extraction parameter configuration module 32 comprises:

[0074] an algorithm type analysis module, configured to analyze the video analysis instruction to obtain a specified algorithm type;

[0075] a time interval query module, configured to query a preset algorithm time consumption database to obtain the video frame extraction time interval corresponding to the algorithm type;

[0076] an extraction parameter generation module, configured to generate the video frame extraction parameter according to the video frame extraction time interval.

[0077] In an example embodiment of the present application, the video frame extraction module 34 comprises:

[0078] an extraction time point calculation module, configured to calculate a target extraction time point according to the video frame extraction time interval;

[0079] a target video frame retrieval module, configured to retrieve a video frame with a time stamp matching the target extraction time point from the queue as the target video frame.

[0080] In an example embodiment of the present application, the analysis instruction acquisition module 31 is configured to acquire the video analysis instruction through a dedicated data channel;

[0081] The video analysis instruction is forwarded through a transfer service, and the transfer service ensures that the data channel is isolated from an external network through an isolated addressing mechanism.

[0082] In an example embodiment of the present application, the video stream decoding module 33 comprises:

[0083] a video frame sequence generation module, configured to decode the video stream through a hardware acceleration decoder to generate a continuous video frame sequence containing time stamps;

[0084] a video frame sequence storage module, configured to store the video frame sequence in the queue in chronological order.

[0085] In an exemplary embodiment of the present application, the video frame detection module 35 comprises:

[0086] a target detection algorithm module, configured to execute a target detection algorithm through the algorithm unit to perform target recognition and / or event classification on the target video frame;

[0087] a detection result generation module, configured to generate a detection result containing target position and / or event type.

[0088] In an exemplary embodiment of the present application, the system further comprises:

[0089] a statistical report generation module, configured to perform time sequence analysis and / or event correlation on the detection result after the video frame detection module 35 performs detection processing on the target video frame through the algorithm unit, and generate a statistical report;

[0090] a statistical report transmission module, configured to transmit the statistical report to an instruction generation system through an encrypted channel, and the statistical report is used to generate an updated video analysis instruction and / or adjust the unmanned aerial vehicle operation parameters.

[0091] For the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts are described in the method embodiment.

[0092] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between the embodiments can be referred to each other.

[0093] Those skilled in the art should understand that the embodiments of the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0094] The embodiments of the present application are described with reference to the flowchart illustrations and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0095] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal devices to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal devices, such that a series of operational steps are performed on the computer or other programmable terminal devices to produce a computer implemented process so that the instructions which execute on the computer or other programmable terminal devices provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0097] Although preferred embodiments of the present application have been described, those skilled in the art will be able to make additional modifications and variations to the embodiments without departing from the scope of the present application. Accordingly, the appended claims are intended to encompass all such modifications and variations as falling within the scope of the embodiments of the present application.

[0098] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0099] The above describes in detail the unmanned aerial vehicle video processing method and the unmanned aerial vehicle video processing system provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for processing video from unmanned aerial vehicles (UAVs), characterized in that, The method comprises: acquiring a video analysis instruction; configuring video frame extraction parameters according to the video analysis instruction, the video frame extraction parameters comprising a video frame extraction time interval; acquiring a video stream of a UAV, decoding the video stream to obtain a video frame sequence, and storing the video frame sequence in a queue; extracting a corresponding target video frame from the queue according to the video frame extraction time interval; detecting the target video frame by an algorithm unit.

2. The method of claim 1, wherein, The video frame extraction parameters configured according to the video analysis instruction comprise a video frame extraction time interval, which comprises: parsing the video analysis instruction to obtain a specified algorithm type; querying a preset algorithm time consumption database to obtain the video frame extraction time interval corresponding to the algorithm type; generating the video frame extraction parameters according to the video frame extraction time interval.

3. The method of claim 1, wherein, The corresponding target video frame is extracted from the queue according to the video frame extraction time interval, which comprises: calculating a target extraction time point according to the video frame extraction time interval; retrieving a video frame with a time stamp matching the target extraction time point from the queue as the target video frame.

4. The method of claim 1, wherein, The video analysis instruction is acquired, which comprises: acquiring the video analysis instruction through a dedicated data channel; wherein the video analysis instruction is forwarded through a transfer service, and the transfer service ensures that the data channel is isolated from the external network through an isolation addressing mechanism.

5. The method of claim 1, wherein, The video stream is decoded to obtain a video frame sequence, and the video frame sequence is stored in a queue, which comprises: decoding the video stream through a hardware acceleration decoder to generate a continuous video frame sequence containing a time stamp; storing the video frame sequence in the queue in chronological order.

6. The method of claim 1, wherein, The target video frame is detected by the algorithm unit, which comprises: performing target detection algorithm through the algorithm unit to identify targets and / or classify events in the target video frame; generating a detection result containing target positions and / or event types.

7. The method of claim 6, wherein, After the target video frame is detected by the algorithm unit, the method further comprises: performing time series analysis and / or event correlation on the detection result to generate a statistical report; transmitting the statistical report to an instruction generation system through an encrypted channel, the statistical report being used to generate an updated video analysis instruction and / or adjust UAV operation parameters.

8. A drone video processing system, comprising: The system comprises: an analysis instruction acquisition module for acquiring a video analysis instruction; an extraction parameter configuration module for configuring video frame extraction parameters according to the video analysis instruction, the video frame extraction parameters comprising a video frame extraction time interval; a video stream decoding module for acquiring a video stream of a UAV, decoding the video stream to obtain a video frame sequence, and storing the video frame sequence in a queue; a video frame extraction module for extracting a corresponding target video frame from the queue according to the video frame extraction time interval; a video frame detection module for detecting the target video frame by an algorithm unit.

9. An electronic device, comprising: comprises: one or more processors; and One or more machine readable media storing instructions thereon that, when executed by one or more processors, cause the electronic device to perform the method of claim 1-7.

10. A computer-readable storage medium, characterized in that, A computer program stored thereon that causes a processor to perform the method of claim 1-7.

Images