Video intelligent analysis method, system and equipment based on unmanned aerial vehicle and medium

Abstract

The embodiment of the invention provides an intelligent video analysis method and system based on an unmanned aerial vehicle, and the method comprises the steps: obtaining video stream data and a first intelligent analysis instruction corresponding to a first AI algorithm; based on the first intelligent analysis instruction, performing intelligent analysis processing on the video stream data; obtaining a second intelligent analysis instruction corresponding to the second AI algorithm; and if the first AI algorithm is different from the second AI algorithm, switching to perform intelligent analysis processing on the video stream data based on the second intelligent analysis instruction without interrupting the intelligent analysis processing. According to the embodiment of the invention, the corresponding AI algorithm is switched to in real time according to the new intelligent analysis instruction under the condition of not interrupting the intelligent analysis processing, so that the problems of video stream interruption and terminal picture rest or delay caused by algorithm switching in the background technology are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video processing, in particular to a video intelligent analysis method based on a UAV, a video intelligent analysis system based on a UAV, an electronic device and a computer readable storage medium. BACKGROUND

[0002] The existing streaming service system mainly relies on traditional video acquisition and transmission technology when processing video data, and can realize the receiving, distribution and display of video streams. However, for video streams collected by a UAV, the existing system has significant technical limitations.

[0003] In the existing scheme, when switching intelligent analysis algorithms during the flight of the UAV, the current analysis task needs to be stopped, and a new analysis task needs to be reconfigured and started. This operation causes a short interruption of video stream processing, and the terminal display screen appears static or delayed, which affects the user experience. SUMMARY

[0004] In view of the above problems, the present application embodiments are proposed to provide a video intelligent analysis method based on a UAV, a video intelligent analysis system based on a UAV, an electronic device and a computer readable storage medium which overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above problems, the present application embodiments disclose a video intelligent analysis method, which comprises:

[0006] obtaining video stream data and a first intelligent analysis instruction corresponding to a first AI algorithm;

[0007] based on the first intelligent analysis instruction, performing intelligent analysis processing on the video stream data;

[0008] obtaining a second intelligent analysis instruction corresponding to a second AI algorithm;

[0009] if the first AI algorithm is different from the second AI algorithm, switching to perform the intelligent analysis processing on the video stream data based on the second intelligent analysis instruction without interrupting the intelligent analysis processing.

[0010] Optionally, if the first AI algorithm is different from the second AI algorithm, switching to perform the intelligent analysis processing on the video stream data based on the second intelligent analysis instruction without interrupting the intelligent analysis processing, comprises:

[0011] when performing the intelligent analysis processing on each frame of image in the video stream data, identifying an AI algorithm corresponding to each frame of image;

[0012] When it is identified that the first AI algorithm corresponding to the current frame image is different from the second AI algorithm corresponding to the next frame image, the intelligent analysis processing is switched to the second AI algorithm without interruption.

[0013] Optionally, after the first intelligent analysis instruction corresponding to the first AI algorithm is obtained, the method further comprises:

[0014] All AI algorithms are preloaded into the video memory of the graphics processing unit.

[0015] Optionally, the switching to the intelligent analysis processing of the video stream data based on the second intelligent analysis instruction comprises:

[0016] The second AI algorithm preloaded into the video memory of the graphics processing unit is called to perform the intelligent analysis processing of the video stream data based on the second AI algorithm.

[0017] Optionally, the intelligent analysis processing of the video stream data based on the first intelligent analysis instruction comprises:

[0018] The first AI algorithm corresponding to the first intelligent analysis instruction is selected from an AI algorithm tool;

[0019] The target object in the video stream data is detected by using the first AI algorithm to generate an analysis result containing a target bounding box;

[0020] The target object includes at least one of the following: a person, a vehicle, a crowd, a ship, and fireworks.

[0021] Optionally, the detection of the target object in the video stream data by using the first AI algorithm to generate an analysis result containing a target bounding box comprises:

[0022] The first AI algorithm is executed to identify the target object specified by the first intelligent analysis instruction;

[0023] The target bounding box is drawn in the picture of the video stream data, and the target object is labeled by using the target bounding box to obtain the analysis result.

[0024] Optionally, after the target bounding box is drawn in the picture of the video stream data, and the target object is labeled by using the target bounding box to obtain the analysis result, the method further comprises:

[0025] The analysis result is sent to a terminal to enable the terminal to display the analysis result.

[0026] The embodiment of the application further discloses a video intelligent analysis system, and the system comprises:

[0027] a video instruction acquisition module, which is used for acquiring video stream data and a first intelligent analysis instruction corresponding to a first AI algorithm;

[0028] an intelligent analysis processing module, which is used for performing intelligent analysis processing on the video stream data based on the first intelligent analysis instruction;

[0029] an instruction acquisition module, which is used for acquiring a second intelligent analysis instruction corresponding to a second AI algorithm;

[0030] an algorithm processing switching module, which is used for switching to perform the intelligent analysis processing on the video stream data based on the second intelligent analysis instruction without interrupting the intelligent analysis processing, if the first AI algorithm is different from the second AI algorithm.

[0031] Optionally, the algorithm processing switching module comprises:

[0032] an AI algorithm identification module, which is used for identifying an AI algorithm corresponding to each frame of image in the video stream data when performing the intelligent analysis processing on the each frame of image;

[0033] an AI algorithm switching module, which is used for switching to perform the intelligent analysis processing on a next frame of image based on the second AI algorithm without interrupting the intelligent analysis processing, when it is identified that the first AI algorithm corresponding to the current frame of image is different from the second AI algorithm corresponding to the next frame of image.

[0034] Optionally, the system further comprises:

[0035] a preloading module, which is used for preloading all AI algorithms into a display memory of a graphics processing unit after the video instruction acquisition module acquires the first intelligent analysis instruction corresponding to the first AI algorithm.

[0036] Optionally, the algorithm processing switching module is used for calling the second AI algorithm preloaded into the display memory of the graphics processing unit, and performing the intelligent analysis processing on the video stream data based on the second AI algorithm.

[0037] Optionally, the intelligent analysis processing module comprises:

[0038] an AI algorithm selection module, which is used for selecting the first AI algorithm corresponding to the first intelligent analysis instruction from an AI algorithm tool;

[0039] an analysis result generation module, which is used for detecting a target object in the video stream data by using the first AI algorithm, and generating an analysis result containing a target identification frame.

[0040] The target object includes at least one of the following: a person, a vehicle, a crowd, a ship, and fireworks.

[0041] Optionally, the analysis result generation module comprises:

[0042] A target object identification module is configured to execute the first AI algorithm to identify the target object specified by the first intelligent analysis instruction.

[0043] A target object labeling module is configured to draw the target bounding box in the picture of the video stream data and label the target object by using the target bounding box to obtain the analysis result.

[0044] Optionally, the system further comprises:

[0045] An analysis result transmission module is configured to send the analysis result to a terminal after the target object labeling module draws the target bounding box in the picture of the video stream data and labels the target object by using the target bounding box to obtain the analysis result, so that the terminal displays the analysis result.

[0046] The embodiment of the application further discloses an electronic device, comprising: one or more processors; and one or more machine-readable media having instructions stored thereon that, when executed by the one or more processors, cause the electronic device to perform the video intelligent analysis method.

[0047] The embodiment of the application further discloses a computer-readable storage medium storing a computer program that causes a processor to execute the video intelligent analysis method.

[0048] The embodiment of the application has the following advantages:

[0049] The video intelligent analysis scheme based on the unmanned aerial vehicle provided by the embodiment of the application acquires video stream data and a first intelligent analysis instruction corresponding to a first AI algorithm, performs intelligent analysis processing on the video stream data based on the first intelligent analysis instruction, acquires a second intelligent analysis instruction corresponding to a second AI algorithm, and if the first AI algorithm is different from the second AI algorithm, switches to perform intelligent analysis processing on the video stream data based on the second intelligent analysis instruction without interrupting the intelligent analysis processing.

[0050] Compared with the prior art, the embodiment of the application has the following beneficial effects:

[0051] The embodiment of the present application solves the problem of video stream interruption, terminal picture freezing or delay caused by algorithm switching in the background technology by switching to the corresponding AI algorithm in real time according to the new intelligent analysis instruction without interrupting intelligent analysis processing. The continuous processing of the video stream is realized, the real-time and smoothness of the terminal display picture are ensured, the user experience in the dynamic scene is significantly improved, the response speed and system efficiency of the analysis task are improved, the high requirements for real-time and flexibility are met, and the application scene that needs to quickly adjust the analysis target is particularly suitable. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is a step flow chart of a video intelligent analysis method based on a UAV according to an embodiment of the present application;

[0053] Figure 2 is a flowchart of a UAV video intelligent analysis scheme based on a video network according to an embodiment of the present application;

[0054] Figure 3 is a structural block diagram of a video intelligent analysis system based on a UAV according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0056] The embodiment of the present application provides a video intelligent analysis scheme based on a UAV, acquires video stream data and intelligent analysis instructions of the UAV, uses an AI algorithm tool to perform real-time target detection and analysis on the video stream data, and realizes dynamic switching of different AI algorithms without interrupting processing. When the intelligent analysis instruction is first received, a plurality of AI algorithm models are preloaded into the GPU display memory, combined with the synchronous processing mechanism, the corresponding model is directly called and the new AI algorithm is applied in the next frame of image when the AI algorithm is switched, so as to avoid initialization overhead and picture interruption, and continuously display the picture containing the target bounding box on the terminal, which significantly improves the flexibility, real-time performance and user experience of the analysis task, and is suitable for dynamic monitoring and fast response scenes.

[0057] Reference Figure 1 , a step flow chart of a video intelligent analysis method based on a UAV according to an embodiment of the present application is shown. The video intelligent analysis method based on a UAV can be applied to a video intelligent analysis system, a video processing system and the like (hereinafter referred to as a system). The video intelligent analysis method based on a UAV can specifically include the following steps:

[0058] Step 101, acquiring video stream data and a first intelligent analysis instruction corresponding to a first AI algorithm.

[0059] The video stream data is usually a high-definition video sequence captured by a drone camera. At the same time, a first intelligent analysis instruction corresponding to a first artificial intelligence algorithm (AI algorithm) is received from a flight control platform. The first intelligent analysis instruction is issued by a user through an interactive interface of the flight control platform, and specifies the type of the first AI algorithm, such as an algorithm model for personnel detection, vehicle detection, or crowd gathering detection. The instruction content of the first intelligent analysis instruction includes but is not limited to algorithm identification, target object type (such as personnel, vehicle, or crowd), and analysis parameters (such as detection threshold or area range).

[0060] Step 102, based on the first intelligent analysis instruction, the video stream data is analyzed and processed intelligently.

[0061] An artificial intelligence algorithm software development kit (AI algorithm SDK) is called, and a first AI algorithm model corresponding to the first intelligent analysis instruction is selected from the AI algorithm SDK. The first AI algorithm model is trained for the target object (such as personnel, vehicle, or crowd) specified by the first intelligent analysis instruction, and can identify specific targets in the video stream data. The video stream data is decoded into a sequence of single-frame images, and each frame of image is input into the first AI algorithm model after pre-processing (such as size adjustment, normalization). The first AI algorithm model scans each frame of image, executes a target detection algorithm, identifies the position and category of the target object, and generates an analysis result containing a target identification box. The target identification box is drawn on the video frame in the form of a rectangular box, labeling the boundary and category label (such as "personnel" or "vehicle") of the target object.

[0062] Step 103, a second intelligent analysis instruction corresponding to a second AI algorithm is obtained.

[0063] The second intelligent analysis instruction is similar to the first intelligent analysis instruction and contains the identification of the second AI algorithm and the target object type (such as switching from person detection to crowd gathering detection). The acquisition process of the second intelligent analysis instruction is consistent with that of the first intelligent analysis instruction. The instruction content of the second intelligent analysis instruction includes but is not limited to the algorithm type, the target object (such as crowd gathering), and possible parameter adjustment (such as detection area or sensitivity). When receiving the second intelligent analysis instruction, the instruction content is parsed, its validity is verified, for example, whether the algorithm identification is in the model list supported by the AI algorithm SDK, and the second intelligent analysis instruction is recorded in the task queue with a timestamp for subsequent switching processing. To support dynamic switching, when receiving the second intelligent analysis instruction, the intelligent analysis task based on the first AI algorithm currently being performed will not be interrupted, but the new instruction will be marked as a to-be-processed state, and the switching will be performed at an appropriate time (such as the next frame processing). The system also needs to ensure the synchronization of the instruction and the video stream data, and avoid instruction processing delay or data misplacement through timestamp matching or queue management mechanism.

[0064] Step 104, if the first AI algorithm is different from the second AI algorithm, switching to intelligent analysis processing of the video stream data based on the second intelligent analysis instruction without interrupting the intelligent analysis processing.

[0065] When receiving the first intelligent analysis instruction, all supported AI algorithm models (including the models of the first AI algorithm and the second AI algorithm) are preloaded into the video memory of the graphics processor through the AI algorithm SDK. This preloading process is used to initialize the models for use, avoiding repeated loading overhead when switching. When receiving the second intelligent analysis instruction, the model corresponding to the second AI algorithm is selected from the preloaded model list according to the algorithm identification in the second intelligent analysis instruction. The system switches to the second AI algorithm model when processing the next frame of the video stream through a synchronous processing mechanism, starts to apply the new model for target detection, and does not terminate the current video stream processing task. After switching, the second AI algorithm model performs target detection on the video frame, generates a new target identification box (such as labeling the crowd gathering area), and superimposes the result into the video stream. The system ensures the coordination of instruction switching and video frame processing through multi-thread management, avoiding data processing conflict.

[0066] The embodiment of the present application solves the problem of video stream interruption and terminal picture freezing or delay caused by algorithm switching in the background technology by switching to the corresponding AI algorithm in real time according to the new intelligent analysis instruction without interrupting intelligent analysis processing. The continuous processing of the video stream is realized, the real-time and smoothness of the terminal display picture are ensured, the user experience in the dynamic scene is significantly improved, the response speed of the analysis task and the system efficiency are improved, the high requirements for real-time and flexibility are met, and the application scene that needs to quickly adjust the analysis target is particularly suitable.

[0067] In an exemplary embodiment of the present application, if the first AI algorithm is different from the second AI algorithm, an implementation of switching to intelligent analysis processing of the video stream data based on the second intelligent analysis instruction without interrupting intelligent analysis processing is that, when each frame of image in the video stream data is processed intelligently, the AI algorithm corresponding to each frame of image is identified; when it is identified that the first AI algorithm corresponding to the current frame of image is different from the second AI algorithm corresponding to the next frame of image, switching to intelligent analysis processing of the next frame of image based on the second AI algorithm without interrupting intelligent analysis processing.

[0068] In actual application process, a task queue can be maintained to record the intelligent analysis instruction corresponding to each frame of image and the AI algorithm identification associated therewith (for example, the first AI algorithm is used for personnel detection, and the second AI algorithm is used for crowd gathering detection). After the video stream data is decoded into a frame sequence, a time stamp is assigned to each frame of image, and the AI algorithm to be used for the current frame of image is determined according to the instruction information in the task queue. When it is detected that the second intelligent analysis instruction corresponding to the next frame of image specifies the second AI algorithm different from the first AI algorithm of the current frame, switching to the second AI algorithm without terminating the current intelligent analysis task. The whole switching process is realized through a synchronization processing mechanism, and the instruction switching and frame processing are managed in parallel by using the multi-threading technology, so that the switching time is controlled within one frame processing period, and the picture interruption or delay is avoided. The processed frame of image is re-encoded into a video stream and transmitted to the terminal for display, so that the picture continuity is maintained.

[0069] This implementation realizes seamless switching from the first AI algorithm to the second AI algorithm without interrupting intelligent analysis processing through the frame-level AI algorithm identification and synchronous switching mechanism, avoids the problem of video stream interruption and picture freezing caused by algorithm switching in the background technology, improves the response speed of the system to dynamic analysis requirements, and enhances the picture smoothness observed by the user on the terminal.

[0070] In an exemplary embodiment of the present application, after the first intelligent analysis instruction corresponding to the first AI algorithm is acquired, an implementation is that all AI algorithms are preloaded into the video memory of the graphics processing unit.

[0071] In actual application, after receiving the first intelligent analysis instruction, all supported AI algorithm models (including the first AI algorithm, the second AI algorithm and other possible algorithm models) are loaded into the video memory of the graphics processor through the AI algorithm SDK. These AI algorithm models are usually based on a deep learning framework, including model parameters, weights and network structures. The preloading process involves transferring model data from system storage (such as a hard disk or memory) to the video memory of the graphics processor, and initializing the computing environment of the model, such as allocating video memory space, configuring a computation graph and optimizing inference parameters. This process avoids repeated loading and initialization overhead when switching AI algorithms by loading all models at once. The preloaded models are stored in the video memory of the graphics processor and remain in standby state, and the calling authority of each model is managed through a task queue, so that the corresponding model can be quickly selected for analysis and processing according to the received intelligent analysis instruction (such as the first or second intelligent analysis instruction).

[0072] This embodiment eliminates the initialization delay when switching algorithms by preloading all AI algorithm models into the video memory of the graphics processor after receiving the first intelligent analysis instruction, and overcomes the problem of video stream interruption caused by reloading models in the background art. Preloading all models in advance in the video memory of the graphics processor enables the system to immediately call the corresponding model when receiving the second intelligent analysis instruction, shortens the switching time to within the frame interval, and thus ensures the real-time processing of video stream and the continuity of terminal display.

[0073] In an exemplary embodiment of the present application, one implementation of switching to intelligent analysis and processing of video stream data based on the second intelligent analysis instruction is to call the second AI algorithm preloaded into the video memory of the graphics processing unit, and perform intelligent analysis and processing of the video stream data based on the second AI algorithm.

[0074] After receiving the second intelligent analysis instruction, if it is detected that the second AI algorithm is different from the currently used first AI algorithm, the second AI algorithm model corresponding to the second intelligent analysis instruction is quickly called from the preloaded AI algorithm model set through the AI algorithm SDK. Previously, as described in the foregoing embodiments, all AI algorithm models are preloaded into the video memory of the graphics processor when the first intelligent analysis instruction is received, and the model parameters and weights are initialized and in standby state. The calling process involves locating the second AI algorithm model in the video memory of the graphics processor according to the algorithm identifier of the second intelligent analysis instruction. The located second AI algorithm model performs target detection on the decoded video stream data frame sequence and identifies the specified target object (such as crowd gathering), and generates an analysis result containing a target bounding box.

[0075] The embodiment realizes fast and seamless algorithm switching by calling the second AI algorithm model preloaded in the video memory of the graphics processor, and overcomes the problems of video stream interruption and picture freezing caused by algorithm switching in the background technology. The preloaded model eliminates the initialization overhead during switching, and the direct calling mechanism compresses the switching time into the interframe gap, ensuring the continuity and real-time performance of video stream processing.

[0076] In an exemplary embodiment of the present application, based on the first intelligent analysis instruction, one implementation of intelligent analysis processing of video stream data is as follows: selecting a first AI algorithm corresponding to the first intelligent analysis instruction from the AI algorithm tool; using the first AI algorithm to detect target objects in the video stream data to generate an analysis result containing a target bounding box; wherein the target objects include at least one of the following: personnel, vehicles, crowds, ships, fireworks.

[0077] In actual application, after receiving the first intelligent analysis instruction, the AI algorithm SDK selects the first AI algorithm corresponding to the first intelligent analysis instruction from the algorithm model set it supports. The first AI algorithm is trained according to the target object type (such as personnel, vehicles, crowds, ships, or fireworks) specified by the first intelligent analysis instruction, and can identify specific targets in video frames. The system decodes the unmanned aerial vehicle video stream data into a sequence of single-frame images, and inputs each preprocessed image (such as size adjustment, color normalization) into the first AI algorithm. The first AI algorithm scans each image and performs target detection to identify the location and category of target objects, and generates an analysis result containing a target bounding box on the image. The target bounding box is drawn in the form of a rectangular box, labeling the boundary and category label (such as "personnel" or "vehicle") of the target object, and can be attached with a confidence score to indicate the detection accuracy.

[0078] The embodiment overcomes the problem of insufficient real-time performance of video analysis in the background technology by accurately selecting the first AI algorithm from the AI algorithm SDK and using it to detect target objects in the video stream data to generate an analysis result containing a target bounding box. The detection capability for multiple target objects (such as personnel, vehicles, crowds, ships, and fireworks) enhances the applicability of the system.

[0079] In an exemplary embodiment of the present application, one implementation of using the first AI algorithm to detect target objects in the video stream data to generate an analysis result containing a target bounding box is as follows: executing the first AI algorithm to identify the target objects specified by the first intelligent analysis instruction; drawing a target bounding box in the picture of the video stream data, and using the target bounding box to label the target objects to obtain the analysis result.

[0080] Specifically, when executing the first AI algorithm, the video stream data is decoded into a sequence of single-frame images, each of which is pre-processed and input into the algorithm model of the first AI algorithm. The first AI algorithm scans each frame of image, identifies the target object specified by the first intelligent analysis instruction through feature extraction and classification operations, determines its position and category, and generates a corresponding confidence score to indicate the detection accuracy. After the identification is completed, a target bounding box is drawn in the video frame, usually a rectangular box, which frames the boundary of the target object, and is attached with a category label and confidence information, generating an analysis result containing the target bounding box. The drawing process is realized through image processing technology, ensuring that the bounding box is accurately aligned with the target object, and the color and style of the box (such as a red solid line box) can be distinguished according to the system configuration. The processed frame image is re-encoded into a video stream, retaining the original frame rate and resolution.

[0081] This embodiment overcomes the problem of insufficient real-time performance of video analysis in the prior art by accurately identifying target objects through the execution of the first AI algorithm and drawing target bounding boxes in video frames to generate intuitive analysis results. The drawing of the target bounding box provides clear visual output, enabling end users to intuitively understand the type and location of the target object. This not only enhances the applicability of the system in various scenarios (such as personnel monitoring and vehicle tracking), but also ensures the real-time performance and smoothness of the terminal screen through continuous video stream processing, improving the user experience.

[0082] In an exemplary embodiment of the present application, after drawing the target bounding box in the frame of the video stream data and labeling the target object with the target bounding box to obtain the analysis result, one embodiment is to send the analysis result to the terminal to enable the terminal to display the analysis result.

[0083] After the detection of the target object is completed, an analysis result containing the target bounding box is generated, that is, a rectangular box is drawn on the video frame to label the target object (such as a person, a vehicle, a crowd, a ship, or fireworks), and a category label and a confidence information are attached. The processed video frame is re-encoded into video stream data in a compatible encoding format with the original video stream, retaining the original resolution and frame rate to ensure picture quality and smoothness. Subsequently, the video stream data containing the target bounding box is transmitted to the terminal through the Real-Time Transport Protocol (RTP) or the like of the vision Internet. The terminal can be a display device (such as a monitoring screen or a mobile device) of the flight control platform, or other display nodes in the vision Internet. The transmission process utilizes the high bandwidth and low delay characteristics of the vision Internet to ensure real-time arrival of the video stream data. After the terminal receives the video stream data, it decodes and renders the picture containing the target bounding box, which is displayed to the user, and the picture clearly presents the labeling information of the target object (such as a red rectangular box indicating “person” or “vehicle”). The system can manage the encoding and transmission tasks through a multi-thread mechanism to ensure uninterrupted processing of the video stream data and maintain the continuity of the picture.

[0084] This embodiment overcomes the defect in the prior art that the picture on the terminal is static or delayed due to algorithm switching or processing interruption by sending the analysis result containing the target bounding box to the terminal in real time and displaying it, so that the terminal can present a clear labeled picture in real time. This not only visually displays the position and category of the target object, improving the user's perception of dynamic scenes (such as real-time monitoring or event response), but also enhances the user experience by maintaining picture smoothness.

[0085] Based on the above description of the video intelligent analysis method embodiment, the following introduces a vision Internet-based unmanned aerial vehicle video intelligent analysis scheme. Referring to Figure 2 , a flowchart of a vision Internet-based unmanned aerial vehicle video intelligent analysis scheme of an embodiment of the application is shown. The scheme involves a flight control platform, a unmanned aerial vehicle, an assistance service system, a streaming media service system, and a terminal.

[0086] In actual application process, the operator issues the unmanned aerial vehicle operation instruction to the unmanned aerial vehicle through the flight control platform, the unmanned aerial vehicle receives the instruction, takes off smoothly, adjusts the flight path according to the aerial flight instruction, and generates a video IP stream by using the camera to collect video pictures in real time. The flight control platform also issues an intelligent analysis instruction to the assistance service system, for example, a "person and vehicle detection instruction", which requires detection of personnel and vehicles in the video IP stream. The video IP stream is transmitted to the coordination service system, which is responsible for forwarding it to the streaming media service system, and the intelligent analysis instruction is also transmitted to the streaming media service system. The coordination service system encapsulates the video IP stream and the intelligent analysis instruction into a video V2V stream in the form of a video V2V stream, ensuring the compatibility of data in the video networking.

[0087] Next, the streaming media service system receives the video V2V stream and the intelligent analysis instruction, and calls the integrated AI algorithm SDK to start the intelligent analysis function. The AI algorithm SDK selects the appropriate AI model according to the intelligent analysis instruction, analyzes the video stream data in real time, identifies personnel and vehicles in the picture, and draws a target bounding box around the detected target object. The analysis result is output in the form of a video stream containing a target bounding box, and the streaming media service system then transmits the video data processed by the intelligent analysis to the video networking terminal through the video networking. The terminal receives the data and displays the analysis result on the screen, showing the personnel and vehicles in the picture marked with a rectangular box, and the label inside the box to clearly indicate the category.

[0088] The intelligent analysis and processing of the video stream data of the unmanned aerial vehicle in the embodiment of the application can include various types of intelligent analysis, such as target detection of people and vehicles, target detection of crowd gathering, etc. For example, the flight control platform generates and issues a first intelligent analysis instruction to the coordination service system through a user interaction interface, which is then forwarded to the streaming media service system by the coordination service system. The first intelligent analysis instruction can specify a first AI algorithm, such as a personnel detection algorithm. The streaming media service system receives the video stream data of the unmanned aerial vehicle and the first intelligent analysis instruction, creates an intelligent analysis task, and then calls the AI algorithm SDK to select the first AI algorithm model, decodes the video stream data, and generates a sequence of single-frame images. In the intelligent analysis task, each frame of image is input into the first AI algorithm model after preprocessing, target detection is performed, personnel position is identified, and analysis results containing target bounding boxes are generated, personnel positions are marked and category labels (such as "personnel") are attached.

[0089] When receiving the first intelligent analysis instruction, the streaming media service system preloads all supported AI algorithm models (including personnel detection, vehicle detection, crowd gathering detection, etc.) to the GPU memory (Graphics Processing Unit Memory, GPU for short) through the AI algorithm SDK.

[0090] The model parameters and weights are initialized and stored in a standby state. This process ensures that subsequent switching does not need to repeat the loading of the model. Subsequently, the user issues a second intelligent analysis instruction through the flight control platform, specifying a second AI algorithm (such as a crowd gathering detection model). After the streaming service system detects that the second AI algorithm is different from the first AI algorithm, it calls the preloaded second AI algorithm model when processing the next frame of image through a synchronous processing mechanism. The switching process is completed within the frame gap, without interrupting the current intelligent analysis task. The streaming service system synchronously processes each frame of video stream data, manages instruction switching and frame processing through a multi-thread mechanism, and the second AI algorithm model generates a new target bounding box (such as marking a crowd gathering area). The processed video frame is re-encoded into a video stream and transmitted to the terminal display device through the video live streaming, and the picture containing the target bounding box is displayed in real time, maintaining continuity and smoothness.

[0091] The embodiment preloads the AI algorithm model to the GPU video memory and uses a synchronous processing mechanism to achieve seamless connection of dynamic algorithm switching, avoiding initialization overhead and video stream interruption during algorithm switching. The terminal display updates the target bounding box in real time, meets the dynamic scene requirements, improves the response speed and user experience of the system, and is suitable for real-time monitoring and emergency response scenes.

[0092] It should be noted that, for the method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited to the action sequence described, because according to the embodiments of the present application, certain steps can be performed in other sequences or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0093] Referring to Figure 3 , a structure block diagram of a video intelligent analysis system based on a UAV is shown. The video intelligent analysis system based on a UAV can specifically include the following modules.

[0094] The video instruction acquisition module 31 is configured to acquire video stream data and a first intelligent analysis instruction corresponding to a first AI algorithm;

[0095] The intelligent analysis processing module 32 is configured to perform intelligent analysis processing on the video stream data based on the first intelligent analysis instruction;

[0096] The instruction acquisition module 33 is configured to acquire a second intelligent analysis instruction corresponding to a second AI algorithm;

[0097] The algorithm processing switching module 34 is configured to, if the first AI algorithm is different from the second AI algorithm, switch to performing the intelligent analysis processing on the video stream data based on the second intelligent analysis instruction without interrupting the intelligent analysis processing.

[0098] In an example embodiment of the present application, the algorithm processing switching module 34 comprises:

[0099] An AI algorithm identification module is configured to, when performing the intelligent analysis processing on each frame of image in the video stream data, identify an AI algorithm corresponding to each frame of image.

[0100] An AI algorithm switching module is configured to, when identifying that the first AI algorithm corresponding to a current frame of image is different from the second AI algorithm corresponding to a next frame of image, switch to performing the intelligent analysis processing on the next frame of image based on the second AI algorithm without interrupting the intelligent analysis processing.

[0101] In an example embodiment of the present application, the system further comprises:

[0102] A preloading module is configured to, after the video instruction acquisition module 31 acquires the first intelligent analysis instruction corresponding to the first AI algorithm, preloading all AI algorithms into the video memory of the GPU.

[0103] In an example embodiment of the present application, the algorithm processing switching module 34 is configured to call the second AI algorithm preloaded into the video memory of the GPU, and perform the intelligent analysis processing on the video stream data based on the second AI algorithm.

[0104] In an example embodiment of the present application, the intelligent analysis processing module 32 comprises:

[0105] An AI algorithm selection module is configured to select the first AI algorithm corresponding to the first intelligent analysis instruction from AI algorithm tools.

[0106] An analysis result generation module is configured to detect a target object in the video stream data by using the first AI algorithm, and generate an analysis result containing a target bounding box.

[0107] The target object includes at least one of the following: a person, a vehicle, a crowd, a ship, and fireworks.

[0108] In an example embodiment of the present application, the analysis result generation module comprises:

[0109] A target object identification module is configured to execute the first AI algorithm, and identify the target object specified by the first intelligent analysis instruction.

[0110] The target object labeling module is configured to draw the target identification frame in the picture of the video stream data and label the target object by using the target identification frame to obtain the analysis result.

[0111] In an exemplary embodiment of the present application, the system further comprises:

[0112] The analysis result transmission module is configured to send the analysis result to the terminal after the target object labeling module draws the target identification frame in the picture of the video stream data and labels the target object by using the target identification frame to obtain the analysis result, so that the terminal displays the analysis result.

[0113] For the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts are described in the part of the method embodiment.

[0114] Each embodiment in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between the embodiments can be referred to each other.

[0115] Those skilled in the art should understand that the embodiments of the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0116] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams according to the method, terminal device (system), and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a machine that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The apparatus that realizes the functions specified in one or more flows and / or blocks. Figure 1 The apparatus that realizes the functions specified in one or more flows and / or blocks.

[0117] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0119] Although preferred embodiments of the present application have been described, those skilled in the art will be able to make additional changes and modifications thereto without departing from the scope of the present application. Accordingly, it is intended to embrace all such changes and modifications as fall within the scope of the present application.

[0120] Finally, it should be noted that the terms "first" and "second" and the like are used merely to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises", "comprising", or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0121] The above provides a kind of video intelligent analysis method based on unmanned aerial vehicle and a kind of video intelligent analysis system based on unmanned aerial vehicle provided by the present application, are introduced in detail, the principle and implementation mode of the present application are described in this paper by specific example, the above example is only for helping to understand the method of the present application and its core idea;For the general technical personnel in the art, according to the idea of the present application, there will be changes in specific implementation mode and application range, as described above, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A video intelligent analysis method, characterized in that, The method includes: Acquire video stream data and the first intelligent analysis command corresponding to the first AI algorithm; Based on the first intelligent analysis instruction, the video stream data is subjected to intelligent analysis and processing; Obtain the second intelligent analysis command corresponding to the second AI algorithm; If the first AI algorithm is different from the second AI algorithm, then without interrupting the intelligent analysis and processing, the intelligent analysis and processing of the video stream data is switched to be performed based on the second intelligent analysis instruction.

2. The method according to claim 1, characterized in that, If the first AI algorithm differs from the second AI algorithm, then without interrupting the intelligent analysis and processing, the process switches to performing the intelligent analysis and processing on the video stream data based on the second intelligent analysis instruction, including: When performing the intelligent analysis and processing on each frame of the video stream data, the AI ​​algorithm corresponding to each frame of the image is identified. When it is detected that the first AI algorithm corresponding to the current frame image is different from the second AI algorithm corresponding to the next frame image, the intelligent analysis and processing is switched to be performed on the next frame image based on the second AI algorithm without interrupting the intelligent analysis and processing.

3. The method according to claim 1, characterized in that, After obtaining the first intelligent analysis instruction corresponding to the first AI algorithm, the method further includes: All AI algorithms are preloaded into the graphics processing unit's video memory.

4. The method according to claim 3, characterized in that, The switching to perform intelligent analysis processing on the video stream data based on the second intelligent analysis instruction includes: The second AI algorithm, preloaded into the video memory of the graphics processing unit, is invoked to perform the intelligent analysis and processing on the video stream data based on the second AI algorithm.

5. The method according to claim 1, characterized in that, The intelligent analysis and processing of the video stream data based on the first intelligent analysis instruction includes: Select the first AI algorithm from the AI ​​algorithm tools that corresponds to the first intelligent analysis instruction; The first AI algorithm is used to detect target objects in the video stream data, and an analysis result containing target bounding boxes is generated. The target objects include at least one of the following: personnel, vehicles, crowds, ships, and fireworks.

6. The method according to claim 5, characterized in that, The step of using the first AI algorithm to detect target objects in the video stream data and generating analysis results containing target bounding boxes includes: Execute the first AI algorithm to identify the target object specified by the first intelligent analysis instruction; The target identification box is drawn in the video stream data, and the target object is marked using the target identification box to obtain the analysis result.

7. The method according to claim 6, characterized in that, After drawing the target bounding box in the video stream data and using the target bounding box to label the target object to obtain the analysis result, the method further includes: The analysis results are sent to the terminal so that the terminal can display the analysis results.

8. A video intelligent analysis system, characterized in that, The system includes: The video instruction acquisition module is used to acquire video stream data and the first intelligent analysis instruction corresponding to the first AI algorithm; The intelligent analysis and processing module is used to perform intelligent analysis and processing on the video stream data based on the first intelligent analysis instruction; The instruction acquisition module is used to acquire the second intelligent analysis instruction corresponding to the second AI algorithm. The algorithm processing switching module is used to switch to intelligent analysis processing of the video stream data based on the second intelligent analysis instruction without interrupting the intelligent analysis processing if the first AI algorithm is different from the second AI algorithm.

9. An electronic device, characterized in that, include: One or more processors; and One or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, cause the electronic device to perform the video intelligent analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The stored computer program causes the processor to execute the video intelligent analysis method as described in any one of claims 1 to 7.

Images