Real-time video stream analysis method based on edge calculation

By preprocessing and analyzing video streams on edge devices, the problem of video stream quality reduction caused by bandwidth limitations in traditional video analysis technology is solved, and more efficient and faster video stream analysis is achieved, suitable for application scenarios that require real-time response .

CN119006986APending Publication Date: 2024-11-22GREATWALL DIGITAL
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411020492.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Traditional video analysis technology requires streaming all videos to the cloud for processing, resulting in bandwidth limitations, video stream quality and resolution, affecting the accuracy of the analysis.

Method used

The real-time video stream analysis method based on edge computing is adopted to obtain video streaming data through the streaming media protocol, perform preprocessing and analysis on the edge device, reduce network transmission, improve efficiency and response speed.

Benefits of technology

It improves the stability and economy of the system, significantly reduces network traffic costs, enhances robustness and response speed, and is suitable for application scenarios that require real-time response such as intelligent traffic and security monitoring.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006986A_ABST
    Figure CN119006986A_ABST
Patent Text Reader

Abstract

The invention provides a real-time video stream analysis method based on edge calculation. The method comprises the following steps: acquiring real-time video stream data of a target area from a video source by using a streaming media protocol; sending the video stream data to the target edge device; preprocessing the video stream data at the target edge device to obtain a plurality of image frames; inputting the image frame into a trained analysis model to obtain an analysis result; and corresponding feedback is made according to an analysis result. According to the scheme, the processing and analysis task of the video stream data is deployed on the edge computing device to be executed instead of being transmitted to a remote cloud server through a network to be processed. By optimizing the data processing and analysis process, the stability and economical efficiency of the system are improved, and the business processing efficiency and response speed are remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of video analysis, and in particular relates to a real-time video stream analysis method based on edge computing. Background Art

[0002] Video analysis is a technology that identifies and understands the content in video streams, such as image classification, target detection, target tracking, etc. Video analysis has a wide range of applications in traffic monitoring, personnel tracking, and event detection.

[0003] For video analysis, all video streams traditionally need to be transmitted to the cloud for processing, which conflicts with the bandwidth limitations of some deployment scenarios. That is, it is impossible to upload video streams from all cameras, and traditional video stream compression / sampling technology will reduce the quality and resolution of the video stream, thereby affecting the accuracy of video analysis. Summary of the invention

[0004] The purpose of the embodiments of this specification is to provide a real-time video stream analysis method based on edge computing.

[0005] The embodiment of the present application is implemented in the following manner:

[0006] In a first aspect, the present application provides a real-time video stream analysis method based on edge computing, the method comprising:

[0007] Using streaming media protocol to obtain real-time video stream data of the target area from the video source;

[0008] Sending the video stream data to a target edge device;

[0009] Preprocessing the video stream data on the target edge device to obtain a number of image frames;

[0010] Inputting the image frame into a trained analysis model to obtain an analysis result;

[0011] Provide corresponding feedback based on the analysis results.

[0012] In one embodiment, a method for determining a target edge device includes:

[0013] The target edge device is determined according to at least one of network delay time, real-time load, performance indicators of each edge device, and network topology.

[0014] In one embodiment, the video stream data is preprocessed to obtain a plurality of image frames, including:

[0015] Extracting consecutive frames from the video stream data;

[0016] adjusting the resolution of the frames to a uniform size;

[0017] Performing an enhancement operation on the frames after the uniform size to obtain enhanced frames;

[0018] The enhanced frames are converted into the format required by the algorithm to obtain the plurality of image frames.

[0019] In one embodiment, the trained analysis model includes a face recognition model and / or a target detection model.

[0020] In one embodiment, providing corresponding feedback according to the analysis result includes:

[0021] The analysis results are fed back to the user and / or the automatic control system.

[0022] In one of the embodiments, if the analysis result meets a preset condition, a preset response mechanism is triggered.

[0023] In one embodiment, the method further comprises:

[0024] The analysis results and corresponding timestamps are sent to a database.

[0025] In a second aspect, the present application provides a real-time video stream analysis device based on edge computing, the device comprising:

[0026] An acquisition module is used to acquire real-time video stream data of a target area from a video source using a streaming media protocol;

[0027] A sending module, used to send the video stream data to a target edge device;

[0028] A preprocessing module, used for preprocessing the video stream data at the target edge device to obtain a plurality of image frames;

[0029] A processing module, used for inputting the image frame into a trained analysis model to obtain an analysis result;

[0030] The feedback module is used to make corresponding feedback according to the analysis result.

[0031] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the real-time video stream analysis method based on edge computing as in the first aspect is implemented.

[0032] In a fourth aspect, the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the real-time video stream analysis method based on edge computing as in the first aspect.

[0033] It can be seen from the technical solution provided in the above embodiments of this specification that:

[0034] The processing and analysis tasks of video stream data are deployed on edge computing devices instead of being transmitted to remote cloud servers for processing over the network. By optimizing the data processing and analysis process, not only the stability and economy of the system are improved, but also the efficiency and response speed of business processing are significantly improved. The advantages of this method are reflected in many aspects:

[0035] First, this method effectively avoids business processing interruptions caused by network instability or failure. Since data processing and analysis are completed on edge devices, there is no need to rely on a stable network connection, which significantly enhances the robustness of the system. At the same time, this processing method also significantly reduces the network traffic costs generated by data transmission and reduces the burden on cloud service providers in video streaming computing.

[0036] Secondly, edge computing enables the system to analyze video streams faster. Since data processing and analysis tasks are performed near the data source, data transmission delays in the network are reduced, resulting in faster response speeds and higher business processing efficiency. This advantage is particularly important for application scenarios that require real-time responses, such as smart transportation and security monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 A flowchart of a real-time video stream analysis method based on edge computing provided in this application;

[0039] Figure 2 A schematic diagram of the structure of a real-time video stream analysis device based on edge computing provided in this application;

[0040] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0042] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0043] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present application description without departing from the scope or spirit of the present application. Other embodiments derived from the present application description will be apparent to those skilled in the art. The present application description and examples are exemplary only.

[0044] The words “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.

[0045] Reference Figure 1 , which shows a flow chart of a real-time video stream analysis method based on edge computing provided by an embodiment of the present application. It is understandable that the application scenario of the real-time video stream analysis method based on edge computing is a scenario that requires real-time response, such as intelligent transportation, security monitoring, etc. Deploy video sources such as high-definition cameras according to the scenario requirements of the application scenario to ensure coverage of the required monitoring area, and deploy edge devices near the video source (such as near the camera), and select edge servers with high-performance GPUs to ensure sufficient computing power. It is also understandable that the edge server should be as close as possible to the source of data generation, that is, the video source, to reduce data transmission delays. In a multi-layer edge computing architecture, the edge server may be located in the core layer, and the edge device is located in the access layer to form a hierarchical processing structure. A high-speed network connection, such as a 10GbE or higher-speed network interface, can be used between the edge device and the edge server; to ensure high availability, multiple physical paths can be used to connect the edge device and the edge server to prevent single point failures. Video stream data can be directly transmitted from the edge device to the edge server for processing. In a specific scenario, the edge device can first send the video stream data to a local small edge node, and then forward it to the edge server.

[0046] like Figure 1 As shown, the real-time video stream analysis method based on edge computing provided in the embodiment of the present application includes:

[0047] S110: Acquire real-time video stream data of a target area from a video source using a streaming media protocol.

[0048] Specifically, the video source may be a camera or other video sources, such as a network camera.

[0049] Streaming media protocols are used to transmit audio, video and data on the Internet. Commonly used streaming media protocols include: RTP (Real-time Transport Protocol) and RTCP (Real-time Transport Control Protocol), RTSP (Real-time Streaming Protocol), HLS (HTTP Live Streaming), etc.

[0050] The target area is the area where real-time video stream analysis is to be performed.

[0051] S120: Send the video stream data to a target edge device.

[0052] The method for determining the target edge device includes: determining the target edge device based on at least one of network delay time, real-time load, performance indicators of each edge device, and network topology.

[0053] Specifically, determining the target edge device according to the network delay time may include:

[0054] Measure the network latency by performing a ping test or using tracert / traceroute tools to send requests from the video source node to multiple edge computing devices. Select the edge device with the shortest response time as the target edge device for video streaming.

[0055] Determining the target edge device according to the real-time load may include:

[0056] Regularly check the CPU, memory, and disk I / O load of each edge device. Based on the real-time load situation, determine the edge device with the lowest current load as the target edge device and dynamically redirect the video stream to the target edge device.

[0057] Determining the target edge device according to the performance indicator of each edge device may include:

[0058] Track performance indicators such as response time and throughput of applications running on edge devices in real time, and select edge devices with the best performance indicators as target edge devices to ensure optimal video analysis performance.

[0059] Determining the target edge device according to the network topology may include:

[0060] Analyze the network topology, understand the network hierarchy and connection quality between different edge devices and video sources, and select the edge device corresponding to the path with the least network hops or the most stable connection as the target edge device based on the network topology.

[0061] It is understandable that in addition to determining the target edge device in the above manner, the target edge device may also be determined in other manners, such as selecting an edge device with sufficient available bandwidth as the target edge device.

[0062] It can also be understood that the above-mentioned methods for determining the target edge device can be selected one by one, or multiple methods can be combined to determine the target edge device. When multiple methods are combined to determine the target edge device, different weights are assigned to the above-mentioned different factors according to the needs of the actual application scenario. A weighted algorithm or a multi-attribute decision method is used to determine the target edge device, which can ensure that the video stream data can be efficiently and reliably transmitted and processed.

[0063] S130: Preprocess the video stream data at the target edge device to obtain a number of image frames.

[0064] The video stream data is preprocessed to obtain a number of image frames, including:

[0065] Extracting consecutive frames from the video stream data;

[0066] adjusting the resolution of the frames to a uniform size;

[0067] Performing an enhancement operation on the frames after the uniform size to obtain enhanced frames;

[0068] The enhanced frames are converted into the format required by the algorithm to obtain the plurality of image frames.

[0069] Specifically, you can use tools such as FFmpeg to decode the video stream into a frame sequence, and then use fixed time intervals or key frame extraction to obtain continuous frames according to actual needs. This step reduces the computational burden of subsequent steps by extracting rather than processing each frame, and a reasonable selection of extraction strategies can retain key information in the video stream and avoid missing important events.

[0070] Scale all frames to a uniform size using an algorithm such as bilinear interpolation or Lanczos interpolation.

[0071] For example, to scale using bilinear interpolation:

[0072] f(x,y)=(1-u)(1-2)f(x0,y0)+u(lv)f(x1,y0)+(1-u)vf(x0,y1)+uvf(1,y1)

[0073] Where u and v are interpolation coefficients, (x0, y0), (x1, y1) are the pixel coordinates in the original image.

[0074] By scaling all frames to a uniform size, we can ensure that the image frames input to the algorithm have a consistent size, which is convenient for subsequent processing. By adjusting the resolution to the smaller size required by the algorithm, we can reduce computing resource consumption and speed up processing.

[0075] The frames after the uniform size are enhanced, which may include denoising, contrast enhancement, etc. Among them, denoising can apply denoising techniques such as Gaussian blur and median filtering to reduce noise interference in the image. The contrast of the image can be increased by methods such as histogram equalization to improve the visual effect.

[0076] Performing frame enhancement operations can make images clearer, improve the accuracy of subsequent algorithms, and strengthen important features in the image, helping the algorithm to better identify and analyze target objects.

[0077] When performing format conversion on the enhanced frame, the enhanced frame can be converted from one format (such as JPEG) to another format (such as NumPy array) according to the algorithm requirements to obtain several image frames.

[0078] Format conversion of enhanced frames can ensure that the image frames fully meet the requirements of subsequent algorithm processing, avoid the problem of data format mismatch, and some algorithms have better processing efficiency for specific data formats. Format conversion can speed up algorithm execution.

[0079] Preprocessing the video stream data in the above manner can not only prepare suitable input data for the real-time video stream analysis algorithm, but also improve the processing efficiency of the overall system while ensuring the analysis quality.

[0080] S140: Input the image frame into a trained analysis model to obtain an analysis result.

[0081] Specifically, the trained analysis model is pre-configured in the edge device.

[0082] In one embodiment, the trained analysis model may include a face recognition model and / or a target detection model.

[0083] Among them, the face recognition model can use a deep learning framework such as MTCNN for face detection. MTCNN gradually and accurately locates the face area through multi-level neural networks P-Net, R-Net, and O-Net.

[0084] Specifically, the image frame is input into the MTCNN framework, and the first-level network P-Net quickly screens out potential face areas to reduce the search space; the second-level network R-Net further screens the candidate face areas and rejects most non-face windows; the third-level network O-Net outputs the final face position and bounding box to achieve high-precision face detection. The detected face area is feature extracted, the face image is converted into a feature vector, the extracted face features are compared with the features stored in the database, and the individual identity of the face is determined based on the result of feature matching, and the recognized face and its corresponding identity information are used as the output result.

[0085] By comparing the extracted facial features with the features stored in the database, the similarity between the two can be calculated.

[0086] Among them, the target detection module can use YOLO, SSD or Faster R-CNN algorithm for real-time target detection.

[0087] Exemplarily, YOLO is used for real-time target detection, including:

[0088] The entire image frame is input into a single convolutional neural network (CNN), and then the network predicts the class probabilities of bounding boxes and objects within the bounding boxes at different scales. Finally, NMS is applied to process overlapping bounding boxes to obtain the final detection output.

[0089] Exemplarily, real-time target detection using SSD includes:

[0090] SSD generates predictions on different layers of CNN, using feature maps of different scales to detect objects of different sizes. Each position on the feature map of each layer is associated with a series of default boxes for prediction and classification of these boxes. Similar to YOLO, NMS is used to remove redundant bounding boxes.

[0091] For example, Faster R-CNN is used for real-time target detection, including:

[0092] First, use RPN to generate high-quality candidate regions. The candidate regions are pooled to a fixed size and input to the fully connected layer for classification and bounding box regression. The precisely adjusted bounding box and the corresponding category label are output.

[0093] The above algorithms all output the category label and bounding box coordinates of each detected target, and integrate this information to obtain a composite image frame containing multiple targets and their locations.

[0094] It is understandable that the above target detection model can be selected according to the actual application scenario.

[0095] It is also understandable that the analysis model is regularly optimized and updated based on feedback from actual application scenarios.

[0096] S150: Providing corresponding feedback according to the analysis result.

[0097] In one embodiment, providing corresponding feedback according to the analysis result includes:

[0098] The analysis results are fed back to the user and / or the automatic control system.

[0099] Specifically, the analysis results can be sent to users and / or automatic control systems, such as monitoring centers or applications, through secure network protocols such as HTTPS. Lightweight protocols such as MQTT and CoAP can also be used to send analysis results to a central server or directly trigger local actions, such as opening a door lock.

[0100] If the analysis result meets the preset conditions, a preset response mechanism is triggered.

[0101] The preset conditions can be set according to actual needs, and the preset response mechanism corresponds to the preset conditions. For example, the preset response mechanism can be an automatic triggering alarm or access control.

[0102] The real-time video stream analysis method based on edge computing provided in the embodiment of the present application also includes:

[0103] The analysis results and corresponding timestamps are sent to a database.

[0104] Specifically, the obtained analysis results and corresponding timestamps are recorded in a database for subsequent analysis.

[0105] It is understandable that the transmitted video stream data in the above-mentioned real-time video stream analysis method can be encrypted to ensure data security; it is also understandable that real-time privacy protection measures can also be taken, such as blurring faces to ensure compliance with regulatory requirements.

[0106] The real-time video stream analysis method based on edge computing provided in the embodiment of the present application refers to deploying the processing and analysis tasks of video stream data on edge computing devices for execution, rather than transmitting them to a remote cloud server for processing via the network. By optimizing the process of data processing and analysis, not only the stability and economy of the system are improved, but also the efficiency and response speed of business processing are significantly improved. The advantages of this method are reflected in many aspects:

[0107] First, this method effectively avoids business processing interruptions caused by network instability or failure. Since data processing and analysis are completed on edge devices, there is no need to rely on a stable network connection, which significantly enhances the robustness of the system. At the same time, this processing method also significantly reduces the network traffic costs generated by data transmission and reduces the burden on cloud service providers in video streaming computing.

[0108] Secondly, edge computing enables the system to analyze video streams faster. Since data processing and analysis tasks are performed near the data source, data transmission delays in the network are reduced, resulting in faster response speeds and higher business processing efficiency. This advantage is particularly important for application scenarios that require real-time responses, such as smart transportation and security monitoring.

[0109] Reference Figure 2 , which shows a structural diagram of a real-time video stream analysis device based on edge computing described according to an embodiment of the present application.

[0110] like Figure 2 As shown, the real-time video stream analysis device based on edge computing may include:

[0111] An acquisition module 210 is used to acquire real-time video stream data of a target area from a video source using a streaming media protocol;

[0112] A sending module 220, configured to send the video stream data to a target edge device;

[0113] A preprocessing module 230, configured to preprocess the video stream data at the target edge device to obtain a plurality of image frames;

[0114] A processing module 240 is used to input the image frame into a trained analysis model to obtain an analysis result;

[0115] The feedback module 250 is used to make corresponding feedback according to the analysis result.

[0116] Optionally, the real-time video stream analysis device based on edge computing further includes a device determination module, which is used to:

[0117] The target edge device is determined according to at least one of network delay time, real-time load, performance indicators of each edge device, and network topology.

[0118] Optionally, the preprocessing module 230 is further used for:

[0119] Extracting consecutive frames from the video stream data;

[0120] adjusting the resolution of the frames to a uniform size;

[0121] Performing an enhancement operation on the frames after the uniform size to obtain enhanced frames;

[0122] The enhanced frames are converted into the format required by the algorithm to obtain the plurality of image frames.

[0123] Optionally, the trained analysis model includes a face recognition model and / or a target detection model.

[0124] Optionally, the feedback module 250 is further configured to:

[0125] The analysis results are fed back to the user and / or the automatic control system.

[0126] Optionally, if the analysis result meets a preset condition, a preset response mechanism is triggered.

[0127] Optionally, the real-time video stream analysis device based on edge computing further includes a storage module, which is used to:

[0128] The analysis results and corresponding timestamps are sent to a database.

[0129] The node marginal electricity price solution device provided in this embodiment can execute the embodiment of the above method. Its implementation principle and technical effect are similar and will not be repeated here.

[0130] Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, a structural schematic diagram of an electronic device 300 suitable for implementing an embodiment of the present application is shown.

[0131] like Figure 3 As shown, the electronic device 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage part 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the device 300 are also stored. The CPU 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0132] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.

[0133] In particular, according to the embodiments of the present disclosure, the above reference Figure 1 The described process can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the above-mentioned edge computing-based real-time video stream analysis method. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 309, and / or installed from the removable medium 311.

[0134] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of the code, and the aforementioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0135] The units or modules involved in the embodiments described in the present application may be implemented by software or hardware. The units or modules described may also be arranged in a processor. The names of these units or modules do not constitute limitations on the units or modules themselves in certain circumstances.

[0136] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop, a mobile phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0137] As another aspect, the present application further provides a storage medium, which may be a storage medium included in the aforementioned device in the above embodiment; or a storage medium that exists independently and is not assembled into a device. The storage medium stores one or more programs, and the aforementioned programs are used by one or more processors to execute the real-time video stream analysis method based on edge computing described in the present application.

[0138] Storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0139] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0140] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A real-time video stream analysis method based on edge computing, characterized in that: include: Using streaming media protocol to obtain real-time video stream data of the target area from the video source; Sending the video stream data to a target edge device; Preprocessing the video stream data on the target edge device to obtain a number of image frames; Inputting the image frame into a trained analysis model to obtain an analysis result; Provide corresponding feedback based on the analysis results.

2. The real-time video stream analysis method based on edge computing according to claim 1 is characterized in that: The method for determining the target edge device includes: The target edge device is determined according to at least one of network delay time, real-time load, performance indicators of each edge device, and network topology.

3. The real-time video stream analysis method based on edge computing according to claim 1 is characterized in that: The preprocessing of the video stream data to obtain a plurality of image frames includes: Extracting consecutive frames from the video stream data; adjusting the resolution of the frames to a uniform size; Performing an enhancement operation on the frames after the uniform size to obtain enhanced frames; The enhanced frames are converted into the format required by the algorithm to obtain the plurality of image frames.

4. The real-time video stream analysis method based on edge computing according to claim 1 is characterized in that: The trained analysis model includes a face recognition model and / or a target detection model.

5. The real-time video stream analysis method based on edge computing according to claim 1 is characterized in that: Providing corresponding feedback based on the analysis results, including: The analysis results are fed back to the user and / or the automatic control system.

6. The real-time video stream analysis method based on edge computing according to claim 5 is characterized in that: If the analysis result meets the preset conditions, a preset response mechanism is triggered.

7. The real-time video stream analysis method based on edge computing according to claim 1 is characterized in that: The method also includes: The analysis results and corresponding timestamps are sent to a database.

8. A real-time video stream analysis device based on edge computing, characterized in that: The device comprises: An acquisition module is used to acquire real-time video stream data of a target area from a video source using a streaming media protocol; A sending module, used to send the video stream data to a target edge device; A preprocessing module, used for preprocessing the video stream data at the target edge device to obtain a plurality of image frames; A processing module, used for inputting the image frame into a trained analysis model to obtain an analysis result; The feedback module is used to make corresponding feedback according to the analysis result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the real-time video stream analysis method based on edge computing is implemented as described in any one of claims 1-7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a real-time video stream analysis method based on edge computing as described in any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Layered processing system for edge device video data

    CN119832466A