Intelligent edge video processing method based on set top box
By deploying the YOLOv8n model in the set-top box and controlling the transmission of target detection image frames using the image change ratio, the problem of high cost of local video stream processing is solved, achieving low-cost, fast video analysis and privacy protection.
Patent Information
- Application Number
- CN202511473115.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-15
- Publication Date
- 2026-01-02
AI Technical Summary
Existing video stream analysis and processing methods are costly to deploy locally and pose risks of privacy breaches and network bandwidth strain.
A YOLOv8n model is deployed in a set-top box, connected to a network camera via a local area network, to acquire and decode video streams. The image change ratio is calculated using the pixel difference between adjacent frames, and the target detection image frames are sent to the YOLOv8n model for detection. The detection results are then plotted on the video, achieving localized processing.
It reduces hardware costs, alleviates the burden on the cloud, improves response speed and privacy protection capabilities, saves computing resources, and enables localized intelligent processing of video streams.
Smart Images
Figure CN121262425A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing, in particular to an edge intelligent video processing method based on a set-top box. BACKGROUND
[0002] With the rapid development of smart home and video security industry, the dependence of home users on Internet Protocol Cameras (IPCs) is deepening. However, the current mainstream video processing system adopts a cloud architecture, which requires uploading video streams to a remote server for analysis and processing. This not only poses a risk of privacy leakage, but also brings high network bandwidth pressure and processing delay.
[0003] If AI image processing is deployed on the IPC side, it requires strong local computing resources, resulting in high hardware costs. However, for a home, a set-top box (STB) is commonly used, which has video decoding capability and integrates NPU / GPU computing units.
[0004] Therefore, if a lightweight AI model is deployed on the set-top box to realize local intelligent processing of IPC video streams, it not only reduces the burden on the cloud, improves system response speed and privacy protection capability, but also reduces the overall deployment cost, which has significant practical application value and commercial promotion potential. SUMMARY
[0005] The technical problem solved by the present application is to provide an edge intelligent video processing method based on a set-top box, which solves the problem of high cost of deploying video stream analysis and processing locally.
[0006] The technical solution adopted by the present application to solve the above technical problem is an edge intelligent video processing method based on a set-top box, wherein a YOLOv8n model is deployed in the set-top box. The method comprises the following steps: S1, the set-top box is connected to the Internet Protocol Camera (IPC) through a local area network, the set-top box interacts with the IPC through the ONVIF protocol, and the video stream is obtained from the IPC through the RTSP protocol; S2, decoding and processing the obtained video stream to extract the original image frame; S3, calculating the image change ratio based on the pixel difference between adjacent frames of the original image; S4, determining the target detection image frame sent to the YOLOv8n model based on the image change ratio; S5, the YOLOv8n model performs target detection on the received target detection image frame and outputs the target bounding box position, target type and confidence information; S6, using OpenGLES to draw the output of the YOLOv8n model as a graphics layer on the original video frame, and displaying it in real time on the TV screen through the video output interface.
[0007] Further, the interaction includes webcam discovery, webcam control, event subscription, and state maintenance mechanisms.
[0008] Further, in S3, the image change ratio is calculated based on the pixel difference between adjacent frames of the original image, including the following steps: S31, converting each frame of the original image into a grayscale image; S32, calculating the grayscale value difference of each pixel point between adjacent frames, the calculation formula is: wherein, represents the frame number, represents the grayscale value of the pixel point in the first frame, represents the grayscale value of the pixel point in the second frame, represents the grayscale value of the pixel point in the third frame, represents the grayscale value difference of the pixel point in the fourth frame, represents the grayscale value difference of the pixel point S33, calculating the ratio of the number of pixel points with a grayscale value difference greater than a first threshold value to the total number of pixel points as the image change ratio.
[0009] Further, the first threshold value is 28.
[0010] Further, in S4, the target detection image frame and frame rate sent to the YOLOv8n model are determined based on the image change ratio, including: if the image change ratio is greater than a second threshold value, then n frames are extracted at equal intervals from the video frames per second, and sent to the YOLOv8n model at a rate of n frames per second, and if the image change ratio is less than the second threshold value, then m frames are extracted at equal intervals from the video frames per second, and sent to the YOLOv8n model at a rate of m frames per second.
[0011] Further, the second threshold value is 7%, n is 20, and m is 5.
[0012] Further, the preset value is 0.4.
[0013] Further, the YOLOv8n model runs on the NPU of the set-top box.
[0014] Further, the method further comprises: if the target type belongs to a preset target and the confidence information exceeds a preset value, generating event JSON data, the event JSON data including a target bounding box position, a target type, a timestamp and a network camera number, and uploading the event JSON data to a cloud platform in real time through an MQTT protocol after the event JSON data is encrypted by AES-128 symmetric encryption.
[0015] The application provides an edge intelligent video processing method based on a set-top box, which deploys a YOLOv8n model in the set-top box, the set-top box acquires a video stream from a network camera, decodes the video stream, extracts original image frames, calculates an image change ratio based on pixel differences between adjacent frames of the original image, determines target detection image frames sent to the YOLOv8n model based on the image change ratio, the YOLOv8n model performs target detection on the received target detection image frames, uses OpenGL ES to draw the output of the YOLOv8n model as a graphics layer on the original video frames, and displays the original video frames on a television screen in real time through a video output interface, so that the deployed YOLOv8n model is used to process the video stream locally, costs are saved, the problem that the existing analysis and processing of the video stream are deployed locally at a high cost is solved, and the processing capability of the video stream is linked to the change of the video content by determining the target detection image frames sent to the YOLOv8n model based on the image change ratio, which is beneficial to saving the computing resources of the set-top box. BRIEF DESCRIPTION OF DRAWINGS
[0016] Figure 1 is a flowchart of an edge intelligent video processing method based on a set-top box provided by the application. DETAILED DESCRIPTION
[0017] The application provides an edge intelligent video processing method based on a set-top box, which deploys a YOLOv8n model in the set-top box, realizes local processing of a video stream, and determines target detection image frames sent to the YOLOv8n model based on an image change ratio of the video stream, so as to save the computing resources of the set-top box.
[0018] As shown in Figure 1 , the application provides an edge intelligent video processing method based on a set-top box, which comprises the following steps:
[0019] S1, the set-top box is connected with the network camera through a local area network, the set-top box interacts with the network camera through an ONVIF protocol, and acquires a video stream from the network camera through an RTSP protocol.
[0020] Specifically, the interaction includes webcam discovery, webcam control, event subscription, and state maintenance mechanisms.
[0021] S2. Decode the acquired video stream and extract the original image frames.
[0022] Specifically, FFmpeg is used to receive video stream data from IPC and decode it into image frame format.
[0023] S3. Calculate the image change rate based on the pixel difference between adjacent frames of the original image.
[0024] Specifically, the image change rate is calculated based on the pixel differences between adjacent frames of the original image, including the following steps:
[0025] S31. Convert each frame of the original image to a grayscale image.
[0026] S32. Calculate the difference in grayscale value of each pixel in adjacent frames. The calculation formula is as follows: ,in, Indicates the number of frames. Indicates the first Frame pixels grayscale value, Indicates the first Frame pixels grayscale value, Indicates the first Frame pixels The difference in grayscale values.
[0027] S33. The ratio of the number of pixels with grayscale differences greater than a first threshold to the total number of pixels is used as the image change ratio. The first threshold can be 28.
[0028] S4. Determine the target detection image frames to be sent to the YOLOv8n model based on the image change ratio.
[0029] Specifically, if the image change rate is greater than the second threshold (which can be 7%), then n frames are extracted at equal intervals from the video frames per second and sent to the YOLOv8n model at a rate of n frames per second. If the image change rate is less than the second threshold, then m frames are extracted at equal intervals from the video frames per second and sent to the YOLOv8n model at a rate of m frames per second. n is greater than m and both are positive integers. n can be 20 and m can be 5. Therefore, the set-top box can avoid excessive computational load for a long time, saving the computing power of running the YOLOv8n model.
[0030] The S5 and YOLOv8n models perform target detection on the received target detection image frames and output the target bounding box position, target type, and confidence information.
[0031] Specifically, the preset value can be 0.4. Based on the ONNX Runtime framework, the YOLOv8n model is run on the set-top box's NPU, the size of the target detection image frame is fixed at 640×480, and the YOLOv8n model is used for target detection, such as human bodies, flames, and pets.
[0032] In particular, for the YOLOv8n model, the set-top box can also obtain the corresponding weight file from the cloud through the TLS channel to update the local YOLOv8n model.
[0033] S6. Use OpenGLES to draw the output of the YOLOv8n model as a graphics layer on the original video frame, and display it on the TV screen in real time through the video output interface.
[0034] Specifically, the rendering is done on the image buffer of the original decoded frame, without changing the original image content, only generating a visualization effect frame. The synthesized image frame is then displayed in real time on the TV screen through a video output interface (such as HDMI), allowing users to view the video output of the overlaid YOLOv8n model. Users can switch between the video feeds of multiple IPCs using a remote control or a set-top box app, and can choose to turn the overlaid YOLOv8n model output function on or off, enabling comparison between the original image and the overlaid YOLOv8n model output. Additionally, users can adjust key parameters for each network camera, including resolution, frame rate, bit rate, exposure mode, infrared night vision, privacy occlusion area, and motion detection sensitivity.
[0035] This invention provides an edge intelligent video processing method based on a set-top box, which further includes: if the target type belongs to a preset target and the confidence information exceeds a preset value, then generating event JSON data. The event JSON data includes the target border position, target type, timestamp, and network camera number. The event JSON data is then encrypted using AES-128 symmetric encryption and uploaded to a cloud platform in real time via the MQTT protocol. This achieves the purpose of monitoring the preset target in the cloud. It eliminates the need to upload large amounts of raw video data; only the event JSON data and the corresponding video stream are uploaded.
Claims
1. An edge-based intelligent video processing method based on a set-top box, characterized in that, The method, which deploys a YOLOv8n model in a set-top box, includes the following steps: S1. The set-top box and the network camera are connected via a local area network. The set-top box interacts with the network camera via the ONVIF protocol and obtains video streams from the network camera via the RTSP protocol. S2. Decode the acquired video stream and extract the original image frames; S3. Calculate the image change rate based on the pixel difference between adjacent frames of the original image; S4. Determine the target detection image frames to be sent to the YOLOv8n model based on the image change ratio; The S5 and YOLOv8n models perform target detection on the received target detection image frames and output the target bounding box position, target type and confidence information. S6. Use OpenGLES to draw the output of the YOLOv8n model as a graphics layer on the original video frame, and display it on the TV screen in real time through the video output interface.
2. The edge intelligent video processing method based on a set-top box according to claim 1, characterized in that, The interaction includes webcam discovery, webcam control, event subscription, and state maintenance mechanisms.
3. The edge intelligent video processing method based on a set-top box according to claim 1, characterized in that, In S3, the image change rate is calculated based on the pixel differences between adjacent frames of the original image, including the following steps: S31. Convert each frame of the original image to a grayscale image; S32. Calculate the difference in grayscale value of each pixel in adjacent frames. The calculation formula is as follows: ,in, Indicates the number of frames. Indicates the first Frame pixels grayscale value, Indicates the first Frame pixels grayscale value, Indicates the first Frame pixels The difference in grayscale values; S33. The ratio of the number of pixels with grayscale differences greater than the first threshold to the total number of pixels is used as the image change ratio.
4. The edge intelligent video processing method based on a set-top box according to claim 3, characterized in that, The first threshold is 28.
5. The edge intelligent video processing method based on a set-top box according to claim 1, characterized in that, In S4, the target detection image frames and frame rate to be sent to the YOLOv8n model are determined based on the image change ratio, including: if the image change ratio is greater than the second threshold, n frames are extracted at equal intervals from the video frames per second and sent to the YOLOv8n model at a rate of n frames per second; if the image change ratio is less than the second threshold, m frames are extracted at equal intervals from the video frames per second and sent to the YOLOv8n model at a rate of m frames per second.
6. The edge intelligent video processing method based on a set-top box according to claim 5, characterized in that, The second threshold is 7%, n is 20, and m is 5.
7. The edge intelligent video processing method based on a set-top box according to claim 1, characterized in that, The preset value is 0.
4.
8. The edge intelligent video processing method based on a set-top box according to claim 1, characterized in that, The YOLOv8n model runs on the set-top box's NPU.
9. The edge intelligent video processing method based on a set-top box according to claim 1, characterized in that, The method further includes: if the target type belongs to a preset target and the confidence information exceeds a preset value, then generate event JSON data. The event JSON data includes the target border position, target type, timestamp and network camera number. After the event JSON data is symmetrically encrypted with AES-128, it is uploaded to the cloud platform in real time using the MQTT protocol.