Image processing and rendering system and method based on pod upper computer

Abstract

The embodiment of the invention discloses an image processing and rendering system and method based on a pod upper computer. According to the embodiment of the invention, compressed video frames can be read from a real-time video stream according to a frame sequence, and decoding and pixel format conversion processing are carried out on the compressed video frames to obtain a video image conforming to a preset image format; storing the video images into an image ready queue; sequentially taking out the to-be-processed images from the image ready queue, and obtaining target pod state data synchronized with the to-be-processed images in time; to-be-superimposed characters and character superimposition style parameters configured by a user for the to-be-processed image are obtained, and the to-be-superimposed characters are generated based on the target pod state data and / or user-defined content; drawing the to-be-superimposed characters on the to-be-processed image according to the character superimposition style parameters to generate a superimposed image; and rendering and displaying the superposed image. Therefore, according to the embodiment of the invention, real-time fusion of the user-defined content and the pod state data is realized, and highly flexible character superposition is supported.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optoelectronic pod video processing, in particular to an image processing and rendering system and method based on a pod host computer. BACKGROUND

[0002] In the prior art, optoelectronic pods are widely used in reconnaissance, inspection, surveying and mapping, etc. The pod usually integrates visible light / infrared cameras, inertial measurement units (IMU), GPS sensors, etc. It can transmit video stream and pod state data containing position, attitude, height, etc. in real time. After receiving these data, the pod host computer often needs to superimpose key state information (such as latitude, longitude, height, and heading angle) on the video screen to assist the operator in making situation judgments and task decisions.

[0003] Currently, the mainstream pod host computer system uses a static configuration method to realize character superimposition: the operator needs to set fixed text format and display parameters in the configuration interface before the task starts; the system automatically fills and renders the received pod state data into each frame of video image according to the preset rules when running. Once the video stream starts processing, the superimposed content and display form are locked and cannot be changed according to the task progress or operation intention.

[0004] Because the superimposition logic completely depends on the pre-written template, the position, font, color and combination method of all characters in the actual output video screen remain unchanged. In complex scenarios such as multi-target tracking, emergency response or cross-region operation, the operator often needs to temporarily highlight specific information (such as target number, threat level) or adjust the information layout to adapt to different observation priorities, but the existing technology cannot provide corresponding screen adaptation capability, resulting in rigid information presentation and limited human-machine collaboration efficiency. SUMMARY

[0005] The embodiments of the present application provide an image processing and rendering system and method based on a pod host computer, which can realize real-time fusion of user-defined content and pod state data without interrupting real-time video stream, support highly flexible character superimposition, and significantly improve the interactive flexibility, information synchronization accuracy and visual customization capability of the system.

[0006] The embodiments of the present application provide an image processing and rendering system based on a pod host computer, which is applied to a pod host computer in communication connection with a pod. The pod host computer sends control instructions to the pod, receives real-time video stream and pod state data returned by the pod, and the system comprises:

[0007] A video decoding unit is configured to read compressed video frames from the real-time video stream in frame sequence, and to decode and perform pixel format conversion processing on the compressed video frames to obtain video images conforming to a preset image format;

[0008] The image storage unit is used to store video images into the image ready queue;

[0009] The data alignment unit is used to sequentially retrieve images to be processed from the image ready queue and obtain target pod status data that is synchronized with the images to be processed in time.

[0010] The character acquisition unit is used to acquire the characters to be superimposed and the character superimposition style parameters configured by the user for the image to be processed. The characters to be superimposed are generated based on the target pod status data and / or user-defined content.

[0011] The image overlay unit is used to draw the characters to be overlaid onto the image to be processed according to the character overlay style parameters, thereby generating an overlay image;

[0012] The rendering unit is used to render and display overlay images.

[0013] This application also provides an image processing and rendering method based on a pod-based host computer, characterized in that it is applied to a host computer that is communicatively connected to the pod. The host computer sends control commands to the pod and receives real-time video streams and pod status data transmitted back from the pod. The method includes:

[0014] Compressed video frames are read sequentially from the real-time video stream, and the compressed video frames are decoded and pixel format converted to obtain a video image that conforms to the preset image format.

[0015] Store the video images in the image ready queue;

[0016] Images to be processed are retrieved sequentially from the image ready queue, and target pod status data synchronized with the images to be processed in time are obtained.

[0017] Obtain the characters to be superimposed and character superimposition style parameters configured by the user for the image to be processed, wherein the characters to be superimposed are generated based on the target pod status data and / or user-defined content;

[0018] Based on the character overlay style parameters, the characters to be overlaid are drawn on the image to be processed to generate an overlay image;

[0019] Render and display the overlay image.

[0020] This application also provides an electronic device, including a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to execute steps in any of the optical camera autofocusing methods provided in this application during target tracking.

[0021] This application also provides a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute steps in any of the optical camera autofocusing methods provided in the target tracking process according to this application.

[0022] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of any of the optical camera autofocusing methods provided in the target tracking process according to this application.

[0023] This application embodiment applies to a host computer connected to a pod in communication. The host computer sends control commands to the pod and receives real-time video streams and pod status data transmitted back by the pod. It can read compressed video frames from the real-time video stream in frame order, decode and convert the compressed video frames to obtain video images conforming to a preset image format; store the video images in an image ready queue; sequentially retrieve images to be processed from the image ready queue, and obtain target pod status data that is synchronized with the images to be processed in time; obtain the characters to be superimposed and character superimposition style parameters configured by the user for the images to be processed, wherein the characters to be superimposed are generated based on the target pod status data and / or user-defined content; draw the characters to be superimposed on the images to be processed according to the character superimposition style parameters to generate a superimposed image; render and display the superimposed image.

[0024] In this application, the system first reads compressed video frames sequentially from the real-time video stream transmitted from the pod, decodes and converts them to pixel format to obtain a standardized video image conforming to a preset image format. Then, the video image is stored in an image-ready queue, decoupling the image decoding thread from subsequent processing and ensuring smooth processing at high frame rates. Furthermore, when retrieving an image to be processed from the image-ready queue, the system obtains target pod status data that is time-synchronized with the image to be processed. Based on this target pod status data and / or user-defined content, the user can dynamically configure characters to be overlaid for the current image to be processed, thereby generating customizable characters that are strictly time-synchronized with the content of the image to be processed. Finally, the system draws the characters to be overlaid onto the image to be processed according to the user-configured character overlay style parameters, generating an overlaid image and rendering it in real time. Therefore, this application can achieve real-time fusion of user-defined content and pod status data without interrupting the real-time video stream, supports highly flexible character overlay, and significantly improves the system's interactive flexibility, information synchronization accuracy, and visualization customization capabilities. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1a This is a flowchart illustrating the image processing and rendering method based on a pod-based host computer provided in an embodiment of this application.

[0027] Figure 1b A detailed flowchart of the image processing and rendering method based on a pod-based host computer provided in this application embodiment;

[0028] Figure 2a This is a schematic diagram of the image processing and rendering system based on a pod-based host computer provided in an embodiment of this application;

[0029] Figure 2b This is a system architecture diagram of the image processing and rendering system based on a pod host computer provided in the embodiments of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] This application provides an image processing and rendering system and method based on a pod-based host computer.

[0032] Specifically, the image processing and rendering system based on the pod-based host computer can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer (PC); the server can be a single server or a server cluster consisting of multiple servers.

[0033] In some embodiments, the image processing and rendering system based on the pod host computer can also be integrated into multiple electronic devices. For example, the image processing and rendering system based on the pod host computer can be integrated into multiple servers, and the image processing and rendering method based on the pod host computer of this application can be implemented by multiple servers.

[0034] In some embodiments, the server may also be implemented as a terminal.

[0035] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0036] In this embodiment, an image processing and rendering method based on a pod-based host computer is provided. This method is applied to a host computer connected to the pod in communication. The host computer sends control commands to the pod and receives real-time video streams and pod status data transmitted back from the pod, such as... Figure 1a The specific process of this image processing and rendering method based on the pod-based host computer can be shown as follows:

[0037] This refers to a modular payload device that integrates multiple sensors (such as visible light cameras, infrared thermal imagers, laser rangefinders, inertial measurement units (IMUs), GPS, etc.). It is typically mounted on UAVs, unmanned vehicles, ships, fixed-wing aircraft, or other mobile or fixed observation platforms to perform tasks such as reconnaissance, surveillance, inspection, mapping, and target tracking. The pod possesses independent attitude control, data acquisition, and transmission capabilities, and can transmit real-time video streams and pod status data (such as position, altitude, heading, pitch angle, roll angle, etc.) to its host computer system via communication connections.

[0038] A communication connection refers to the logical or physical link established between the pod's host computer and the pod for bidirectional data transmission, used to issue control commands and transmit status information. This connection can be established via wired methods (such as Ethernet or USB) or wireless methods (such as Wi-Fi, 4G / 5G, microwave image transmission, or data radio), supporting reliable transmission of real-time video streams, pod status data (such as position, attitude, and timestamps), and control commands (such as gimbal rotation, zoom, and recording start / stop). In this embodiment, the communication connection is initialized and maintained based on preset network protocols (such as RTSP, RTP, and TCP / UDP) and pre-configured pod network addresses (such as IP addresses and port numbers).

[0039] The pod host computer refers to the ground-based or remote control computer system (including embedded industrial computers, laptops, servers, or mobile terminals, etc.) that establishes a communication connection with the optoelectronic pod. It is used for centralized monitoring of the pod, issuing commands, receiving data, and visual processing.

[0040] Control commands refer to command signals generated by the host computer of the pod and sent to the pod, used to remotely regulate the working status of the pod or the behavior of its sensors. These control commands include, but are not limited to: gimbal rotation commands (such as azimuth and pitch angle adjustments), lens control commands (such as optical / digital zoom and focus), video acquisition commands (such as start / stop recording and switching video channels), power management commands (such as standby and restart), and mission mode switching commands (such as reconnaissance mode and tracking mode). In this application, the control commands are transmitted to the pod through the communication connection (such as a network link based on the TCP / UDP protocol) and parsed and executed by the pod's control unit, thereby achieving real-time and precise operational intervention on the pod.

[0041] Real-time video streams refer to continuous video data sequences that are continuously acquired by imaging sensors (such as visible light cameras and infrared thermal imagers) mounted on a pod, encoded and compressed, and then transmitted in real-time to the pod's host computer via a communication connection. These video streams are characterized by low latency and high timeliness, typically outputting at a fixed frame rate (such as 15 fps, 30 fps, or 60 fps), encapsulated using standard video encoding formats (such as H.264 and H.265), and transmitted via streaming media protocols (such as RTSP and RTP / UDP).

[0042] Pod status data refers to structured information related to the pod's own operating status and environmental perception, which is collected in real time by the pod's built-in sensors during operation and transmitted back to the host computer. This information includes, but is not limited to, location information (such as longitude, latitude, and altitude), attitude information (such as heading angle, pitch angle, and roll angle), motion parameters (such as speed, angular velocity, and acceleration), equipment status (such as battery level, gimbal angle, lens focal length, operating mode, and temperature), and timestamps (used to identify the time when the status data was collected), etc.

[0043] 101. Read compressed video frames from the real-time video stream in frame order, and decode and convert the compressed video frames to obtain a video image that conforms to the preset image format.

[0044] Frame order refers to the sequential numbering of video frames in a real-time video stream according to their acquisition or transmission time. In this application, frame order is used to ensure that video frames are read and processed sequentially according to time sequence, so as to maintain the continuity and timing consistency of the video picture and avoid picture jumps or information misalignment caused by out-of-order processing.

[0045] A compressed video frame is a data unit generated by encoding and compressing the original image captured by the imaging sensor on the pod (such as standard video encoding formats like H.264 and H.265). It serves as the basic unit of the real-time video stream and is transmitted back to the host computer of the pod via a communication connection.

[0046] Decoding refers to the process of parsing and restoring compressed video frames read from a real-time video stream. Specifically, it includes calling the corresponding video decoder (such as the decoding module based on FFmpeg) according to the video encoding format (such as H.264) to convert the compressed data into uncompressed raw pixel data (such as YUV or RGB format) for subsequent image processing and rendering.

[0047] Pixel format conversion refers to the operation of converting the decoded raw pixel data from its default color space or storage format (such as YUV420P) to a unified pixel format (such as RGB24, BGR32) required for target processing. This conversion ensures that the image data is compatible with subsequent modules such as character drawing and display controls, improving processing efficiency and rendering consistency.

[0048] The preset image format is a pre-defined standard image data format used to unify the internal image processing workflow, including parameters such as pixel format (e.g., RGB24), bit depth, and channel order. In this application, all decoded video frames are converted to this format to ensure consistency between image overlay and the input of the rendering module.

[0049] A video image refers to uncompressed image data that conforms to a preset image format after decoding and pixel format conversion, representing the complete scene content at a certain moment in a real-time video stream. This video image serves as the basic input for subsequent character overlay processing and is stored in the image ready queue to await further operations.

[0050] In some embodiments, in order to avoid interface lag or processing delay caused by video decoding blocking the main thread, the real-time performance, stability and smoothness of the system under high frame rate video streams are significantly improved, while ensuring the orderly execution of subsequent character overlay and rendering operations. The pod host computer runs an image processing sub-thread and an image processing main thread.

[0051] Compressed video frames are read sequentially from the real-time video stream, and then decoded and pixel-formatted to obtain a video image conforming to a preset image format, including:

[0052] The image processing sub-thread reads compressed video frames from the real-time video stream in frame order, decodes and converts the compressed video frames to pixel format, and obtains a video image that conforms to the preset image format. The image processing sub-thread then sends an image processing signal carrying the video image to the image processing main thread.

[0053] Store video images into the image-ready queue, including:

[0054] The main image processing thread stores video images into the image ready queue based on image processing signals.

[0055] The image processing sub-thread refers to a background worker thread created in the pod's host computer, specifically designed to perform low-level video stream processing tasks. Its main responsibilities include: continuously reading compressed video frames from the communication connection, calling the video decoder for decoding, performing pixel format conversion, and transmitting the processed video image to the main thread via inter-thread communication. This sub-thread runs independently of the user interface, ensuring that the decoding process does not block the system's main logic.

[0056] The main image processing thread refers to the core thread in the pod's host computer responsible for coordinating the image processing flow, managing data queues, executing character overlay logic, and driving graphics rendering. It typically runs in the same context as the user interface. The main thread receives processing signals from child threads, stores video images in the image ready queue, and sequentially executes operations such as state synchronization, user interaction response, character drawing, and screen display. It is the scheduling center of the entire image processing pipeline.

[0057] Image processing signals are inter-thread notification messages sent by the image processing sub-thread to the image processing main thread to inform the main thread that "a new video frame is ready." These signals can be implemented using operating system-provided event mechanisms (such as Qt's signal slots, Windows' PostMessage, and Linux's pipe / eventfd) or custom message queues. They typically carry a memory pointer or reference to the processed video frame to achieve efficient, low-overhead data transfer.

[0058] In some embodiments, to enable the system to immediately begin receiving and decoding real-time video streams without performing time-consuming protocol negotiation, format detection, or decoder creation operations when video reception is triggered by an image processing instruction, significantly shortening the latency from task initiation to first frame display and improving system response speed and task deployment efficiency, the system further includes the following steps before receiving the real-time video stream and pod status data transmitted back from the pod:

[0059] Establish a communication connection with the preset pod network address;

[0060] Based on the communication connection, the media description information of the video stream is obtained from the pod indicated by the preset pod network address;

[0061] The media description information is parsed to obtain the video encoding format;

[0062] Create a corresponding video decoder context based on the video encoding format;

[0063] Configure the video decoding initialization parameters to obtain the video decoding configuration parameters, and apply the video decoding configuration parameters to the video decoder context;

[0064] Decoder initialization is completed based on the configured video decoder context, putting the video decoder into a ready state. When an image processing instruction is received, the video decoder receives the real-time video stream transmitted back from the pod via the communication connection and decodes the real-time video stream by the video decoder in the ready state.

[0065] The preset pod network address refers to the communication endpoint information pre-configured in the pod's host computer, used to uniquely identify a specific pod in the network. It typically includes an IP address and port number (such as 192.168.1.100:554). This address is used to establish a communication connection with the specific pod, ensuring the accuracy and security of data transmission.

[0066] The preset pod network address indicates a physical pod device that uniquely corresponds to that address. Through this address, the host computer can connect to a specific pod, avoiding confusion with other devices and enabling precise one-to-one control and data interaction.

[0067] The media description information of a video stream refers to the metadata returned by the corresponding streaming media service of the pod, which describes the basic attributes of the video stream. It is usually in the SDP (Session Description Protocol) format and includes key parameters such as video encoding format, resolution, frame rate, transmission protocol (such as RTP / RTSP), and payload type. It is a prerequisite for correctly initializing the decoder.

[0068] Video encoding format refers to the compression standard used by a video stream, which characterizes the organization and encoding method of video data. Common video encoding formats include H.264 (AVC), H.265 (HEVC), and MJPEG. In this application, this format is obtained by parsing media description information and used to select a matching decoder.

[0069] The video decoder context refers to the core data structure created by a video decoding library (such as FFmpeg's AVCodecContext) to manage the decoding process. It stores all the configuration information required for decoding (such as encoding format, width and height, color space) and internal state, and is an essential carrier for performing the actual decoding operation.

[0070] Video decoding initialization parameters refer to a set of basic configuration items that need to be set before the decoder is created, including but not limited to: maximum frame width / height, number of threads, low-latency mode on / off, error tolerance policy, etc. These parameters affect decoding performance and compatibility, and are usually preset according to the application scenario.

[0071] Video decoding configuration parameters refer to the specific set of configurations generated based on the video encoding format and initialization parameters, which are ultimately applied to the video decoder context. After adaptation and verification, this parameter set is written into the decoder context to ensure that it can correctly parse the video stream from the pod.

[0072] The ready state indicates that the video decoder has completed all initialization steps (including context creation, parameter configuration, and internal resource allocation) and is in a ready-to-receive state, waiting to receive video data and immediately begin decoding. In this state, the decoder can respond to video stream input with zero latency and without additional startup overhead.

[0073] Image processing commands refer to control signals that are triggered by user operation or automatic system activation to initiate the video reception and processing flow. For example, after clicking the "Start Preview" button, the host computer generates image processing commands, instructing the system to start retrieving and processing the real-time video stream from the ready decoder.

[0074] Understandably, completing decoder initialization based on the configured video decoder context and putting the video decoder into a ready state means that after obtaining the media description information of the pod video stream and parsing the video encoding format, the system first creates a video decoder context (such as the AVCodecContext object in FFmpeg) corresponding to the encoding format, and then writes the pre-set or dynamically generated video decoding configuration parameters (such as resolution, color format, number of threads, low latency mode, etc.) into the context; on this basis, the initialization function provided by the decoding library (such as avcodec_open2()) is called to formally initialize the decoder.

[0075] Once initialization is successful, the decoder completes internal resource allocation (such as buffers, hardware acceleration interfaces, state machines, etc.) and enters a ready state—at this point, although the decoder has not yet received any video data, it already has the ability to decode immediately. When a real-time video stream is subsequently received via the communication connection, the system can directly send the compressed video frames to the ready decoder for processing, without having to perform time-consuming startup steps such as context creation, parameter probing, or protocol negotiation. This significantly reduces the latency from task triggering to the display of the first frame and improves system response speed.

[0076] 102. Store the video image in the image ready queue.

[0077] The image-ready queue refers to a first-in, first-out (FIFO) data buffer structure maintained in the memory of the pod's host computer. It is used to temporarily store video images that have completed decoding and pixel format conversion but have not yet undergone character overlay and rendering. Each video image can be accompanied by its corresponding timestamp or other metadata when enqueued to support precise time alignment with subsequent pod status data.

[0078] 103. Take out the images to be processed sequentially from the image ready queue, and obtain the target pod status data that is synchronized with the images to be processed in time.

[0079] The image to be processed refers to a video image retrieved from the image ready queue in a first-in-first-out order, which has completed decoding and pixel format conversion but has not yet undergone character overlay operation. This image represents the content of the scene at a specific moment in the real-time video stream, and usually comes with a capture timestamp corresponding to its original compressed video frame, which is used for subsequent time alignment with the pod status data to ensure that the overlay information is strictly consistent with the scene in time.

[0080] Target pod status data refers to the set of status information among all received pod status data that is closest to the image to be processed in the time dimension and meets the preset synchronization accuracy requirements. Specifically, the system selects the status data with the smallest time difference as "target pod status data" by comparing the timestamp of the image to be processed with the acquisition time of each pod status data. This target pod status data includes key parameters such as position, attitude, altitude, and heading angle, and is used to generate superimposed characters that are semantically consistent with the current image content, avoiding information mismatch caused by state lag or advancement.

[0081] In some embodiments, high-precision alignment of video images and pod status data in the time dimension is achieved, effectively avoiding distortion of overlay content caused by lag or advance of pod status data (such as displaying the height of the previous second on the current video image), significantly improving the accuracy, reliability and task decision support capabilities of character overlay, and there is a one-to-one correspondence between video images and compressed video frames.

[0082] Store video images into the image-ready queue, including:

[0083] Obtain the frame order of the compressed video frame in the real-time video stream, and determine the acquisition timestamp corresponding to the compressed video frame based on the frame rate and start acquisition time of the real-time video stream.

[0084] Based on the one-to-one correspondence between video images and compressed video frames, the acquisition timestamp is used as the timestamp of the video image;

[0085] Store the video image and its timestamp into the image ready queue;

[0086] Acquire target pod status data that is time-synchronized with the image to be processed, including:

[0087] The target pod status data is determined from the pod status data based on the time difference between the timestamp of the image to be processed and the acquisition time corresponding to the pod status data.

[0088] The frame rate of the real-time video stream refers to the number of image frames per second (fps) in the video stream output by the pod. Common values ​​include 15 fps, 25 fps, 30 fps, or 60 fps. This parameter is determined by the pod hardware or encoding configuration and is used to characterize the temporal density of the video. It is a key basis for calculating the acquisition time of any frame.

[0089] The start acquisition time of a real-time video stream refers to the absolute point in time (usually expressed as UTC time or system startup time) when the pod begins acquiring and outputting the real-time video stream. This time is recorded by the pod during the video stream initialization phase and can be synchronized to the pod's host computer via media description information or control protocols, serving as the reference starting point for calculating the acquisition timestamps of each frame.

[0090] The acquisition timestamp corresponding to the compressed video frame refers to the precise time stamp of the compressed video frame when it is captured by the sensor at the pod end. In this application, the timestamp is not directly extracted from the video frame (because some encoding formats do not embed time information), but is calculated using the formula: Acquisition Timestamp = Start Acquisition Time + Frame Order / Frame Rate, thereby establishing a reliable time coordinate for each frame.

[0091] The time difference refers to the absolute difference (usually in milliseconds or microseconds) between the timestamp of the image to be processed and the acquisition time of a certain pod status data. The system iterates through the cached pod status data, calculates the time difference between each data point and the image timestamp, and selects the one with the smallest time difference as the synchronization matching result, ensuring that the status information and the screen content are as consistent as possible in physical time.

[0092] In some embodiments, without relying on a strict hardware synchronization mechanism for pod alignment, high-precision time alignment between video images and pod status data can be achieved solely through software-level timestamp matching. This effectively avoids information mismatches caused by lag or advancement in pod status data (such as overlaying an altitude value from one second ago onto the currently processed video image), significantly improving the accuracy of the overlaid content and the reliability of the task. Based on the time difference between the timestamp of the image to be processed and the acquisition time corresponding to the pod status data, the target pod status data is determined from the pod status data, including:

[0093] Calculate the absolute time difference between the timestamp of the image to be processed and the acquisition time corresponding to each pod status data;

[0094] The pod status data with the smallest absolute time difference is used as the target pod status data.

[0095] The absolute time difference refers to the non-negative time interval between the timestamp of the image to be processed and the acquisition time of a certain pod status data, that is, the absolute value of the difference between the two time values.

[0096] Absolute time difference = |T 图像 -T 状态 |, where T 图像 T is the timestamp of the image to be processed. 状态 The acquisition time corresponds to the pod status data. This metric is used to quantify the similarity between the image to be processed and the pod status data on the time axis.

[0097] The pod status data with the smallest absolute time difference refers to the pod status data whose acquisition time and the timestamp of the image to be processed have the smallest absolute time difference among all available pod status data. Since pod status data is usually reported continuously at a high frequency (e.g., 10 Hz, 20 Hz), while the video frame rate is relatively low (e.g., 30 fps), there will always be a status data that is closest to the current image frame in time. Selecting this data as the "target" can maximize the physical consistency between the target pod status data as the character to be overlaid and the scene of the image to be processed.

[0098] 104. Obtain the characters to be superimposed and character superimposition style parameters configured by the user for the image to be processed, wherein the characters to be superimposed are generated based on the target pod status data and / or user-defined content.

[0099] Among them, "user" refers to the personnel who operate the pod's host computer system, including but not limited to task operators, monitoring personnel, system maintenance engineers, or command and dispatch personnel.

[0100] The text information to be overlaid refers to the text information that is planned to be drawn onto the image to be processed. It is used to dynamically display the status or instruction content related to the current image to be processed on the image screen. The text information to be overlaid is generated from at least one of the target pod status data (such as "ALT:150m", "HDG:210°") and user-defined content (such as "Target A", "Inspection Point #3").

[0101] Character overlay style parameters refer to a set of configuration parameters used to control the visual presentation effect of the characters to be overlaid, including but not limited to optional attributes such as font type, font size, color, overlay position, transparency, stroke, and alignment.

[0102] User-defined content refers to content that is actively entered or selected by the user through the human-machine interface and is not derived from the pod status data. This content can be any string such as task identifier, target name, remarks, or operation instructions, used to enhance the information expression capabilities of the video footage. For example, in a power line inspection scenario, the user can enter "#Pole07 Anomaly", which will be displayed in real time on the image to be processed as part of the characters to be overlaid.

[0103] In some embodiments, users can construct personalized information overlay schemes in real time and intuitively during video playback without pre-setting templates, significantly improving human-computer collaboration efficiency, task adaptability, and the accuracy of information expression. This solves the technical bottleneck of "content locking and style fixing" in traditional systems, and obtains the characters to be overlaid and character overlay style parameters configured by the user for the image to be processed, including:

[0104] In response to editing operations on the image to be processed, the image editing page is displayed, which includes a custom area, a status configuration area, and a style configuration area;

[0105] Responding to content input operations in a custom area, retrieve user-defined content;

[0106] In response to the selection operation of target pod status data in the status configuration area, determine the configuration status corresponding to the target pod status data;

[0107] Based on the configuration status, user-defined content, and target pod status data, generate the characters to be overlaid;

[0108] In response to style setting operations for characters to be overlaid in the style configuration area, retrieve the corresponding character overlay style parameters.

[0109] Editing operations refer to interactive behaviors initiated by users on the currently displayed image to be processed through mouse clicks, touchscreen controls, or keyboard shortcuts, which trigger image editing functions. Typical editing operations include: selecting "Edit Characters to be Overlaid" from the right-click menu, clicking the "Annotation" button in the toolbar, or double-clicking on the image.

[0110] The image editing page refers to the graphical user interface (GUI) that pops up or is embedded in the host computer of the pod. It is dedicated to configuring the content and style of the characters to be overlaid. This page contains a structured configuration area, which allows users to customize the characters to be overlaid without interrupting the real-time video stream.

[0111] A custom area refers to an interactive control in an image editing page used to receive free text input from the user, typically appearing as a text box or rich text editor. Users can enter unstructured information such as task identifiers, target names, and notes in this area as components of the characters to be overlaid.

[0112] The status configuration area refers to the configuration area in the image editing page used to select pod status data fields. It is usually presented in the form of checkboxes, drop-down lists, or multi-select labels. Users can use this area to select the status items to be overlaid (such as "latitude and longitude", "altitude", "battery level"), and the system will determine which target pod status data should be included in the characters to be overlaid.

[0113] The style configuration area refers to the control panel in the image editing page used to set the visual attributes of characters, providing options for adjusting parameters such as font, size, color, position, and transparency. In some embodiments, this area supports setting styles for multiple character fields separately to achieve differentiated rendering.

[0114] Content input operations refer to text entry behaviors performed by users in a custom area, including keyboard input, pasting, speech-to-text, etc., which are used to generate user-defined content.

[0115] The selection operation refers to the interactive behavior of the user selecting the fields contained in the target pod status data in the status configuration area, such as checking the "altitude" checkbox or selecting "heading angle" from the drop-down menu, which is used to specify the status data fields that need to participate in the generation of the characters to be superimposed.

[0116] Configuration status refers to the logical identifier generated based on the user's selection operation in the status configuration area. It is used to indicate whether a certain field in the target pod status data is selected and included in the generation logic of the characters to be overlaid. For example, if the user selects "height", the configuration status corresponding to the "height" field is "enabled"; otherwise, it is "disabled". This configuration status directly determines whether the status value corresponding to the field in the target pod status data is included in the characters to be overlaid when generating the characters to be overlaid.

[0117] Style setting operations refer to user interactions in the style configuration area to adjust character display attributes, including dragging the slider to set the font size, clicking the color swatch to select a color, and dragging on the preview image to position the characters. The system responds to these operations by updating and saving the corresponding character overlay style parameters in real time.

[0118] In some embodiments, while retaining full customization capabilities, it supports on-demand fusion of real-time pod status information, avoiding redundant display of irrelevant data while ensuring accurate presentation of pod status information. This significantly improves the information density, readability, and task adaptability of video images. Based on configuration status, user-defined content, and target pod status data, it generates characters to be overlaid, including:

[0119] If the configuration status indicator needs to be superimposed with the target pod status data, the target pod status data will be combined with user-defined content to generate the characters to be superimposed;

[0120] If the configuration status indicator does not require the overlay of target pod status data, the user-defined content will be used as the character to be overlaid.

[0121] Understandably, when a user selects "Altitude" and "Heading" in the status configuration area and enters "Target A" in the custom area, both the "Altitude" and "Heading" fields are configured as "Enabled." The actual values ​​in the target pod status data (e.g., altitude "150m" and heading "210°") are combined with the user's input to generate the character to be overlaid: "Target A| ALT:150m | HDG:210°". However, if the user does not select any status fields (i.e., all configurations are "Disabled") and only enters "Emergency Marker," then "Emergency Marker" is directly used as the character to be overlaid, without being appended to any field in the target pod status data.

[0122] 105. Based on the character overlay style parameters, draw the characters to be overlaid onto the image to be processed to generate an overlay image.

[0123] In this context, the overlay image refers to a composite image generated by drawing the characters to be overlaid onto a specified position based on the image to be processed, according to the user-configured character overlay style parameters. This image retains all the visual content of the original video frame while incorporating textual information relevant to the current scene (such as target pod status data, task identifiers, etc.), and can be directly used for subsequent display, recording, or forwarding. In this application, the overlay image is the final output of the character overlay processing flow and also the input source for the video rendering module.

[0124] In some embodiments, differentiated and precise layout and visual presentation of multiple information fields are realized, avoiding the information stacking or obstruction problems caused by the uniform format and fixed position of all text in traditional solutions, significantly improving the information readability, space utilization and user customization freedom of video screens. The characters to be superimposed include multiple characters, and the character superimposition style parameters include the superimposition position and character style corresponding to each character.

[0125] Based on the character overlay style parameters, the characters to be overlaid are drawn onto the image to be processed, generating an overlay image, including:

[0126] For each character, determine the pixel coordinates of the character in the image to be processed based on the character's corresponding overlay position;

[0127] Based on the character style corresponding to the character, the character is drawn at pixel coordinates to obtain an overlay image.

[0128] In this context, "multiple characters" refers to characters to be superimposed consisting of two or more logically independent text units. Each unit can originate from a different data source (such as one from pod status data and another from user-defined content) and can be independently configured in terms of display position and style. For example, "ALT:150m", "HDG:210°", and "#inspection point 3" are three independent characters.

[0129] The overlay position of a character refers to the layout reference point on the image to be processed, which is individually specified for each character. It is usually expressed as an offset relative to the top left corner of the image (e.g., "100 pixels from the left, 50 pixels from the top"). This position can be set by the user through dragging, coordinate input, or preset templates, and is converted into specific pixel coordinates during drawing.

[0130] The character style refers to the set of visual attributes bound to a single character, including but not limited to: font type (e.g., Microsoft YaHei), font size (e.g., 18pt), color (e.g., red #FF0000), bolding, transparency, and stroke. Different characters can have completely different styles, achieving information layering and highlighting of key points.

[0131] Pixel coordinates refer to the precise coordinates of the actual drawing position of the character in the image to be processed. The origin (0, 0) is the top-left corner of the image to be processed, the horizontal axis is the X-axis, and the vertical axis is the Y-axis. The unit is pixels. The pixel coordinates are calculated based on the corresponding superposition position of the character (such as "top-left offset (100, 50)"), and the graphics drawing interface (such as OpenCV's putText or Qt's drawText) is called to render the character at that position.

[0132] 106. Render and display the overlay image.

[0133] In some embodiments, not only is visual consistency between the overlaid character content and the original video image ensured, but smooth playback and accurate timing restoration under cross-platform and multi-resolution conditions are also achieved, significantly improving the system's display compatibility, image stability, and user experience. Rendering and displaying the overlaid image includes:

[0134] Each superimposed image is converted to the target bitmap format to obtain a superimposed image in bitmap format;

[0135] Based on a preset resolution, the overlay image in bitmap format is scaled to obtain a processed image adapted to the display size.

[0136] According to the generation sequence of each superimposed image, the processed images are sequentially output to the video display control to form a real-time video stream;

[0137] Render and display a live video stream using video display controls.

[0138] The target bitmap format refers to a pre-defined standard pixel data format for unifying image processing and display workflows. It is typically an uncompressed bitmap structure arranged in rows in memory (such as RGB24, BGR32, ARGB, etc.). This format is compatible with underlying graphics libraries (such as OpenCV, Qt, DirectX) or operating system display interfaces, facilitating efficient drawing and transmission.

[0139] Bitmap overlay images refer to image data that has been overlaid with characters and converted to the target bitmap format. Essentially, it is a two-dimensional array of pixels, with each pixel containing color information, which can be directly read by the graphics rendering engine as an input source for subsequent scaling, display, or encoding.

[0140] An image adapted to the display size refers to an image obtained by scaling a bitmap overlay image to maintain its proportions or fill the gaps according to the preset resolution (such as 1920×1080, 1280×720) of the target display window or screen. This processing ensures that the image content fully fits the display area, avoiding stretching distortion, black borders, or cropping, and improving the visual experience.

[0141] The generation sequence of overlay images refers to the temporal order in which each overlay image is generated in the image processing pipeline. Because video streams have inherent temporal continuity, the system maintains this sequence (e.g., through timestamps or queue order) to ensure that images are output in the original logical order of acquisition → processing → display, preventing screen jumps or information misordering.

[0142] The video display control refers to the graphical component in the pod's host computer interface used to receive and render video frames, such as QVideoWidget in Qt, DirectShow Video Renderer in Windows, or a custom OpenGL rendering window. This control is responsible for frame rate control, double buffering management, vertical synchronization (VSync), etc., and is a key interface connecting the image processing module and the user's visual perception.

[0143] Real-time video streams refer to a continuous video sequence composed of processed images arranged in the order of their generation. Their frame rate is basically the same as that of the original real-time video stream (e.g., 30 fps), with low end-to-end latency (usually <200ms). They can realistically and instantly reflect the scene and overlaid content currently seen by the pod, meeting the timeliness requirements of applications such as task monitoring and command and dispatch.

[0144] Figure 1bThis is a flowchart of the image processing and rendering method based on the pod host computer provided in this application embodiment. It fully demonstrates the end-to-end processing logic from system initialization to real-time display of video images. It intuitively reveals how the method of this application achieves real-time fusion and display of user-defined content and pod status data without interrupting the real-time video stream through the core link of "video initialization → decoding → overlay (including data alignment and user configuration) → rendering" and combined with human-computer interaction and communication mechanisms.

[0145] In summary, this application enables real-time fusion of user-defined content and pod status data without interrupting the real-time video stream, supports highly flexible character overlay, and significantly improves the system's interactive flexibility, information synchronization accuracy, and visualization customization capabilities.

[0146] To better implement the above methods, this application also provides an image processing and rendering system based on a pod-based host computer. This system can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster consisting of multiple servers.

[0147] For example, in this embodiment, the method of this application embodiment will be described in detail by taking an image processing and rendering system based on a pod host computer specifically integrated into an electronic device.

[0148] For example, such as Figure 2a As shown, this image processing and rendering system based on a pod-based host computer is applied to a host computer connected to the pod. The host computer sends control commands to the pod and receives real-time video streams and pod status data transmitted back from the pod. The system may include a video decoding unit 201, an image storage unit 202, a data alignment unit 203, a character acquisition unit 204, an image overlay unit 205, and a rendering unit 206, as follows:

[0149] (a) Video decoding unit 201.

[0150] The video decoding unit 201 is used to read compressed video frames from the real-time video stream in frame order, and to decode and convert the compressed video frames to obtain a video image that conforms to a preset image format.

[0151] In some embodiments, before receiving the real-time video stream and pod status data transmitted back by the pod, the method further includes:

[0152] Establish a communication connection with the preset pod network address;

[0153] Based on the communication connection, the media description information of the video stream is obtained from the pod indicated by the preset pod network address;

[0154] The media description information is parsed to obtain the video encoding format;

[0155] Create a corresponding video decoder context based on the video encoding format;

[0156] Configure the video decoding initialization parameters to obtain the video decoding configuration parameters, and apply the video decoding configuration parameters to the video decoder context;

[0157] Decoder initialization is completed based on the configured video decoder context, putting the video decoder into a ready state. When an image processing instruction is received, the video decoder receives the real-time video stream transmitted back from the pod via the communication connection and decodes the real-time video stream by the video decoder in the ready state.

[0158] (ii) Image storage unit 202.

[0159] Image storage unit 202 is used to store video images into the image ready queue.

[0160] In some embodiments, the pod host computer runs an image processing sub-thread and an image processing main thread;

[0161] Compressed video frames are read sequentially from the real-time video stream, and then decoded and pixel-formatted to obtain a video image conforming to a preset image format, including:

[0162] The image processing sub-thread reads compressed video frames from the real-time video stream in frame order, decodes and converts the compressed video frames to pixel format, and obtains a video image that conforms to the preset image format. The image processing sub-thread then sends an image processing signal carrying the video image to the image processing main thread.

[0163] Store video images into the image-ready queue, including:

[0164] The main image processing thread stores video images into the image ready queue based on image processing signals.

[0165] (iii) Data alignment unit 203.

[0166] The data alignment unit 203 is used to sequentially retrieve images to be processed from the image ready queue and obtain target pod status data that is synchronized with the images to be processed in time.

[0167] In some embodiments, there is a one-to-one correspondence between video images and compressed video frames;

[0168] Store video images into the image-ready queue, including:

[0169] Obtain the frame order of the compressed video frame in the real-time video stream, and determine the acquisition timestamp corresponding to the compressed video frame based on the frame rate and start acquisition time of the real-time video stream.

[0170] Based on the one-to-one correspondence between video images and compressed video frames, the acquisition timestamp is used as the timestamp of the video image;

[0171] Store the video image and its timestamp into the image ready queue;

[0172] Acquire target pod status data that is time-synchronized with the image to be processed, including:

[0173] The target pod status data is determined from the pod status data based on the time difference between the timestamp of the image to be processed and the acquisition time corresponding to the pod status data.

[0174] In some embodiments, determining the target pod status data from the pod status data based on the time difference between the timestamp of the image to be processed and the acquisition time corresponding to the pod status data includes:

[0175] Calculate the absolute time difference between the timestamp of the image to be processed and the acquisition time corresponding to each pod status data;

[0176] The pod status data with the smallest absolute time difference is used as the target pod status data.

[0177] (iv) Character acquisition unit 204.

[0178] The character acquisition unit 204 is used to acquire the characters to be superimposed and the character superimposition style parameters configured by the user for the image to be processed, wherein the characters to be superimposed are generated based on the target pod status data and / or user-defined content.

[0179] In some embodiments, obtaining the characters to be overlaid and the character overlay style parameters configured by the user for the image to be processed includes:

[0180] In response to editing operations on the image to be processed, the image editing page is displayed, which includes a custom area, a status configuration area, and a style configuration area;

[0181] Responding to content input operations in a custom area, retrieve user-defined content;

[0182] In response to the selection operation of target pod status data in the status configuration area, determine the configuration status corresponding to the target pod status data;

[0183] Based on the configuration status, user-defined content, and target pod status data, generate the characters to be overlaid;

[0184] In response to style setting operations for characters to be overlaid in the style configuration area, retrieve the corresponding character overlay style parameters.

[0185] In some embodiments, based on configuration status, user-defined content, and target pod status data, characters to be overlaid are generated, including:

[0186] If the configuration status indicator needs to be superimposed with the target pod status data, the target pod status data will be combined with user-defined content to generate the characters to be superimposed;

[0187] If the configuration status indicator does not require the overlay of target pod status data, the user-defined content will be used as the character to be overlaid.

[0188] (v) Image overlay unit 205.

[0189] The image overlay unit 205 is used to draw the characters to be overlaid onto the image to be processed according to the character overlay style parameters, thereby generating an overlay image.

[0190] In some embodiments, the characters to be superimposed include multiple characters, and the character superimposition style parameters include the superimposition position and character style corresponding to each character;

[0191] Based on the character overlay style parameters, the characters to be overlaid are drawn onto the image to be processed, generating an overlay image, including:

[0192] For each character, determine the pixel coordinates of the character in the image to be processed based on the character's corresponding overlay position;

[0193] Based on the character style corresponding to the character, the character is drawn at pixel coordinates to obtain an overlay image.

[0194] (vi) Rendering Unit 205.

[0195] Rendering unit 205 is used to render and display overlay images.

[0196] In some embodiments, rendering and displaying the overlay image includes:

[0197] Each superimposed image is converted to the target bitmap format to obtain a superimposed image in bitmap format;

[0198] Based on a preset resolution, the overlay image in bitmap format is scaled to obtain a processed image adapted to the display size.

[0199] According to the generation sequence of each superimposed image, the processed images are sequentially output to the video display control to form a real-time video stream;

[0200] Render and display a live video stream using video display controls.

[0201] like Figure 2b As shown, this application clearly illustrates how the system constructs a complete solution integrating "communication, decoding, intelligent overlay, and rendering" through the collaborative work of six functional units. Among them, the image overlay unit acts as the hub, cleverly integrating real-time status data from the communication unit and user-defined commands from the human-machine interface unit, thereby achieving the "highly flexible character overlay" effect pursued by the technical solution.

[0202] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0203] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0204] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the image processing and rendering methods based on a pod-based host computer provided in embodiments of this application.

[0205] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0206] Since the instructions stored in the storage medium can execute the steps in any of the image processing and rendering methods based on the pod host computer provided in the embodiments of this application, the beneficial effects that any of the image processing and rendering methods based on the pod host computer provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0207] According to one aspect of this application, a computer program product or computer program is provided, comprising a computer program / instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instructions from the computer-readable storage medium and executes the computer program / instructions, causing the electronic device to perform the method provided in the above embodiments regarding image processing and rendering based on a pod-based host computer.

[0208] The above provides a detailed description of an image processing and rendering system and method based on a pod-based host computer, as provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A gondola host-based image processing and rendering system, comprising: The system is applied to a gondola host computer in communication connection with a gondola, the gondola host computer sends control instructions to the gondola, and receives real-time video streams and gondola state data returned by the gondola, and the system comprises: A video decoding unit is configured to read compressed video frames in frame sequence from the real-time video streams, and to decode and perform pixel format conversion processing on the compressed video frames to obtain video images conforming to a preset image format; An image storage unit is configured to store the video images in an image ready queue; A data alignment unit is configured to sequentially take out to-be-processed images from the image ready queue, and to obtain target gondola state data synchronous in time with the to-be-processed images; A character obtaining unit is configured to obtain to-be-overlaid characters and character overlay style parameters configured by a user for the to-be-processed images, wherein the to-be-overlaid characters are generated based on the target gondola state data and / or user-defined content; An image overlay unit is configured to draw the to-be-overlaid characters on the to-be-processed images according to the character overlay style parameters to generate overlaid images; A rendering unit is configured to render and display the overlaid images.

2. A method for image processing and rendering based on a gondola host computer, characterized in that, The method is applied to a gondola host computer in communication connection with a gondola, the gondola host computer sends control instructions to the gondola, and receives real-time video streams and gondola state data returned by the gondola, and the method comprises: Reading compressed video frames in frame sequence from the real-time video streams, and decoding and performing pixel format conversion processing on the compressed video frames to obtain video images conforming to a preset image format; Storing the video images in an image ready queue; Sequentially taking out to-be-processed images from the image ready queue, and obtaining target gondola state data synchronous in time with the to-be-processed images; Obtaining to-be-overlaid characters and character overlay style parameters configured by a user for the to-be-processed images, wherein the to-be-overlaid characters are generated based on the target gondola state data and / or user-defined content; Drawing the to-be-overlaid characters on the to-be-processed images according to the character overlay style parameters to generate overlaid images; Rendering and displaying the overlaid images.

3. The method of claim 2, wherein, The gondola host computer runs an image processing sub-thread and an image processing main thread; The reading of compressed video frames in frame sequence from the real-time video streams, and the decoding and pixel format conversion processing on the compressed video frames to obtain video images conforming to a preset image format, comprises: The image processing sub-thread reads compressed video frames in frame sequence from the real-time video streams, and decodes and performs pixel format conversion processing on the compressed video frames to obtain video images conforming to a preset image format, and sends image processing signals carrying the video images to the image processing main thread; The storing of the video images in an image ready queue, comprises: The image processing main thread stores the video images in an image ready queue according to the image processing signals.

4. The method of claim 2, wherein, The video images and the compressed video frames have a one-to-one correspondence; The storing of the video images in an image ready queue, comprises: acquire a frame sequence of the compressed video frame in the real-time video stream, and determine a collection timestamp corresponding to the compressed video frame according to a frame rate and a starting collection time of the real-time video stream; determine the collection timestamp as a timestamp of the video image based on a one-to-one correspondence between the video image and the compressed video frame; store the video image and the timestamp in the image ready queue; the target pod state data synchronized in time with the to-be-processed image comprises: determine target pod state data from the pod state data based on a time difference between the timestamp of the to-be-processed image and a collection time corresponding to the pod state data.

5. The method of claim 4, wherein, the target pod state data determined from the pod state data based on a time difference between the timestamp of the to-be-processed image and a collection time corresponding to the pod state data comprises: calculate absolute time differences between the timestamp of the to-be-processed image and collection times corresponding to each piece of pod state data; determine the pod state data with the smallest absolute time difference as the target pod state data.

6. The method of claim 2, wherein, the to-be-overlaid character and the character overlay style parameter configured by the user for the to-be-processed image comprise: in response to an editing operation on the to-be-processed image, display an image editing page, the image editing page comprising a custom area, a state configuration area and a style configuration area; in response to a content input operation on the custom area, acquire user-defined content; in response to a selection operation on the target pod state data in the state configuration area, determine a configuration state corresponding to the target pod state data; generate a to-be-overlaid character based on the configuration state, the user-defined content and the target pod state data; in response to a style setting operation on the to-be-overlaid character in the style configuration area, acquire corresponding character overlay style parameters.

7. The method of claim 6, wherein, the to-be-overlaid character generated based on the configuration state, the user-defined content and the target pod state data comprises: if the configuration state indicates that the target pod state data needs to be overlaid, combine the target pod state data and the user-defined content to generate the to-be-overlaid character; if the configuration state indicates that the target pod state data does not need to be overlaid, take the user-defined content as the to-be-overlaid character.

8. The method of claim 2, wherein, the to-be-overlaid character comprises a plurality of characters, and the character overlay style parameter comprises an overlay position and a character style corresponding to each character; the to-be-overlaid character is drawn on the to-be-processed image according to the character overlay style parameter to generate an overlay image, comprising: for each character, determine a pixel coordinate of the character in the to-be-processed image according to the overlay position corresponding to the character; based on the character style corresponding to the character, draw the character at the pixel coordinate to obtain an overlay image.

9. The method of claim 2, wherein, the rendering and display of the overlay image comprises: convert each overlay image into a target bitmap format to obtain a bitmap format overlay image; The superimposed image in the bitmap format is scaled based on a preset resolution to obtain a processed image with an adaptive display size; The processed image is sequentially output to a video display control according to the generation time sequence of each superimposed image to form a real-time video picture stream; The real-time video picture stream is rendered and displayed through the video display control.

10. The method of claim 2, wherein, Before the receiving of the real-time video stream and the gondola state data returned by the gondola, further comprising: establishing a communication connection with a preset gondola network address; based on the communication connection, obtaining the media description information of the video stream from the gondola indicated by the preset gondola network address; analyzing the media description information to obtain a video encoding format; creating a corresponding video decoder context according to the video encoding format; configuring the video decoding initialization parameters to obtain the video decoding configuration parameters, and applying the video decoding configuration parameters to the video decoder context; based on the configured video decoder context, completing the decoder initialization, so that the video decoder enters a ready state, so that when an image processing instruction is received, the real-time video stream returned by the gondola is received through the communication connection, and the real-time video stream is decoded by the video decoder in the ready state.

Images