Image real-time processing method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image vision, and provides an image real-time processing method and device, electronic equipment and a storage medium. A to-be-scheduled frame queue is generated by performing framing compression coding and metadata priority evaluation sorting on an original video stream, and multi-target optimization distribution is performed on the to-be-scheduled frame queue according to real-time state data of edge nodes to generate a local to-be-processed frame queue of each edge node. According to the scene event feature set, parallel decoding and frame-by-frame event recognition are carried out on compressed frames in the local to-be-processed frame queue to obtain a potential scene event set of each edge node, and time-space associated evidence fusion and context collaborative verification are carried out on the potential scene event set to generate an image monitoring report. According to the method, low-time-delay and high-reliability image real-time monitoring is realized under the condition that resources are limited through metadata priority ranking, multi-target optimization distribution of the real-time state of the edge node and event identification and space-time evidence fusion verification.

Description

Image real-time processing methods, devices, electronic equipment, and storage media Technical Field

[0001] This application relates to the field of image vision technology, and in particular to a real-time image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Vehicle-mounted or mobile edge devices with high image data timeliness requirements impose strict end-to-end constraints on real-time image processing. These constraints include not only extremely low capture-transmission-processing latency and controllable latency jitter to ensure event detection is completed within the available decision window, but also priority for key frames, controllable frame loss, and minimizing loss of critical information when bandwidth or computing power is limited. Simultaneously, these devices require real-time perception and adaptive adjustment to heterogeneous computing power (CPU / GPU / NPU / different peripherals) and varying wireless links (Wi-Fi Direct, hotspots, mobile networks), balancing energy consumption, thermal limitations, time synchronization, and privacy security. This ensures continuous and reliable output of event-level semantic information for alarm or control purposes in dynamic driving scenarios.

[0003] Existing real-time image processing methods generally follow two paths: one is based on low-latency local inference, using lightweight models (such as MobileNet / MoveNet, EfficientDet-Lite) to complete end-to-end detection on a single device, coupled with fixed frame rates, H.264 or MJPEG-based encoding, and simple queuing; the other is based on cloud / edge-cloud collaboration, achieving higher accuracy by uploading video or keyframes to the cloud for centralized inference, or using static weighted round-robin and threshold-based frame dropping strategies to alleviate bursty loads. In addition, common engineering practices include fixed priority labeling, experience-based bandwidth adaptation (reducing resolution or quality), and simple NMS and temporal filtering to reduce false alarms.

[0004] Existing methods lack end-to-end, real-time perception-driven collaborative scheduling and spatiotemporal evidence fusion mechanisms: single-device solutions are often forced to sacrifice key frame processing or accuracy when computing power or energy consumption is limited, while cloud collaborative solutions are difficult to meet the timeliness requirements for emergency event determination due to transmission delays and packet loss. Frame rate / quality adjustment based on static rules and simple polling scheduling cannot simultaneously optimize latency, recognition rate, and resource balance. Therefore, when facing heterogeneous nodes, dynamic networks, and multi-source uncertain evidence, it often leads to delays in key event detection or a significant decrease in confidence. Summary of the Invention

[0005] In view of this, this application provides a real-time image processing method, apparatus, electronic device, and storage medium to solve the problem of lack of passive response to dynamic load and heterogeneous failure in real-time image processing.

[0006] The first aspect of this application provides a real-time image processing method, the method comprising: performing frame-by-frame compression encoding and metadata priority evaluation and sorting on the original video stream to generate a queue of frames to be scheduled; performing multi-objective optimization allocation across devices on the queue of frames to be scheduled based on the received real-time status data of edge nodes to generate a local queue of frames to be processed for each edge node; performing parallel decoding and frame-by-frame event recognition on the compressed frames in the local queue of frames to be processed based on a preset scene event feature set to obtain a potential scene event set for each edge node; and performing spatiotemporal evidence fusion and contextual collaborative verification on the potential scene event set to generate an image monitoring report.

[0007] In an optional implementation, the step of performing frame-by-frame compression encoding and metadata priority evaluation and sorting on the original video stream to generate a queue of frames to be scheduled includes: performing pixel block division and two-dimensional discrete cosine transform on each frame image in the original video stream according to a preset pixel block size to obtain the frequency domain coefficient vector of each pixel block in each frame image; adjusting the preset compression intensity coefficient by linear transformation according to the detected current network bandwidth and the received real-time status data of edge nodes to generate a current compression quantization table, and performing nonlinear transformation and encapsulation on the frequency domain coefficient vector according to the current compression quantization table to obtain compressed frame data packets from each data source; extracting metadata from each compressed frame data packet according to a preset data type, evaluating the temporal importance coefficient corresponding to each metadata according to the current system time and the acquisition time of the metadata, and determining the priority weight coefficient corresponding to the metadata from a preset priority weight set according to the acquisition source type of the metadata; and performing priority evaluation and data packet sorting on the metadata according to the priority weight coefficient and the temporal importance coefficient to generate a queue of frames to be scheduled.

[0008] In an optional implementation, the step of performing cross-device multi-objective optimization allocation of the queue of frames to be scheduled based on the received real-time status data of the edge nodes to generate a local queue of frames to be processed for each edge node includes: parsing the received real-time status data to obtain the current processing queue length, historical average frame processing time, and network connection quality index of each edge node; using a preset performance evaluation strategy, weighting and summing the current processing queue length, the historical average frame processing time, and the network connection quality index based on the detected current operating temperature of each edge node and a preset scene weight coefficient set to calculate the real-time performance score of each edge node; normalizing the real-time performance score to obtain the allocation weight coefficient of each edge node; and using a preset improved genetic strategy, iteratively optimizing the allocation of the queue of frames to be scheduled based on the allocation weight coefficient, the current processing queue length, the historical average frame processing time, and the network connection quality index to generate a local queue of frames to be processed for each edge node.

[0009] In an optional implementation, the step of performing parallel decoding and frame-by-frame event recognition on compressed frames in the local frame queue to obtain potential scene event sets for each edge node based on a preset scene event feature set includes: performing entropy decoding and inverse quantization on each compressed frame in the local frame queue to obtain image spatial data corresponding to each compressed frame; performing bilinear interpolation scaling and color channel conversion on the image spatial data to generate a restored image corresponding to each compressed frame; performing semantic similarity calculation on the restored image based on the preset scene event feature set and the data acquisition source type corresponding to the compressed frame to identify scene feature data and type confidence values ​​of each frame image in each data acquisition source type; sorting the scene feature data according to the acquisition time order of each frame image in the same data acquisition source type to generate event records corresponding to the data acquisition source type; and aggregating the event records according to the local frame queue to generate potential scene event sets for each edge node.

[0010] In an optional implementation, the step of performing spatiotemporal evidence fusion and contextual collaborative verification on the potential scene event set to generate an image monitoring report includes: performing spatiotemporal alignment and grouping processing on the potential scene event set according to a preset sliding time window, scene acquisition distance threshold, acquisition source location data, and acquisition source type to obtain event groups to be fused within the same time window and the same scene; performing incremental probability allocation and synthesis based on the type confidence value of the corresponding image of each event record in the event group to be fused using a preset evidence fusion mechanism to calculate the matching confidence value between the event group to be fused and each scene event type in the scene event feature set; when the matching confidence value reaches a preset primary threshold, performing logical rationality verification of the scene events in the event group to be fused according to a preset scene context rule base to filter out scene recognition event groups, and marking and integrating the scene recognition event groups according to a preset item structure to generate monitoring item data; extracting a state summary set from the real-time state data according to a preset summary type, and encapsulating the monitoring item data and the state summary set into an image monitoring report according to a preset report template.

[0011] In an optional implementation, the method further includes: performing multi-dimensional data collection on the length change trend of the queue of frames to be scheduled, the processing latency information in the real-time status data of each edge node, and the number of currently active edge nodes through a preset load monitoring strategy to obtain a current load feature set; performing nonlinear fitting prediction of the load trend on the current load feature set based on preset historical load data to obtain a short-term load prediction value; when the short-term load prediction value reaches a preset latency tolerance threshold, calculating the load deviation difference between the short-term load prediction value and the latency tolerance threshold, so as to determine the target frame rate adjustment amount and the compression intensity adjustment amount based on the load deviation difference through a preset adjustment amount mapping table; and controlling the input rate and data volume of the original video stream based on the target frame rate adjustment amount and the compression intensity adjustment amount.

[0012] In an optional implementation, the improved genetic strategy includes a tournament selection mechanism and a hill-climbing local search mechanism. The step of using a preset improved genetic strategy to iteratively optimize the allocation of the queue of frames to be scheduled across devices based on multi-objective constraints, according to the allocation weight coefficient, the current processing queue length, the historical average frame processing time, and the network connection quality index, to generate local queues of frames to be processed for each edge node includes: Step S21, generating an initial allocation set based on the amount of metadata in the queue of frames to be scheduled, a preset number of edge nodes, and the allocation weight coefficient using a preset encoding strategy; Step S22, performing multi-objective fitness evaluation of the edge nodes for each allocation group in the initial allocation set based on the current processing queue length, the historical average frame processing time, and the network connection quality index to calculate the fitness of each allocation group. Step S23: Based on the fitness value, the tournament selection mechanism selects candidate allocation groups from the initial allocation group set, and performs genetic crossover and mutation on the allocation groups in the candidate allocation group set to generate offspring allocation groups. The offspring allocation groups are then optimized using the hill-climbing local search mechanism, and the optimized offspring allocation groups are updated as the initial allocation group set. Step S24: The number of iterations is accumulated according to a preset counting rule to obtain the current iteration number. Steps S22 to S24 are repeated until the variance of the fitness value is less than a preset convergence threshold, or the current iteration number reaches a preset maximum generation threshold. Based on the fitness value, the allocation group corresponding to the highest fitness value is selected from the initial allocation group set as the optimal allocation group, and the optimal allocation group is converted into a local unprocessed frame queue for each edge node using a preset decoding strategy.

[0013] A second aspect of this application provides a real-time image processing apparatus, comprising: a scheduling and sorting module for performing frame-by-frame compression encoding and metadata priority evaluation and sorting on the original video stream to generate a queue of frames to be scheduled; a node allocation module for performing multi-objective optimization allocation across devices on the queue of frames to be scheduled based on received real-time status data of edge nodes to generate a local queue of frames to be processed for each edge node; a parallel recognition module for performing parallel decoding and frame-by-frame event recognition on the compressed frames in the local queue of frames to be processed based on a preset scene event feature set to obtain a potential scene event set for each edge node; and a fusion verification module for performing spatiotemporal evidence fusion and contextual collaborative verification on the potential scene event set to generate an image monitoring report.

[0014] A third aspect of this application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the real-time image processing method described above.

[0015] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the real-time image processing method described above.

[0016] In summary, this application includes at least the following beneficial technical effects: 1. By using frame-level priority sorting, adaptive compression and frame rate closed-loop adjustment, as well as weighted cross-device scheduling and priority-aware transmission, it can prioritize the timely processing of key frames when bandwidth / computing power is limited, thereby reducing end-to-end latency.

[0017] 2. After parallel decoding and frame-by-frame recognition at each edge node, the use of spatiotemporal alignment combined with evidence fusion and contextual verification can effectively suppress single-source false alarms / missed alarms and improve the confidence of the final decision.

[0018] 3. Weight allocation based on real-time performance profiles and multi-objective (e.g., latency, load, transmission overhead) optimization allocation can balance computing load. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 is a flowchart of a real-time image processing method provided in an embodiment of this application; Figure 2 is a functional block diagram of a real-time image processing device provided in an embodiment of this application; Figure 3 is a structural schematic diagram of an electronic device provided in an embodiment of this application; Figure 4 is a schematic diagram of the operating environment of a real-time image processing method provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Figure 4 illustrates the operating environment of the real-time image processing method provided in this embodiment. The real-time image processing method in this embodiment runs on a hierarchical in-vehicle distributed edge computing system. This system uses a real-time image processing device as its core execution and coordination hub, integrating various heterogeneous computing devices and image acquisition devices within the vehicle to jointly construct a collaborative, resource-scalable real-time analysis environment. By aggregating distributed computing resources, it meets the stringent requirements of low-latency, high-throughput intelligent analysis of multiple video streams in driving safety scenarios. In this embodiment, the real-time image processing device is a software system or application deployed on an in-vehicle computing node, adopting a logically centralized but physically distributed architecture. It typically employs a master instance (running at the central node in the diagram) and multiple subordinate instances (running at the edge nodes in the diagram) working collaboratively.

[0023] The central node carries the core control and coordination module of the real-time image processing device in this application embodiment, and is responsible for global management and task scheduling. Specifically, the central node includes the following functions: 1. Automatically discover and register available edge nodes in the vehicle to build a dynamic resource pool; 2. Receive and manage video frames from all image acquisition sources, prioritize them, and form a queue of frames to be scheduled; 3. Based on the real-time acquired edge node performance data (as described in claim 3), run algorithms such as improved genetic strategies to make optimal frame allocation decisions; 4. Receive the analysis results returned by each edge node, perform evidence fusion and context verification operations, and generate the final image monitoring report; 5. Implement load prediction and adaptive frame rate / compression intensity adjustment, and issue control commands to the image acquisition sources.

[0024] Each edge node carries a worker instance of the real-time image processing device in this application embodiment, responsible for the specific computing tasks in this application embodiment—receiving and processing tasks assigned by the central node. The functions of the edge nodes include: 1. Obtaining compressed frames from the local queue and performing operations such as decoding and image preprocessing in parallel; 2. Loading and running specific AI analysis models (corresponding to scene event feature sets), recognizing images, and generating preliminary event results (i.e., potential scene event sets); 3. Continuously monitoring the performance indicators of this node (such as CPU utilization, queue length, and processing latency), and periodically reporting to the coordinator of the central node to provide data support for dynamic scheduling. The edge nodes and the central node form a distributed computing network through a high-speed in-vehicle wireless local area network. Typically, the central node creates a Wi-Fi hotspot or utilizes Wi-Fi Direct technology to enable each edge node to access the same private network. Edge nodes can be authorized users' personal mobile devices, such as smartphones, tablets, or in-vehicle infotainment systems. When executing the real-time image processing method in this application embodiment, they contribute idle CPU / GPU / NPU computing resources, memory, and battery / vehicle power supply, transforming into a group of collaborative edge servers. Each edge node independently executes the video frame analysis task issued by the central node and sends the results back. Their dynamic joining or leaving (such as when the device battery is low or the user leaves the vehicle) is sensed and responded to by the central node in real time.

[0025] The operational environment diagram also includes image acquisition sources that communicate directly with the central node via an in-vehicle wired network or a high-speed wireless connection (such as Wi-Fi 6E). These image acquisition sources include: an external wide-angle camera facing the road environment in front of / behind the vehicle, continuously acquiring high-definition video streams to detect external environmental events such as vehicles, pedestrians, traffic signs, lane lines, and obstacles; and a built-in infrared camera facing the driver's cabin, acquiring video streams of the driver's face, hands, and posture to monitor internal state events such as driver fatigue, distraction, and dangerous actions. The image acquisition sources are not only responsible for raw video acquisition but also need to receive and execute adaptive adjustment commands from the central node of the real-time image processing device, dynamically adjusting the frame rate, resolution, or encoding parameters to achieve closed-loop load balancing.

[0026] The following explanation of the real-time image processing method of this application will be provided from the perspective of a real-time image processing device consisting of a central node and several edge nodes, combined with the usage environment of a dashcam.

[0027] Figure 1 shows a flowchart of a real-time image processing method provided in an embodiment of this application. The real-time image processing method provided in this embodiment includes the following steps.

[0028] Step S1: Perform frame-by-frame compression encoding and metadata priority evaluation and sorting on the original video stream to generate a queue of frames to be scheduled.

[0029] It should be understood that the raw video stream specifically refers to the sequence of uncompressed or only preliminarily processed digital video signals continuously captured and output at a constant frequency by camera sensors installed at different locations on the vehicle body during vehicle operation. Specifically, the embodiments of this application include at least two independent image acquisition sources: an outward-facing camera facing the external road environment of the vehicle, and an inward-facing camera facing the driver's cabin. Each camera captures a raw data stream containing the raw pixel array, time synchronization signal, and sensor parameters at a rate of, for example, thirty frames per second. This raw video stream constitutes the source data of the entire real-time processing flow. Its continuous nature requires subsequent processing stages to have high throughput and low latency capabilities. Any processing backlog or delay will directly lead to a lag in the detection of safety events, thus violating the original intention of driving safety applications.

[0030] When the original video stream is input to the central node of the real-time image processing device, the temporally continuous analog or digital video stream needs to be divided into a series of discrete, independently processable static image units, i.e., frames. Each frame encapsulates complete visual scene information at a specific moment. Frame-by-frame processing is performed based on the frame synchronization signal embedded in the video stream or according to a preset fixed time interval, thereby generating an original image set strictly ordered by timestamps. After obtaining the original image set, each frame needs to be preprocessed and transformed to meet compression and transmission requirements. First, a single frame image is divided into multiple non-overlapping pixel blocks according to a preset pixel block size, providing more manageable and optimized data units by localizing the global image processing task. The pixel value in each pixel block represents the brightness or color intensity at a specific location; after the Discrete Cosine Transform (DCT) in this embodiment, the data is represented as a series of frequency component coefficients, where low-frequency coefficients represent the general outline and background information of the image, and high-frequency coefficients correspond to the details and edges of the image. The Direct Current Transform (DCT) operation concentrates the energy of the data—that is, the main visual information of the image is compressed into a few low-frequency coefficients, while the amplitudes of a large number of high-frequency coefficients, which have little impact on visual perception, approach zero. This facilitates a significant reduction in data volume for compression operations without substantial loss of subjective image quality. After the DCT transformation, each frame of the image is converted into a set of frequency domain coefficient vectors composed of the frequency coefficients of each pixel block.

[0031] To achieve a dynamic balance between compression efficiency and image quality, adapting to the network environment and computational load, this application employs the following compression intensity adjustment mechanism: Based on a preset quantization table, which defines initial quantization step sizes for components at different frequency positions, the compression intensity coefficients in the table serve as scaling factors for these quantization step sizes. The compression intensity coefficients are dynamically adjusted based on two real-time feedback parameters: the available bandwidth of the current in-vehicle network and the overall processing load reflected in the real-time status data converged from each edge node to the central node. When network bandwidth is limited or edge node computing resources are strained, this application's device tends to employ a more aggressive compression strategy to reduce data volume, thereby reducing transmission latency and alleviating processing pressure; conversely, when resources are abundant, the compression quality is moderately increased to retain more detail and improve analysis accuracy. Specifically, through a preset coefficient mapping table, the detected current network bandwidth and computational load quantization values ​​(i.e., the real-time status data of edge nodes) are mapped to linear transformation parameters of each compression intensity coefficient in the basic quantization table. The linear transformation in this application can be described as a scaling operation: Q c (u,v)=α×Q b (u,v), where Q b (u,v) is the quantization step size of the basic quantization table at the frequency coordinate (u,v), α is a scaling factor dynamically determined by the current network bandwidth and the real-time status data of the edge nodes through a coefficient mapping table, and Q c (u,v) represents the corresponding value in the current compression quantization table obtained after adjustment. The above operations yield a current compression quantization table that matches the current operating environment.

[0032] Subsequently, the frequency domain coefficient vector obtained above is subjected to a nonlinear transformation using the current compression quantization table, i.e., the quantization process. Quantization is a key step in image compression to reduce data volume. Each frequency domain coefficient is removed using a large quantization step size, and the resulting data is rounded (usually by rounding). The quantization step size is typically set larger for high-frequency components, causing many low-amplitude high-frequency coefficients to become zero after division and rounding. Therefore, the compression operation used in this application is lossy compression, but by precisely controlling the values ​​of the quantization table, a very high compression ratio can be achieved with acceptable visual quality loss. After quantization, non-zero coefficients are mainly concentrated in the low-frequency region, and their values ​​are significantly reduced. Next, entropy encoding, such as using Huffman coding, is performed on the quantized coefficients to further utilize their statistical properties to eliminate redundancy, ultimately generating the compressed coding set for the frame image. To facilitate data transmission and scheduling, the compressed coding sets from different image acquisition sources also need to be encapsulated with necessary control information. The encapsulation operation used in this application adds a header to the compressed data of each frame. The header explicitly contains key information such as data source identifier, frame sequence number, timestamp, compression parameters, and data packet length, thereby generating self-contained and independently parsable compressed frame data packets, ensuring the integrity and traceability of data when it flows in the distributed system.

[0033] Next, the central node further extracts metadata of a preset data type from the compressed frame data packet. This metadata describes the attributes of the data packet itself, including at least the acquisition timestamp, frame sequence number, data source identifier, and compression parameters, without fully decompressing the image content itself. After obtaining the metadata, the difference between the current system time and the acquisition timestamp in the metadata is immediately calculated. In this application, this difference is defined as the temporal importance coefficient. The temporal importance coefficient directly reflects how long the frame data has been waiting in the system. The larger the value, the more severe the real-time value decay of the frame. If it is not prioritized, its analysis results may lose their warning significance due to excessive lag. For example, a frame acquired 100 milliseconds ago that may contain suddenly appearing pedestrian images must receive a higher processing priority than a normal road surface frame acquired 10 milliseconds ago. When evaluating priority, the data source is another decisive factor. This application determines the corresponding priority weight coefficient from a preset priority weight set based on the acquisition source type in the metadata. This weight set typically assigns different inherent importance levels to different acquisition sources. For example, in a driving safety scenario, data from outward-facing cameras, responsible for detecting road hazards, is pre-prioritized with a higher weighting coefficient than data from inward-facing cameras, responsible for monitoring driver status. This is because the analysis results of the former directly relate to avoiding immediate safety risks such as collisions, while the latter, although important, has a slightly different level of urgency. Ultimately, the priority score of this frame's metadata is calculated by combining the temporal importance coefficient and the priority weighting coefficient. The evaluation function used in this embodiment multiplies the two (in another optional implementation, a weighted summation can also be used) – that is, priority score = priority weighting coefficient × temporal importance coefficient. Through the above priority evaluation, the abstract concepts of "importance" and "urgency" are transformed into a comparable numerical score, providing a precise ranking basis for global scheduling.

[0034] Finally, the central node sorts all compressed frame data packets in descending order based on their calculated priority scores. The data packet with the highest score, i.e., the most urgent and important frame, is placed at the front of the queue, resulting in a well-structured and ordered queue of frames to be scheduled. This queue serves as the input source and scheduling basis for the distributed processing flow. Its orderliness ensures that limited edge computing resources can be prioritized for processing image data that is most likely to contain critical security information and has the highest timeliness requirements. This optimizes the end-to-end detection latency of critical events at the system level, meeting the high timeliness requirements of real-time image processing for dashcams.

[0035] Step S2: Based on the received real-time status data of the edge nodes, perform multi-objective optimization allocation across devices for the queue of frames to be scheduled, and generate a local queue of frames to be processed for each edge node.

[0036] It should be understood that real-time status data is a set of quantitative indicators reflecting the instantaneous working status of each edge node, periodically collected and actively reported by worker modules running on each edge node. Given the stringent real-time processing requirements of dashcams, no single performance dimension can accurately describe the ability and efficiency of an edge node in handling new tasks; therefore, comprehensive observation from multiple levels is necessary. Real-time status data includes the current processing queue length, historical average frame processing time, and network connection quality indicators. The current processing queue length directly reveals the backlog of tasks waiting to be processed on the edge node. A continuously growing queue indicates that the edge node is under high load; continuing to allocate tasks will prolong the queuing time for new tasks, jeopardizing the overall system latency target. The historical average frame processing time reflects the statistical level of the time required for an edge node to complete a single image analysis calculation. This indicator is limited by factors such as node hardware performance, current CPU frequency, and background process resource usage, and is a key measure of its inherent computing speed. Network connectivity quality metrics, such as round-trip time or packet loss rate, characterize the reliability of the data transmission channel from the central node to the edge node. Poor network quality introduces additional, unpredictable transmission delays and may even lead to data retransmissions, compromising the determinism of processing. By parsing real-time status data, the abstract edge node state is transformed into a series of structured parameters that can be used for quantitative analysis and decision-making.

[0037] After obtaining the aforementioned basic indicators, the central node calculates the real-time performance score of each edge node using a pre-defined performance evaluation strategy. This strategy assigns different weights to each indicator based on its impact on the efficiency of real-time processing tasks. The performance evaluation strategy incorporates the current operating temperature of the edge nodes as a key adjustment factor. Mobile devices generate heat during sustained high-load computation. When the temperature exceeds a certain threshold, the system's integrated thermal management strategy forcibly reduces the CPU / GPU operating frequency to protect the hardware, leading to a precipitous drop in computing performance. In automotive environments, devices may be exposed to direct sunlight, making the temperature rise effect even more significant. Therefore, performance evaluation must proactively consider the impact of temperature. Furthermore, this application uses a pre-defined set of scenario weight coefficients to define basic weights for queue length, processing time, network quality, and temperature impact. Specifically, the basic computing performance weights are first reduced based on the current temperature. For example, when the temperature is detected to be close to the device's frequency reduction threshold, the negative weight of the historical average frame processing time indicator is amplified. Subsequently, the adjusted indicator weights are linearly weighted and summed with their corresponding normalized values ​​to calculate a comprehensive real-time performance score. Real-time performance scoring uses a single numerical value to comprehensively characterize the expected performance of an edge node in receiving and processing a new frame task at the current moment, taking into account its computing power, current load, communication status, and thermal state. Edge nodes with higher scores are considered more powerful computing resource providers. To enable comparison of scores from different nodes on the same scale and direct use in calculating task allocation ratios, the real-time performance scores of all edge nodes need to be normalized. Specifically, each edge node's score is divided by the sum of all node scores, ensuring that the sum of the normalized values ​​for all nodes is one. The resulting value is the allocation weight coefficient for that edge node. The allocation weight coefficient intuitively represents the proportion of the total system workload that the node should ideally handle. For example, in a system with three edge nodes, if their allocation weight coefficients are 0.5, 0.3, and 0.2, it indicates that under perfectly balanced conditions, the first node should handle 50% of the frames, the second node 30%, and the third node 20%. This proportional allocation operation based on real-time performance is the theoretical basis for achieving dynamic load balancing.

[0038] However, simple proportional allocation ignores the complex matching relationship between task attributes and node states, as well as the mutual constraints between multiple objectives. Therefore, this application employs a pre-defined improved genetic strategy to perform the final, refined cross-device multi-objective optimization allocation. This strategy aims to solve a complex combinatorial optimization problem: how to allocate numerous frames in the queue of frames to be scheduled to various edge nodes in order to minimize the overall task completion time, maximize load balancing, and minimize additional latency caused by network transmission, while satisfying the processing capacity constraints of each node. The genetic strategy simulates the biological evolution process, first generating a series of possible allocation schemes as an initial population; this process is called encoding and initialization. Through a pre-defined encoding strategy, the allocation scheme is expressed as chromosomes. A direct encoding method is to use a sequence of length equal to the number of frames in the queue of frames to be scheduled, where the value at each position in the sequence represents the edge node number to which the frame is allocated. The population initialization process randomly generates multiple such chromosomes to form an initial allocation set, but the random generation takes into account the aforementioned allocation weight coefficients, so that the probability of each node being randomly selected is roughly proportional to its weight coefficient, thus providing a better initial starting point for evolution.

[0039] Next, the iterative optimization phase begins. In each iteration, a multi-objective fitness evaluation is first performed on each allocation group in the current population. The fitness function used in this application transforms state data such as "current processing queue length," "historical average frame processing time," and "network connection quality indicators," as well as the structure of the allocation scheme itself, into a fitness value that measures the merits of the scheme. This function typically calculates three key sub-objectives: first, estimating the total processing time for all frames, which requires considering the existing queue lengths of each node, the amount of newly allocated tasks, and their historical processing times; second, assessing the degree of imbalance in the final load among all nodes; and third, calculating the total overhead caused by cross-node data transmission. These three sub-objectives often conflict with each other. In the embodiments of this application, the fitness function combines them into a single scalar value (i.e., the fitness value) through weighted summation. A higher fitness value indicates a better scheme.

[0040] Subsequently, the improved genetic strategy employs a tournament selection mechanism to screen candidate allocation groups from the current population. This mechanism randomly selects a small number of individuals for comparison, retaining only the individuals with the highest fitness into the mating pool, repeating this process until the mating pool is full. This selection method ensures the preservation of excellent genes while maintaining a certain level of population diversity, avoiding premature convergence. Next, genetic crossover and mutation processes are applied to the allocation groups in the candidate allocation group set to generate offspring allocation groups. The crossover operation randomly swaps segments of two parent chromosomes, combining them to generate new schemes; the mutation operation randomly changes the values ​​of individual positions on the chromosomes, introducing new exploration possibilities. Furthermore, a hill-climbing local search mechanism is introduced to optimize these newly generated offspring allocation groups. Hill-climbing, as a local optimization algorithm, attempts to make subtle adjustments to the offspring allocation groups, such as randomly swapping nodes belonging to two frames. If this swap improves the fitness value of the allocation group, the change is accepted. This mechanism is like adding a fast learning phase to the evolutionary process, accelerating the local refinement of excellent solutions and improving convergence speed and quality. The offspring allocation set optimized by local search will be updated to the initial allocation set of the new generation.

[0041] Each complete evaluation, selection, crossover, mutation, and local search constitutes an iteration. The system accumulates the number of iterations according to a preset counting rule and monitors changes in population fitness. This evolutionary cycle is repeated until preset termination conditions are met: either the variance of the population's fitness values ​​is less than a convergence threshold, indicating that the population has converged and found an approximate optimal solution; or the number of iterations reaches a preset maximum generation threshold to prevent infinite loops. When the iteration terminates, the allocation group with the highest fitness value is selected from the final population and determined as the optimal allocation group.

[0042] Finally, the optimal allocation group, encoded in chromosome form, is converted into actual operation instructions using a preset decoding strategy. The decoding process directly parses the chromosome sequence: if the value at position i in the sequence is j, it means that the i-th compressed frame data packet in the queue of frames to be scheduled is sent to edge node j. The central node distributes the frame data packets to the corresponding edge nodes through the network based on this mapping relationship. After receiving the frame data packets assigned to it, each edge node arranges them according to their arrival order or in combination with simple local priority rules (such as following the original global priority in the data packets), forming its local queue of frames to be processed.

[0043] Step S3: Based on the preset scene event feature set, perform parallel decoding and frame-by-frame event recognition on the compressed frames in the local frame queue to be processed, and obtain the potential scene event set of each edge node.

[0044] Step S3 of this application begins with each edge node processing its local queue of frames to be processed. This queue is a list formed by compressed frame data packets arriving at the edge node via network transmission, arranged in the order of reception, after the central node has optimized its allocation across devices. Each compressed frame data packet encapsulates efficiently compressed and encoded image data and its metadata, serving as the basic unit to be analyzed. As distributed computing units, the core task of the edge nodes is to restore this compressed data to visual information and identify meaningful scene events. This process must be completed under strict timing constraints to ensure the timeliness of driving safety warnings.

[0045] First, edge nodes sequentially or in parallel retrieve compressed frame data packets from their local queue of frames to be processed, performing entropy decoding and inverse quantization to obtain the image spatial data corresponding to each compressed frame. Entropy decoding is the inverse process of compression coding. Before encapsulation, compressed frame data packets undergo entropy coding to further eliminate statistical redundancy; for example, Huffman coding is used to represent frequently occurring quantization coefficients with short codes and less frequent ones with long codes. Entropy decoding, based on a preset code table, accurately restores the received binary bitstream into a series of integer values, representing the quantized frequency domain coefficients. Inverse quantization is then performed, utilizing the quantization step size information corresponding to the encoder to recover the integer values ​​as approximate original frequency domain coefficients. Specifically, each original frequency domain coefficient is approximated by multiplying by the corresponding quantization step size: C r (u,v)=C q (u,v)×Q(u,v), where C q (u,v) is the integer obtained from entropy decoding, Q(u,v) is the quantization step size, and C r (u,v) are the reconstructed frequency domain coefficients. Because irreversible information loss occurs during lossy image compression, inverse quantization can only approximate the recovery, and its accuracy directly affects the subsequent image quality. After inverse quantization, the resulting image is in the frequency domain. At this point, the inverse discrete cosine transform (DCT) needs to be applied to convert the data back from the frequency domain to the spatial domain. The inverse DCT transforms the frequency domain coefficients of each 8x8 pixel block back to pixel values ​​in the spatial domain, mathematically achieved through a linear combination of a set of cosine basis functions. After this series of transformations, the originally compressed and encoded bitstream is finally reconstructed into image spatial data that can be intuitively understood and further processed—that is, an image represented by a matrix of pixel values ​​(i.e., image spatial data).

[0046] After obtaining the image space data, the original reconstructed image may not match the input requirements of the subsequent analysis model in terms of size and color space. Therefore, it is necessary to further perform bilinear interpolation scaling and color channel conversion on the image space data according to a preset data format to generate the restored image corresponding to each compressed frame. The bilinear interpolation scaling used in this application is an image resampling technique used to adjust the image to a fixed size required by the model. Unlike simple nearest neighbor interpolation, bilinear interpolation considers the weights of the four neighboring pixels around the target pixel in the original image, calculating new pixel values ​​through linear interpolation, thereby producing smoother edges and reducing jagged edges when zooming in or out. In dashcam scenarios, the original resolutions from different cameras may vary, but event recognition models typically require a uniform input size; scaling ensures data consistency. The color channel conversion used in this application is because the original image acquisition often uses YUV or YCbCr color spaces to optimize compression efficiency, while most deep learning models use the RGB color space during training. The conversion process requires calculations according to the standard color space conversion matrix. For example, the conversion from YUV to RGB involves a linear transformation: R = Y + 1.402 × (V - 128), G = Y - 0.344136 × (U - 128) - 0.714136 × (V - 128), B = Y + 1.772 × (U - 128). These conversion operations ensure that the image's color representation meets the requirements of subsequent processing. After scaling and conversion, the resulting restored image not only visually approximates the original scene but also fully adapts to subsequent input specifications in terms of data structure.

[0047] Subsequently, the edge nodes enter the core event recognition stage. The pre-defined scene event feature set is a predefined, structured knowledge base that defines possible event categories and their feature representations for different image acquisition source types. For example, for an outward-facing camera facing the road, the feature set might include categories such as "vehicles," "pedestrians," "traffic signs," and "obstacles"; for an inward-facing camera facing the cockpit, the feature set might include categories such as "driver with eyes closed," "holding a mobile phone," and "eyes off-target." The operation used in this embodiment to calculate the semantic similarity between the scene event feature set and the reconstructed image is implemented through a deep neural network model. This model takes the reconstructed image as input, performs multiple convolutions and nonlinear transformations to extract high-level semantic feature vectors, and then measures the similarity between the data acquisition source type of this feature vector and the reference feature vector for each event category in the scene event feature set. Cosine similarity is commonly used for similarity calculation: Sim=(AB) / (||A||||B||), where A is the image feature vector and B is the event category feature vector. The calculated values ​​range from [-1, 1], with values ​​closer to 1 indicating greater semantic similarity. For each event category, the model outputs a type confidence value, representing the probability that the image belongs to a particular scene event category. Simultaneously, the recognition process may also output more refined scene feature data, such as the bounding box coordinates of the target event, keypoint locations, or semantic segmentation masks. Transforming pixel-level image data into symbol-level semantic information through these operations is key to giving the system the ability to understand scenes. For example, the system can no longer simply perceive a set of colors and shapes in an image; it can also determine that there is a person in the image with a confidence level of 92%.

[0048] After recognition, for the same data acquisition source type, the recognition results of each frame are discrete and independent. To form a continuous and contextualized event stream, the scene feature data needs to be sorted according to the acquisition time order of each frame within the same data acquisition source type, generating event records corresponding to that data acquisition source type. This application uses an acquisition timestamp embedded in the compressed frame metadata as the sorting basis. This timestamp is generated during image acquisition and remains unchanged throughout the processing chain. After sorting by time, the recognition results of multiple frames belonging to the same acquisition source are organized into a time series. Each event record not only contains the event type, confidence level, and feature data identified in that frame, but also is associated with a precise timestamp and frame sequence number. This temporal organization ensures that events are no longer isolated snapshots, but rather constitute a preliminary trajectory of the event's evolution over time. For example, if an outward-facing camera detects a "pedestrian" in three consecutive frames, with the location gradually approaching the lane, this time series pattern itself may be an important risk indicator. The above sorting by acquisition source ensures that subsequent analysis can provide a coherent interpretation from a specific perspective.

[0049] Finally, the edge nodes need to integrate all processing results. Based on the event records aggregated from their local queues of frames to be processed, a potential scene event set is generated for each edge node. This means that each node collects all event records generated by all compressed frames processed from its local queue in this round, regardless of their source type, into a unified, structured dataset. The potential scene event set not only contains the event records themselves but may also include metadata about the processing process, such as node ID, start and end times, and average processing latency. The potential scene event set represents all the "preliminary conclusions" reached by the edge node after independently analyzing the assigned video segments. It is called "potential" because these conclusions are based on single-node, single-frame, or short-time-series analysis and may contain false positives, false negatives, or uncertainties, requiring fusion and verification by the central node across a broader spatiotemporal dimension. After generating this set, the edge node sends it back to the central node via the network.

[0050] Step S4: Perform spatiotemporal evidence fusion and contextual collaborative verification on the potential scene event set to generate an image monitoring report.

[0051] First, the central node performs spatiotemporal alignment and grouping of all potential scene event records gathered, based on a preset sliding time window, scene acquisition distance threshold, acquisition source location data, and acquisition source type, to obtain event groups to be fused that belong to the same scene within the same time window. The sliding time window is a dynamic time interval, such as 500 milliseconds, that slides along the time axis in fixed increments. The time window is set based on reasonable expectations of event duration; too short a window may break up consecutive events, while too long a window may confuse unrelated events. The scene acquisition distance threshold is used to determine whether different acquisition sources spatially observe the same scene. In a vehicle environment, different cameras have fixed installation positions, and their location data is known. By calculating the target position reported in the event records or based on the geometric relationship of the camera's viewpoint, the proximity of the physical areas represented by different records can be estimated. The acquisition source type is used to distinguish observation perspectives. For example, records from a forward-facing camera and a driver monitoring camera should not be directly fused even if they are close in time, because they focus on completely different scenes. The grouping process involves clustering event records that fall within the same sliding time window, have a spatial distance less than the acquisition distance threshold, and share the same acquisition source type into a single event group to be fused. For example, suppose that within a 300-millisecond time window, three consecutive frames from the left-front camera detect a "pedestrian," and the pedestrian's position in the image changes continuously but remains spatially close. These three event records would then be grouped into the same event group to be fused. This process organizes massive amounts of fragmented event records according to their inherent spatiotemporal correlations.

[0052] Next, incremental probability allocation and synthesis are performed through a pre-defined evidence fusion mechanism to calculate the matching confidence values ​​of the event group to be fused and each scene event type in the scene event feature set. The evidence fusion mechanism used in this application aims to handle recognition results from different frames that may contain uncertainties. Its core idea is to treat each event record as evidence supporting or opposing a certain event hypothesis, and to use a mathematical model to quantify the overall confidence after combining the evidence. A commonly used and effective method is the Dempster-Shafer evidence theory. Within this theoretical framework, the recognition frame contains all possible event type hypotheses. Each event record is a basic probability allocation function on this recognition frame, which assigns a probability mass to each event type hypothesis (or set of hypotheses), derived from the type confidence value of the record. The incremental probability allocation and synthesis in this application refer to the repeated application of the Dempster combination rule to merge old and new evidence as records within the group are added sequentially in chronological order. Two independent basic probability allocation functions are used. and For example, its combined allocation function For any nonempty hypothesis A, we have ,in The degree of conflict between pieces of evidence is measured. By iteratively combining the evidence from all records within a group, a comprehensive basic probability assignment for that event group is finally obtained. From this assignment, the matching confidence value of the event group belonging to each specific event type in the scene event feature set can be extracted, usually expressed as a confidence function value for a single-point hypothesis. For example, a group containing three frames of "pedestrian" detection records, after evidence fusion, may have a matching confidence of 0.95 for the "pedestrian" event, while the confidence of belonging to the "vehicle" event may be only 0.02. The above operation provides a rigorous mathematical tool to integrate multi-frame information, reducing the impact of false positives or false negatives in a single frame, thereby obtaining a more robust event type judgment.

[0053] When the calculated matching confidence value reaches a preset primary threshold, it indicates that the event group has a high probability of representing a real-world scene event. However, high confidence is not entirely equivalent to logical plausibility. Therefore, it is necessary to verify the logical plausibility of the scene events in the event group to be fused based on a preset scene context rule base to filter out truly credible scene recognition event groups. The scene context rule base is a set of logical rules that encode domain knowledge and are used to check the plausibility and consistency of events in a specific environment. These rules may include temporal consistency, spatial continuity, physical probability between events, and scene-specific constraints. For example, one rule might stipulate that "in a highway driving scenario, the confidence of detecting a 'stop sign' should be significantly reduced unless a low-speed driving or construction zone sign is detected simultaneously"; another rule might require that "the detection of a 'driver closing their eyes' event must be compatible with the state of 'the vehicle is driving straight and the lane line is stable' to avoid false alarms when the driver blinks normally." The verification process in this application compares the event type, spatiotemporal attributes, and known states of the event group to be fused with other events or systems within the group (such as current vehicle speed or geographical location) to check for any rule violations. If a violation is found, its confidence level may be lowered or it may be directly marked as suspicious and excluded. Only event groups that pass the verification of all applicable rules will be retained as scene recognition event groups. For example, if the system fuses a high-confidence event of "the vehicle in front suddenly brakes," but the context rules detect that the actual distance between the current vehicle and the vehicle in front is increasing, the event may be judged as unreasonable and filtered out. By introducing knowledge-based reasoning, the shortcomings of a purely data-driven model are compensated for, significantly improving the reliability and interpretability of the system output.

[0054] For each selected scene identification event group, key evidence frames need to be marked and integrated according to a preset item structure to generate standardized monitoring item data. The preset item structure defines the fields that each monitoring item should include, such as: unique event identifier, event type, final fusion confidence level, time range of event occurrence, list of involved data sources, key evidence frame identifier, and event descriptive parameters (such as location, speed, size, etc.). Key evidence frame marking involves selecting one or several of the most representative frames from the original frames corresponding to the event group, such as the single frame with the highest confidence level, or multiple frames that best show the start, peak, and end states of the event, and associating its frame sequence number or storage index with the item. The integration process involves filling all relevant information of the event group into the various fields of the item structure. The generated monitoring item data is a structured data object that encapsulates a complete information summary about an independent scene event, making it easy for both humans to read and for machines to process later. For example, a monitoring entry for "pedestrians crossing the road" might include an event type of "pedestrian crossing", a confidence level of 0.88, a time range of t1 to t3, key evidence frames of frames 123, 125, and 127, and the image coordinates of the starting and ending points of the pedestrian crossing.

[0055] Finally, to generate a complete image monitoring report, a status summary set needs to be extracted from the system's real-time status data according to a preset summary type. The status summary set used in this application aims to provide an overview of the system's overall operational status at the time of report generation. Its content may include: the total duration of video streams processed within the reporting period, the number of edge nodes involved in the calculation and their average utilization rate, the total number of event detections, the distribution of various event types, the average processing latency, and the system health score. This data originates from the real-time status data continuously monitored by the central node. Simultaneously, according to a preset report template, all monitoring item data generated within this period are encapsulated with the status summary set. The report template defines the report's structure, format, and presentation method, and may use JSON, XML, or a custom binary format. The encapsulation process uses the monitoring item data as the main content of the report and the status summary set as an appendix or header information, packaging them together to form the final image monitoring report. The image monitoring report not only lists the scene events confirmed after rigorous verification and fusion, along with their detailed evidence, but also provides the system's own performance statistics, enabling the report to be used for both real-time security alerts and subsequent log analysis and system optimization. Thus, the complete process from distributed perception to centralized collaborative decision-making is realized, and the original video stream is transformed into a concise, reliable, and information-rich semantic report, meeting the dual requirements of timeliness and accuracy for driving safety applications.

[0056] In an optional implementation, to proactively maintain a dynamic balance between system processing capacity and input load, thereby fundamentally ensuring the continuous satisfaction of real-time requirements, the method further includes the following closed-loop control process: First, a preset load monitoring strategy is used to periodically and continuously monitor the system's operating status. The load monitoring strategy defines the frequency, source, and indicators of data collection. Specifically, the central node accesses its maintained queue of frames to be scheduled at fixed intervals, calculates the instantaneous value of the queue length and its rate of change over the past few sampling periods (i.e., the length change trend), and the increasing trend of the length change directly indicates a potential insufficiency in the system's processing capacity. Simultaneously, the central node parses the latest real-time status data packets reported by each edge node, extracting the processing latency information contained therein. This information reflects the time elapsed from when a frame is allocated to a node to when the identification result is returned, and is a direct indicator of the system's actual processing speed. Furthermore, the central node, based on its maintained device registry, counts the number of edge nodes currently in an active response state and capable of accepting task allocation. These three dimensions—the backlog trend of the input queue, the processing timeliness of the computing units, and the scale of available computing resources—together constitute the current load feature set characterizing the system's immediate pressure. For example, in one scenario, the current load characteristics might be: the global queue length increased by 20 frames within 100 milliseconds, the average processing latency increased from 50 milliseconds to 80 milliseconds, and the number of active nodes decreased from 3 to 2 due to one device overheating and exiting the system. This multi-dimensional data acquisition process aggregates the discrete state signals within the system into a quantifiable and analyzable feature vector, providing a data foundation for predictive decision-making.

[0057] Next, nonlinear predictions are made using historical load data. Historical load data is a sequence of multiple load feature sets collected over a past period and their subsequent actual system behavior, implicitly revealing dynamic patterns of system load changes. It should be understood that directly using a linear model for prediction may fail to capture the suddenness and complexity of load changes in an in-vehicle environment. For example, a vehicle entering a complex intersection might cause a surge in video content complexity, or a mobile device might suddenly reduce its frequency due to increased temperature. Therefore, this application employs a nonlinear fitting method, such as using a lightweight recurrent neural network unit or a support vector regression model. This model has been trained using historical data and can learn the nonlinear mapping relationship between load features and the system load level in the next short term (e.g., the next 500-millisecond period). The prediction process in this application inputs the current load feature set into the model, and the model outputs a scalar value, namely the short-term load prediction value. This value represents the level of load pressure the system will experience in the near future under the current operating conditions without any intervention; its value corresponds to the probability of the system experiencing risks such as excessive processing latency or queue overflow. For example, based on the above operations to obtain the current load feature set, the model may predict that within the next 500 milliseconds, the system load will exceed 120% of its stable processing capacity. Through the above operations, a leap from current status monitoring to future risk prediction is achieved, giving the system a forward-looking capability.

[0058] When the calculated short-term load prediction reaches or exceeds a preset latency tolerance threshold, it indicates that the system is about to face or has already faced the risk of failing to meet real-time latency requirements. The latency tolerance threshold is a key parameter set according to the application scenario; for example, in driving safety applications, it might be set as the upper limit of the system load level corresponding to an end-to-end latency of 200 milliseconds. Once this condition is triggered, the system immediately calculates the load deviation difference between the short-term load prediction and the latency tolerance threshold. This difference quantifies the degree to which the predicted load exceeds the safety margin. Subsequently, by querying a preset adjustment amount mapping table, the system determines the target frame rate adjustment amount and compression intensity adjustment amount to be implemented based on this load deviation difference. The adjustment amount mapping table is a predefined empirical or theoretical lookup table that maps different load deviation difference ranges to specific adjustment action parameters. For example, a small positive deviation difference might be mapped to "reducing the image acquisition source frame rate by 5% and reducing the compression quality coefficient by 5%", while a large positive deviation difference might be mapped to "reducing the frame rate by 15% and reducing the compression quality coefficient by 10%". The aforementioned decision-making method based on mapping tables avoids the additional latency caused by complex online optimization calculations, ensuring the immediacy of control response. Specifically, the target frame rate adjustment determines the extent to which the image acquisition device should reduce its image capture and output frequency, directly reducing the number of frames entering the system per unit time. The compression intensity adjustment corresponds to adjusting the compression intensity coefficient of the quantization table during encoding; increasing the compression intensity further reduces the data volume of each frame, thereby alleviating network transmission pressure and the decoding computation burden on each edge node. These two adjustments work synergistically to reduce the system input load from both the "flow" and "volume" dimensions.

[0059] Finally, based on the determined target frame rate adjustment and compression intensity adjustment, specific control commands are generated and sent to each image acquisition source. Upon receiving the commands, each image acquisition source adjusts its internal sensor reading and signal processing clock frequency to achieve the target frame rate, and simultaneously modifies its encoder's quantization parameters to achieve the target compression intensity. This control action directly affects the source of the original video stream, reducing the input rate of the data stream and the data volume of each frame, thereby realigning the system input load with the current and predicted processing capabilities. For example, after predicting an overload risk, the system instructs the forward-facing camera to reduce its frame rate from 30 frames per second to 25 frames per second, while simultaneously adjusting the JPEG compression quality from 80 to 75.

[0060] Through the aforementioned closed-loop control operations, the amount of data entering the queue of frames to be scheduled and the amount of tasks requiring scheduling are immediately reduced. This provides a buffer for the system to cope with sudden high loads or temporary resource depletion, preventing uncontrolled delays or system crashes caused by the infinite growth of the queue. By continuously and periodically operating the complete "monitoring-prediction-decision-control" cycle, the system is dynamically maintained in a stable and efficient operating condition, forming an adaptive safeguard mechanism that ensures the entire real-time image processing method can reliably meet timeliness requirements even when facing dynamically changing driving environments and computing resources.

[0061] Figure 2 shows a functional block diagram of an image real-time processing device provided in an embodiment of this application.

[0062] In some embodiments, the real-time image processing apparatus 2 may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the real-time image processing apparatus 2 may be stored in the memory of a server and executed by at least one processor to perform the functions of the real-time image processing method (see FIG1 for details).

[0063] In this embodiment, the real-time image processing device 2 can be divided into multiple functional modules according to the functions it performs. These functional modules may include: a scheduling and sorting module 21, a node allocation module 22, a parallel recognition module 23, a fusion verification module 24, and a prediction and adjustment module 25. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0064] The scheduling and sorting module 21 is used to perform frame-by-frame compression encoding and metadata priority evaluation and sorting on the original video stream to generate a queue of frames to be scheduled; the node allocation module 22 is used to perform cross-device multi-objective optimization allocation on the queue of frames to be scheduled based on the received real-time status data of the edge nodes to generate a local queue of frames to be processed for each edge node; the parallel recognition module 23 is used to perform parallel decoding and frame-by-frame event recognition on the compressed frames in the local queue of frames to be processed based on a preset scene event feature set to obtain a potential scene event set for each edge node; the fusion verification module 24 is used to perform spatiotemporal evidence fusion and contextual collaborative verification on the potential scene event set to generate an image monitoring report.

[0065] In an optional implementation, the scheduling and sorting module 21 is specifically used for: performing pixel block division and two-dimensional discrete cosine transform on each frame image in the original video stream according to a preset pixel block size to obtain the frequency domain coefficient vector of each pixel block in each frame image; adjusting the preset compression intensity coefficient by linear transformation according to the detected current network bandwidth and the received real-time status data of the edge nodes to generate a current compression quantization table, and performing nonlinear transformation and encapsulation on the frequency domain coefficient vector according to the current compression quantization table to obtain compressed frame data packets from each data source; extracting metadata from each compressed frame data packet according to a preset data type, evaluating the temporal importance coefficient corresponding to each metadata according to the current system time and the acquisition time of the metadata, and determining the priority weight coefficient corresponding to the metadata from a preset priority weight set according to the acquisition source type of the metadata; and performing priority evaluation and data packet sorting on the metadata according to the priority weight coefficient and the temporal importance coefficient to generate a queue of frames to be scheduled.

[0066] In an optional implementation, the node allocation module 22 is specifically used to: parse the received real-time status data to obtain the current processing queue length, historical average frame processing time, and network connection quality index of each edge node; calculate the real-time performance score of each edge node by performing a weighted summation of the current processing queue length, the historical average frame processing time, and the network connection quality index based on the detected current operating temperature of each edge node and a preset scene weight coefficient set using a preset performance evaluation strategy; normalize the real-time performance score to obtain the allocation weight coefficient of each edge node; and perform iterative optimization allocation of the frame queue to be scheduled across devices based on the allocation weight coefficient, the current processing queue length, the historical average frame processing time, and the network connection quality index using a preset improved genetic strategy, thereby generating a local frame queue to be processed for each edge node.

[0067] In an optional implementation, the parallel recognition module 23 is specifically used for: performing entropy decoding and inverse quantization on each compressed frame in the local frame queue to obtain image spatial data corresponding to each compressed frame; performing bilinear interpolation scaling and color channel conversion on the image spatial data to generate a restored image corresponding to each compressed frame; calculating semantic similarity on the restored image according to a preset scene event feature set and the data acquisition source type corresponding to the compressed frame to identify the scene feature data and type confidence value of each frame image in each data acquisition source type; sorting the scene feature data according to the acquisition time order of each frame image in the same data acquisition source type to generate an event record corresponding to the data acquisition source type; and aggregating the event records according to the local frame queue to generate a potential scene event set for each edge node.

[0068] In an optional implementation, the fusion verification module 24 is specifically used for: performing spatiotemporal alignment and grouping processing on the potential scene event set according to a preset sliding time window, scene acquisition distance threshold, acquisition source location data, and acquisition source type to obtain event groups to be fused within the same time window and the same scene; performing incremental probability allocation and synthesis based on the type confidence value of the corresponding image of each event record in the event group to be fused using a preset evidence fusion mechanism to calculate the matching confidence value between the event group to be fused and each scene event type in the scene event feature set; when the matching confidence value reaches a preset primary threshold, performing logical rationality verification of the scene events in the event group to be fused according to a preset scene context rule base to filter out scene recognition event groups, and marking and integrating the scene recognition event groups according to a preset item structure to generate monitoring item data; extracting a status summary set from the real-time status data according to a preset summary type, and encapsulating the monitoring item data and the status summary set into an image monitoring report according to a preset report template.

[0069] In an optional embodiment, the real-time image processing device 2 further includes a prediction and adjustment module 25, which is specifically used to: collect multi-dimensional data on the length change trend of the queue of frames to be scheduled, the processing delay information in the real-time status data of each edge node, and the number of currently active edge nodes through a preset load monitoring strategy to obtain a current load feature set; perform nonlinear fitting prediction of the load trend on the current load feature set based on preset historical load data to obtain a short-term load prediction value; when the short-term load prediction value reaches a preset latency tolerance threshold, calculate the load deviation difference between the short-term load prediction value and the latency tolerance threshold, so as to determine the target frame rate adjustment amount and the compression intensity adjustment amount according to the load deviation difference through a preset adjustment amount mapping table; and control the input rate and data volume of the original video stream according to the target frame rate adjustment amount and the compression intensity adjustment amount.

[0070] In an optional implementation, the node allocation module 22 is further configured to: Step S21, generate an initial allocation group set based on the number of metadata in the queue of frames to be scheduled, the number of edge nodes, and the allocation weight coefficient according to a preset encoding strategy; Step S22, perform multi-objective fitness evaluation of edge nodes on each allocation group in the initial allocation group set based on the current processing queue length, the historical average frame processing time, and the network connection quality index, so as to calculate the fitness value of each allocation group; Step S23, select candidate allocation group sets from the initial allocation group set according to the fitness value through the tournament selection mechanism, and divide the candidate allocation group sets into... The mating groups undergo genetic crossover and mutation processing to generate offspring allocation groups. These offspring allocation groups are then optimized using the hill-climbing local search mechanism, and the optimized offspring allocation groups are updated as the initial allocation group set. Step S24: The number of iterations is accumulated according to a preset counting rule to obtain the current iteration number. Steps S22 to S24 are repeated until the variance of the fitness value is less than a preset convergence threshold, or the current iteration number reaches a preset maximum generation threshold. Based on the fitness value, the allocation group corresponding to the highest fitness value is selected from the initial allocation group set as the optimal allocation group, and the optimal allocation group is converted into a local unprocessed frame queue for each edge node using a preset decoding strategy.

[0071] It should be understood that the various variations and specific embodiments of the methods provided in the above embodiments are also applicable to the real-time image processing apparatus of this embodiment. Through the foregoing detailed description of the real-time image processing method, those skilled in the art can clearly understand the implementation method of the real-time image processing apparatus of this embodiment. For the sake of brevity, it will not be described in detail here.

[0072] Figure 3 shows a schematic diagram of the structure of the electronic device provided in an embodiment of this application.

[0073] In a preferred embodiment of the present invention, the electronic device 3 may include, but is not limited to, a memory 31, at least one processor 32, and at least one communication bus 33.

[0074] Those skilled in the art should understand that the structure of the electronic device 3 shown in FIG3 does not constitute a limitation of the embodiments of the present invention. The electronic device 3 may also include more or fewer other hardware or software than shown in the figure, or different component arrangements.

[0075] In some embodiments, the electronic device 3 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, and embedded devices.

[0076] It should be noted that the electronic device 3 is merely an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0077] In some embodiments, the memory 31 stores a computer program that, when executed by the at least one processor 32, implements all or part of the steps in the real-time image processing method described above. The memory 31 includes read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data. Further, the computer-readable storage medium may primarily include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application program required for a function, etc.

[0078] In some embodiments, the at least one processor 32 is the control unit of the electronic device 3, connecting various components of the electronic device 3 via various interfaces and lines. It executes programs or modules stored in the memory 31 and calls data stored in the memory 31 to perform various functions and process data. For example, when the at least one processor 32 executes a computer program stored in the memory 31, it implements all or part of the steps of the real-time image processing method described in this application embodiment; or it implements all or part of the functions of the real-time image processing device. The at least one processor 32 may be composed of integrated circuits, such as a single-packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0079] In some embodiments, the at least one communication bus 33 is configured to enable communication between the memory 31 and the at least one processor 32, etc. Although not shown, the electronic device 3 may also include a power supply (e.g., a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 32 via a power management device, thereby enabling functions such as charging, discharging, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 3 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0080] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause an electronic device (which may be a personal computer, electronic device, or network device, etc.) or processor to execute portions of the methods described in the various embodiments of this application.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0082] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A real-time image processing method, characterized in that, The method includes: performing frame-by-frame compression encoding and metadata priority evaluation and sorting on the original video stream to generate a queue of frames to be scheduled; performing multi-objective optimization allocation across devices on the queue of frames to be scheduled based on the received real-time status data of the edge nodes to generate a local queue of frames to be processed for each edge node; performing parallel decoding and frame-by-frame event recognition on the compressed frames in the local queue of frames to be processed based on a preset scene event feature set to obtain a potential scene event set for each edge node; and performing spatiotemporal evidence fusion and contextual collaborative verification on the potential scene event set to generate an image monitoring report.

2. The real-time image processing method according to claim 1, characterized in that, The step of performing frame-by-frame compression encoding and metadata priority evaluation and sorting on the original video stream to generate a queue of frames to be scheduled includes: dividing each frame image in the original video stream into pixels and performing a two-dimensional discrete cosine transform according to a preset pixel block size to obtain the frequency domain coefficient vector of each pixel block in each frame image; adjusting a preset compression strength coefficient by linear transformation according to the detected current network bandwidth and the received real-time status data of edge nodes to generate a current compression quantization table, and performing a nonlinear transformation and encapsulation on the frequency domain coefficient vector according to the current compression quantization table to obtain compressed frame data packets from each data source; extracting metadata from each compressed frame data packet according to a preset data type, evaluating the temporal importance coefficient corresponding to each metadata according to the current system time and the acquisition time of the metadata, and determining the priority weight coefficient corresponding to the metadata from a preset priority weight set according to the acquisition source type of the metadata; and performing priority evaluation and data packet sorting on the metadata according to the priority weight coefficient and the temporal importance coefficient to generate a queue of frames to be scheduled.

3. The real-time image processing method according to claim 1, characterized in that, The step of performing cross-device multi-objective optimization allocation of the queue of frames to be scheduled based on the received real-time status data of the edge nodes to generate a local queue of frames to be processed for each edge node includes: parsing the received real-time status data to obtain the current processing queue length, historical average frame processing time, and network connection quality index of each edge node; using a preset performance evaluation strategy, based on the detected current operating temperature of each edge node and a preset scene weight coefficient set, performing weighted summation on the current processing queue length, the historical average frame processing time, and the network connection quality index to calculate the real-time performance score of each edge node; normalizing the real-time performance score to obtain the allocation weight coefficient of each edge node; and using a preset improved genetic strategy, performing cross-device multi-objective constraint iterative optimization allocation of the queue of frames to be scheduled based on the allocation weight coefficient, the current processing queue length, the historical average frame processing time, and the network connection quality index to generate a local queue of frames to be processed for each edge node.

4. The real-time image processing method according to claim 1, characterized in that, The step of performing parallel decoding and frame-by-frame event recognition on compressed frames in the local frame queue to obtain potential scene event sets for each edge node based on a preset scene event feature set includes: performing entropy decoding and inverse quantization on each compressed frame in the local frame queue to obtain image spatial data corresponding to each compressed frame; performing bilinear interpolation scaling and color channel conversion on the image spatial data to generate a restored image corresponding to each compressed frame; calculating semantic similarity on the restored image based on the preset scene event feature set and the data acquisition source type corresponding to the compressed frame to identify scene feature data and type confidence values ​​of each frame image in each data acquisition source type; sorting the scene feature data according to the acquisition time order of each frame image in the same data acquisition source type to generate event records corresponding to the data acquisition source type; and aggregating the event records according to the local frame queue to generate potential scene event sets for each edge node.

5. The real-time image processing method according to claim 4, characterized in that, The process of performing spatiotemporal evidence fusion and contextual collaborative verification on the potential scene event set to generate an image monitoring report includes: performing spatiotemporal alignment and grouping on the potential scene event set according to a preset sliding time window, scene acquisition distance threshold, acquisition source location data, and the acquisition source type to obtain event groups to be fused within the same time window and the same scene; using a preset evidence fusion mechanism, performing incremental probability allocation and synthesis based on the type confidence value of the images corresponding to each event record in the event group to be fused to calculate the matching confidence value between the event group to be fused and each scene event type in the scene event feature set; when the matching confidence value reaches a preset primary threshold, performing logical rationality verification of the scene events in the event group to be fused according to a preset scene context rule base to filter out scene recognition event groups, and marking and integrating the scene recognition event groups according to a preset item structure to generate monitoring item data; extracting a state summary set from the real-time state data according to a preset summary type, and encapsulating the monitoring item data and the state summary set into an image monitoring report according to a preset report template.

6. The real-time image processing method according to claim 1, characterized in that, The method further includes: acquiring multi-dimensional data on the length change trend of the queue of frames to be scheduled, the processing latency information in the real-time status data of each edge node, and the number of currently active edge nodes through a preset load monitoring strategy to obtain a current load feature set; performing nonlinear fitting prediction of the load trend on the current load feature set based on preset historical load data to obtain a short-term load prediction value; when the short-term load prediction value reaches a preset latency tolerance threshold, calculating the load deviation difference between the short-term load prediction value and the latency tolerance threshold, and determining the target frame rate adjustment amount and compression intensity adjustment amount based on the load deviation difference through a preset adjustment amount mapping table; and controlling the input rate and data volume of the original video stream based on the target frame rate adjustment amount and the compression intensity adjustment amount.

7. The real-time image processing method according to claim 3, characterized in that, The improved genetic strategy includes a tournament selection mechanism and a hill-climbing local search mechanism. The step of using a preset improved genetic strategy to iteratively optimize the allocation of the queue of frames to be scheduled across devices based on the allocation weight coefficients, the current processing queue length, the historical average frame processing time, and the network connection quality index, generating local queues of frames to be processed for each edge node, includes: Step S21, generating an initial allocation set based on the amount of metadata in the queue of frames to be scheduled, the preset number of edge nodes, and the allocation weight coefficients using a preset encoding strategy; Step S22, performing multi-objective fitness evaluation of the edge nodes for each allocation group in the initial allocation set based on the current processing queue length, the historical average frame processing time, and the network connection quality index, to calculate the fitness value of each allocation group; Step S2 3. Based on the fitness value, candidate allocation groups are selected from the initial allocation group set through the tournament selection mechanism. Genetic crossover and mutation are performed on the allocation groups in the candidate allocation group set to generate offspring allocation groups. The offspring allocation groups are then optimized through the hill-climbing local search mechanism, and the optimized offspring allocation groups are updated as the initial allocation group set. Step S24: The number of iterations is accumulated according to the preset counting rule to obtain the current iteration number. Steps S22 to S24 are repeated until the variance of the fitness value is less than the preset convergence threshold, or the current iteration number reaches the preset maximum generation threshold. Based on the fitness value, the allocation group corresponding to the highest fitness value is selected from the initial allocation group set as the optimal allocation group. The optimal allocation group is then converted into the local unprocessed frame queue of each edge node through the preset decoding strategy.

8. A real-time image processing apparatus, applied to the real-time image processing method of claim 1, characterized in that, The device includes: a scheduling and sorting module for performing frame-by-frame compression encoding and metadata priority evaluation and sorting on the original video stream to generate a queue of frames to be scheduled; a node allocation module for performing multi-objective optimization allocation across devices on the queue of frames to be scheduled based on the received real-time status data of edge nodes to generate a local queue of frames to be processed for each edge node; a parallel recognition module for performing parallel decoding and frame-by-frame event recognition on the compressed frames in the local queue of frames to be processed based on a preset scene event feature set to obtain a potential scene event set for each edge node; and a fusion verification module for performing spatiotemporal evidence fusion and contextual collaborative verification on the potential scene event set to generate an image monitoring report.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the real-time image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the real-time image processing method according to any one of claims 1 to 7.

Images