Method for improving reasoning efficiency of multi-machine algorithm based on cloud edge collaboration

Abstract

The invention relates to a method for improving the reasoning efficiency of a multi-machine algorithm based on cloud edge collaboration, and the method comprises the following steps: a cloud end issues a reasoning task instruction to an edge end, and the reasoning task instruction comprises at least one reasoning algorithm and at least one video stream address; the edge end responds to the reasoning task instruction, carries out stream pulling scheduling on the video stream address by adopting a time wheel mechanism, and decodes the video stream to obtain image data; the edge end writes the image data into a shared memory area through a shared memory mechanism so as to realize cross-process sharing; the edge end reads the image data from the shared memory area and executes a reasoning algorithm specified in the reasoning task instruction to generate a reasoning result; and the edge end reports the reasoning result to the cloud end. The method has the advantage of efficiently coordinating the strong computing power of the cloud end and the low delay characteristic of the edge end.

Description

Technical Field

[0001] The present application relates to the field of cloud-edge collaboration, and in particular to a method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration. Background Art

[0002] With the widespread adoption of the Internet of Things (IoT) and smart devices, edge computing has become a key data processing technology. By performing computations close to the data source, it effectively reduces latency and improves real-time performance. It is particularly widely used in the field of visual algorithms. In scenarios such as intelligent monitoring, logistics sorting, and autonomous driving, a large number of IoT devices, such as cameras, generate an explosive growth of image and video stream data.

[0003] In existing technologies, a common processing approach is to collect all image data and transmit it directly to the cloud for centralized computing and inference. While the cloud possesses powerful computing resources capable of running complex deep learning models, this approach faces significant challenges. First, uploading massive amounts of image data consumes significant network bandwidth. Second, long-distance data transmission and cloud-based queuing lead to high processing latency. For example, traditional cloud-based inference solutions can experience latency of up to 5 seconds, which is unacceptable for applications requiring real-time feedback.

[0004] To overcome the latency issues of cloud-based solutions, another approach is to perform computing tasks as much as possible at the edge. However, edge devices often have limited hardware resources, such as the number of central processing unit (CPU) cores, graphics processing unit (GPU) cores, and memory capacity. These limited computing resources are unable to independently support increasingly complex visual algorithm models, especially when processing multiple video streams simultaneously or running multiple algorithms in parallel. This leads to low computing efficiency and fails to meet business needs. Summary of the Invention

[0005] In order to efficiently coordinate the powerful computing power of the cloud and the low latency characteristics of the edge, and to solve the problems of high latency, large bandwidth consumption and limited edge computing resources in the existing technology, this application provides a method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration.

[0006] This application provides a method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration, which adopts the following technical solutions: A method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration includes the following steps: S1. The cloud sends an inference task instruction to the edge, wherein the inference task instruction includes at least one inference algorithm and at least one video stream address; S2. The edge responds to the inference task instruction, uses a time wheel mechanism to schedule the video stream address, and decodes the video stream to obtain image data; S3. The edge end writes the image data into the shared memory area through the shared memory mechanism to achieve cross-process sharing; S4. The edge terminal reads the image data from the shared memory area and executes the inference algorithm specified in the inference task instruction to generate an inference result; S5. The edge reports the inference result to the cloud.

[0007] By employing the above technical solutions, this solution clearly divides the functions of the cloud and edge. In S1, the cloud is responsible for top-level, non-real-time task scheduling and management, acting as the brain of the system. In S4, the edge is responsible for executing local, compute-intensive, real-time reasoning tasks, acting as the system's sensory organs and nerve endings. Finally, the edge only reports lightweight, valuable reasoning results to the cloud, forming a complete data processing and decision-making closed loop. This division of labor enables the entire system to combine the management flexibility of the cloud with the real-time processing capabilities of the edge.

[0008] To address the challenge of processing multiple video streams simultaneously at the edge, this solution introduces a time wheel mechanism. This mechanism distributes a large number of scheduled stream pulling tasks onto different ticks on the time wheel. The scheduler simply executes the tasks on each tick sequentially as time progresses. This results in extremely low scheduling overhead, independent of the total number of tasks. This fundamentally avoids system performance bottlenecks caused by an excessive number of concurrent tasks, ensuring smooth, orderly, and efficient data access.

[0009] To maximize processing efficiency within the edge, this solution utilizes a shared memory mechanism for cross-process communication. This mechanism allows the decoding and inference processes to directly access the same physical memory, eliminating the data copy overhead required for traditional inter-process communication (known as "zero copy"). This creates a "data highway" between the two key processes, ensuring that decoded image data can be instantly transferred to the GPU for efficient inference with minimal memory and CPU consumption.

[0010] Optionally, the S1 includes the following steps: S11. The cloud generates the inference task instruction including the algorithm identifier corresponding to the inference algorithm and the video stream address according to the preset business logic or the received user request; S12. The cloud sends the inference task instruction to the designated edge device through a preset communication link.

[0011] By employing this technical solution, commands can be automatically triggered by pre-set business logic, allowing the system to operate autonomously and continuously according to predetermined plans. This reduces reliance on manual maintenance and improves automation. For example, security algorithms can automatically activate at night and production management algorithms can automatically switch to them during the day. Furthermore, a retained user request portal allows administrators to dynamically adjust the monitoring tasks of any camera based on actual needs, meeting diverse and temporary business scenarios.

[0012] By sending instructions to designated edge devices and associating specific algorithm identifiers with tasks, the cloud can independently and accurately assign tasks to each edge node. This is crucial in large-scale deployment scenarios, as it ensures that edge nodes in different locations and with different functions can work together without interfering with each other.

[0013] Optionally, the S2 includes the following steps: S21. The edge parses the inference task instruction, extracts the target video stream address, and registers the stream pulling task to the specified scale of the time wheel data structure according to the preset screenshot frequency; S22. The edge periodically advances the pointer, and when the time wheel reaches the specified scale, the stream pulling task is triggered, and the streaming media processing module is connected to the corresponding video stream address and pulls a frame of compressed video data; S23. The edge decodes the pulled compressed video data to generate raw image data in a specific format that can be used for algorithm reasoning.

[0014] By adopting the above technical solution, by pre-registering the streaming tasks with the time wheel based on frequency and triggering them periodically, this solution can effectively avoid instantaneous CPU, I / O, or network load spikes caused by concurrently pulling multiple video streams. The scheduling principle of the time wheel advancing the pointer makes the CPU overhead of task scheduling extremely low and constant. Compared with traditional timer solutions, this frees up valuable CPU resources for core decoding and inference tasks, greatly improving the scalability of a single edge device, and can support a far greater number of concurrent video streams than traditional solutions at a very low performance cost.

[0015] Optionally, the edge end creates a named shared memory area by the first process executing the decoding, wherein the storage space size of the named shared memory area is set to be sufficient to accommodate one or more frames of the image data; S32. The edge end maps the named shared memory area to the virtual address space of the first process, thereby obtaining a memory pointer that can be directly read and written; S33 edge end of the image data, through a memory copy operation is written to the named shared memory area pointed to by the memory pointer; S34. After the image data is written, the edge end publishes a data ready notification through the publish / subscribe function.

[0016] By adopting the above technical solution, this approach builds a "zero-copy" data path by creating a shared memory area and directly memory-mapping it. Compared to traditional IPC methods, this eliminates multiple time-consuming memory copy operations, allowing decoded image data to be transferred from the decoding process to the inference process with extremely low latency, close to the speed of physical memory.

[0017] The communication method based on named shared memory and a publish / subscribe mechanism eliminates the need for data input and data receiving ends to directly perceive each other's existence. They interact solely through agreed-upon memory names and message topics. This highly decoupled architecture means that any module (process) can be independently restarted, upgraded, or even replaced without affecting the normal operation of other modules.

[0018] The asynchronous publish / subscribe notification mechanism allows data consuming processes to remain dormant when no data is available, completely eliminating CPU resource consumption. Compared to polling methods that require the receiving process to continuously idle and query, this significantly reduces unnecessary CPU overhead, freeing up CPU resources for algorithmic reasoning, which truly requires computational power.

[0019] Optionally, the S4 includes the following steps: S41. The edge end, in response to the data ready notification, maps the named shared memory area to the virtual address space of the second process executing the inference algorithm to obtain a read-only access pointer to the named shared memory area; S42. The edge end reads the image data from the named shared memory area to the local memory of the second process through the read-only access pointer; S43. The edge end performs reasoning on the read image data using the corresponding reasoning engine according to the reasoning task instruction to obtain the original reasoning data; S44. The edge processes the raw inference data to generate structured inference results.

[0020] By adopting the above technical solutions, an event-driven mechanism, directly awakened by data-ready notifications, ensures that inference tasks can be started with zero delay after data arrives. The use of a highly optimized dedicated inference engine ensures that the most critical computational steps are completed in the shortest possible time.

[0021] Furthermore, by performing the critical step of reading image data from shared memory into the process's local memory, this solution effectively mitigates the risk of data tearing. This ensures that the inference process (the second process) operates on a stable and complete data snapshot when performing analysis, preventing inconsistent or corrupted data from being read by concurrent writes from the data access end (the first process).

[0022] Step S44 converts the original tensor output by the inference engine into standardized, clearly structured, machine-readable business event information. This structured inference result can be directly used for reporting, storage, alarming, and data analysis, allowing the computing output at the edge to be seamlessly integrated into the entire business process.

[0023] Optionally, the S5 includes the following steps: S51. The edge encapsulates the structured reasoning results into a data transmission object containing metadata; S52. Send the data transmission object to the cloud via the communication link.

[0024] Optionally, the S43 includes the following sub-steps: S431. Preprocessing the read image data so that its format and size meet the input specifications of the inference engine; S432. The pre-processed image data is input into the inference engine to perform forward calculations; S433. Output the original inference data including the location and category confidence of the detected target.

[0025] By adopting the above technical solution and strictly standardizing each frame of input image, the risk of inference errors or significant accuracy drops caused by inconsistent input data formats and sizes is eliminated. Furthermore, the establishment of a standard pipeline of preprocessing → forward calculation → output of raw data is equivalent to defining a clear plug-and-play interface for algorithm models. Developers can train new, higher-performing algorithm models. As long as the new model also follows this set of interface specifications, that is, clearly defines its input preprocessing requirements and raw output format, it can seamlessly replace the old model without having to modify the data access and post-processing code before and after. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The present invention illustrates a system architecture for implementing a method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration in one embodiment of the present invention.

[0027] Figure 2 A flowchart of a method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration in one embodiment of the present invention is depicted.

[0028] Figure 3 A flowchart of a method for improving multi-machine algorithm reasoning efficiency based on cloud-edge collaboration in one embodiment of the present invention is depicted. DETAILED DESCRIPTION

[0029] The present application will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0030] In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding of the inventive concepts. Some of the figures in the drawings of the present disclosure, which are part of this specification, represent structures and devices in block diagram form to avoid making the disclosed principles complicated and obscure. For the sake of clarity, not all features of an actual implementation are necessarily described. In addition, the language used in this disclosure has been selected primarily for readability and instructional purposes and may not have been selected to delineate or limit the subject matter of the invention, thereby resorting to the necessary claims to determine such inventive subject matter. References in this disclosure to "one embodiment" or "an embodiment" mean that the specific features, structures or characteristics described in conjunction with that embodiment are included in at least one embodiment, and multiple references to "one embodiment" or "an embodiment" should not be understood to necessarily all refer to the same embodiment.

[0031] Unless expressly limited, the terms "a", "an" and "the" are not intended to refer to a singular entity, but rather to include a general class of which a specific example may be used for illustration. Thus, the use of the term "a" or "an" may mean any number of at least one, including "one", "one or more", "at least one", and "one or more than one". The term "or" means any of the alternatives and any combination of the alternatives, including all, unless the alternatives are expressly indicated to be mutually exclusive. The phrase "at least one of" when combined with a list of items refers to a single item in the list or any combination of the items in the list. The phrase does not require all of the listed items unless expressly limited to that.

[0032] The embodiment of the present application discloses a method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration, referring to Figure 1 ,The system architecture on which it is based is physically divided into two parts: the edge and the cloud.,Each part consists of different functional modules that work together.

[0033] The edge is the core of real-time data processing and algorithm reasoning, and is mainly composed of the following three modules: visual streaming module, visual shared memory module and visual algorithm reasoning module.

[0034] The visual streaming module is the data access terminal at the edge. It is responsible for acquiring raw image or video data from the outside world and completing preliminary processing to prepare for subsequent algorithm analysis. The core functions of this module include: Data access: Responsible for connecting and pulling RTSP video streams, or reading image data from object storage such as MinIO.

[0035] Task scheduling: Built-in time wheel mechanism is used to efficiently and smoothly schedule the screenshot tasks of multiple cameras to avoid system overload.

[0036] Decoding and conversion: Use tools such as FFmpeg to decode the compressed video stream into raw image data (such as OpenCV Mat format).

[0037] Data publishing: As a producer, it writes the processed image data into shared memory and publishes a data-ready notification.

[0038] The visual shared memory module is a data bus that implements high-speed communication between processes within the edge. It is not an independent business process, but a set of underlying communication components jointly called by the streaming media module and the algorithm inference module.

[0039] The core functions of this module include: Zero-copy communication: Based on Linux's mmap and shm_open mechanisms, it enables different processes to directly access the same physical memory, avoiding the time-consuming kernel-mode data copying in traditional IPC methods.

[0040] Asynchronous notification: Provides publish / subscribe functionality, allowing the data access end to efficiently and asynchronously wake up the data receiving end through event notification when the data is ready.

[0041] Data transmission: supports flexible data transmission modes such as one-to-one and one-to-many.

[0042] The visual algorithm reasoning module is the computing core that performs intelligent analysis and converts raw data into effective information. The core functions of this module include: Data processing: Respond to notifications from the shared memory module and read image data from the shared memory at high speed.

[0043] Algorithm reasoning: Load the specified AI model and use optimized inference engines such as OpenVINO and TensorRT to perform core inference calculations on the CPU or GPU.

[0044] Multi-algorithm support: Supports multiple algorithm scenarios in the logistics industry, such as climbing, bag blocking, and damage identification.

[0045] Result processing and reporting: Post-process the raw data output by the inference engine to generate structured and readable business results, and be responsible for reporting the results to the cloud.

[0046] The cloud serves as the system's management and analysis center, primarily responsible for global scheduling and data aggregation and analysis. The cloud processing module is a general term for all cloud functions, acting as the command center and data center for the entire distributed system. The core functions of the cloud processing module include: Task scheduling and distribution: Generate inference task instructions based on business needs and distribute them to designated edge nodes to start the entire workflow.

[0047] Result reception and analysis: Receive structured inference results and related image data reported by the edge, and perform storage, statistics, and in-depth business data analysis.

[0048] Device management and interaction control: Responsible for managing the status of all edge devices and handling the interaction logic between the cloud and the edge.

[0049] Based on the above system architecture, refer to Figure 2 The method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration includes the following steps S1-S5.

[0050] S1. The cloud sends an inference task instruction to the edge end, wherein the inference task instruction includes at least one inference algorithm and at least one video stream address.

[0051] Optionally, in a certain embodiment, S1 includes the following steps S11-S12.

[0052] S11. The cloud generates the inference task instruction including the algorithm identifier corresponding to the inference algorithm and the video stream address according to the preset business logic or the received user request.

[0053] S12. The cloud sends the inference task instruction to the designated edge device through a preset communication link.

[0054] Specifically, in step S11, the generation of the inference task instructions can be driven by two modes. The first is pre-set business logic, which refers to pre-configured rules or scheduling plans in the cloud processing module that can be automatically triggered. For example, in a logistics center application scenario, a business logic can be set to automatically activate the congestion identification algorithm for all cameras on the sorting lines during the peak period from 9:00 am to 5:00 pm daily.

[0055] The second type is received user requests, which refer to manual, immediate operations performed by operations or management personnel through the cloud management interface (such as a webpage or client). For example, if security personnel detect an anomaly in a certain area through monitoring, they can manually select the cameras in that area and immediately issue an inference task for the climbing recognition algorithm. Regardless of the triggering method, the cloud generates an inference task instruction. This instruction is a structured data packet, the core content of which includes the algorithm identifier corresponding to the inference algorithm and the video stream address.

[0056] An inference algorithm, as used here, refers to a trained AI model that analyzes images or videos to perform specific recognition tasks. For example, the aforementioned "damage detection algorithm" is an inference algorithm. Instructions typically don't directly include a large model file. Instead, they use a short, unique algorithm identifier, such as the string "damage_detection_v1.2" to represent "Damage Recognition Algorithm Version 1.2." This identifier allows edge devices to load the corresponding model file locally.

[0057] At the same time, the video stream address included in the instruction clarifies the source of the data, which is usually an RTSP (Real Time Streaming Protocol) address, such as rtsp: / / 192.168.1.100 / stream1, pointing to a specific network camera. Then in step S12, this generated instruction will be sent through a preset communication link. The communication link here refers to a pre-established, stable and reliable network connection channel between the cloud and the specified edge device. For example, it can be a long HTTPS connection based on the public network and encrypted with TLS, or a more stable internal connection established through a dedicated network (such as SD-WAN).

[0058] S2. The edge end responds to the inference task instruction, uses a time wheel mechanism to schedule the video stream address, and decodes the video stream to obtain image data.

[0059] Optionally, in a certain embodiment, the S2 includes the following steps S21-S23.

[0060] S21. The edge end parses the inference task instruction, extracts the target video stream address, and registers the stream pulling task to the specified scale of the time wheel data structure according to the preset screenshot frequency.

[0061] S22. The edge end periodically advances the pointer, and when the time wheel reaches the specified scale, triggers the stream pulling task, connects to the corresponding video stream address through the streaming media processing module, and pulls a frame of compressed video data.

[0062] S23. The edge decodes the pulled compressed video data to generate raw image data in a specific format that can be used for algorithm reasoning.

[0063] After receiving the instruction, the edge end starts to execute step S2, that is, the visual streaming module performs data access and processing.

[0064] Specifically, in step S21, the edge schedules the stream pulling task. A stream pulling task here refers to a specific operation unit whose task is to connect to and capture a frame of image data from a specified video stream address. Stream pulling scheduling, on the other hand, refers to the efficient and orderly management and execution planning of multiple such tasks. This embodiment uses a time wheel mechanism for scheduling. The time wheel data structure it relies on can be understood as a circular queue similar to a clock face. The face is divided into a fixed number of scales, with each designated scale representing a future time slice. The principle of S21 is that it does not passively wait. Instead, it pre-calculates when the next task should be executed based on a preset capture frequency (for example, 10 fps, i.e., once every 100 milliseconds). It then registers the stream pulling task as an event on the time wheel at the designated scale corresponding to the next 100 milliseconds. In this way, stream pulling tasks from multiple cameras are evenly distributed across different scales of the time wheel, thereby balancing peak load and valley load on the system and avoiding the impact of instantaneous concurrent requests on the system.

[0065] Then, in step S22, the time wheel scheduler begins. The time wheel scheduler is a background service program that continuously and periodically advances a pointer within it, moving it from one tick to the next, at a very short, fixed interval (e.g., every 1 millisecond). The principle behind S22 is its extremely low scheduling overhead. The scheduler does not need to iterate through all pending tasks; it simply checks whether a stream pull task is registered at the tick pointed to by the current pointer. When the pointer reaches the tick for which the task was registered in S21, the scheduler triggers the execution of the stream pull task, ensuring that the task is woken up and executed precisely and with low latency at the scheduled time. Once the task is triggered, the streaming media processing module immediately connects to the corresponding video stream address and pulls a frame of compressed video data (e.g., an H.264 frame). This frame of data serves as the source for all subsequent analysis. After completion, the task may be recalculated for the next execution tick and re-registered with the time wheel, enabling continuous, periodic capture.

[0066] S3. The edge end writes the image data into the shared memory area through the shared memory mechanism to achieve cross-process sharing.

[0067] Optionally, in a certain embodiment, the S3 includes the following steps S31-S34.

[0068] S31. The edge end creates a named shared memory area by the first process that executes the decoding, wherein the storage space size of the named shared memory area is set to be sufficient to accommodate one or more frames of the image data.

[0069] S32. The edge end maps the named shared memory area to the virtual address space of the first process, thereby obtaining a memory pointer that can be directly read and written.

[0070] S33. The edge end writes the image data into the named shared memory area pointed to by the memory pointer through a memory copy operation.

[0071] S34. After the image data is written, the edge end publishes a data ready notification through the publish / subscribe function.

[0072] In step S31, the first process creates a named shared memory area. This step involves requesting a block of physical memory from the operating system for shared access by multiple processes and assigning it a globally unique name. This establishes a "common location" for communication between processes without requiring direct dependencies. For example, the first process might create a shared memory area named / cam01_buffer and set its size to 8MB, ensuring it's large enough to accommodate a high-resolution raw image.

[0073] In step S32, the system maps this named shared memory region into the virtual address space of the first process. This step utilizes the memory mapping (mmap) mechanism to create a pointer directly to the shared physical memory in the current process's address space, thus avoiding the time-consuming data copying required by traditional communication methods. This advantage is that it establishes a zero-copy data channel, the foundation for achieving millisecond-level communication.

[0074] After the channel is established, step S33 executes the data writing operation, that is, through an efficient memory copy operation, the decoded image data is directly written from the private memory of the first process to the shared memory area pointed to by the aforementioned pointer.

[0075] In step S34, the system uses publish / subscribe functionality to issue data readiness notifications to notify downstream processes. After writing data, the first process, acting as the publisher, sends a message to a predefined topic, regardless of who is receiving the message. The advantage of this asynchronous notification mechanism is that it avoids inefficient polling and waiting on the data receiving process, thereby conserving CPU resources. It also allows the data sending and receiving processes to be completely decoupled and run independently, improving the robustness and responsiveness of the entire system.

[0076] S4. The edge end reads the image data from the shared memory area and executes the inference algorithm specified in the inference task instruction to generate an inference result.

[0077] Optionally, in a certain embodiment, the S4 includes the following steps S41-S44.

[0078] S41. The edge end maps the named shared memory area to the virtual address space of the second process executing the inference algorithm in response to the data ready notification to obtain a read-only access pointer to the named shared memory area.

[0079] S42. The edge end reads the image data from the named shared memory area to the local memory of the second process through the read-only access pointer.

[0080] S43. The edge end performs reasoning on the read image data using the corresponding reasoning engine according to the reasoning task instruction to obtain the original reasoning data; S44. The edge processes the raw inference data to generate structured inference results.

[0081] Step S4 is completed by the second process responsible for calculation (i.e., the visual algorithm reasoning module), which processes the data and produces analysis results.

[0082] In step S41, the second process responds to the data-ready notification and maps the named shared memory area. This step is based on the principle of event-driven passive wakeup and data channel connection. For example, if a dormant second process is awakened by a message on the / events / cam01 topic, it will immediately open and map the same shared memory area using the same name, / cam01_buffer, to obtain access to the data. This has the advantage that the computing process is only activated when there is work to do, effectively conserving system resources.

[0083] In step S42, the second process reads the image data from the named shared memory area into its local memory. This step ensures data consistency during the inference process by creating a local snapshot of the data. For example, the second process will completely copy the image data in the shared memory into its own private memory buffer. Even if the first process responsible for decoding writes the next updated image frame to the shared memory at this time, this will not affect the second process's ongoing, potentially time-consuming inference calculations. This avoids computational errors caused by mid-process data modifications and ensures the reliability of the algorithm.

[0084] In step S43, the core algorithm inference begins, using a specially optimized inference engine for calculations. For example, based on the algorithm identifier in the task instruction, the second process loads the damage_detection_v1.2 model and inputs the image data read in S42 into the TensorRT inference engine. The engine then leverages the parallel computing power of the GPU to complete the model's forward calculations, ultimately producing raw inference data containing the locations, sizes, and category scores of all potential objects in the image. This approach maximizes hardware performance and enables real-time inference of complex algorithms.

[0085] In step S44, the system needs to process this raw inference data. This process relies on information interpretation and abstraction, converting machine-readable raw tensor data into information readable by humans or higher-level business systems. For example, post-processing logic will filter out damaged objects with a confidence level higher than 0.9 and integrate their coordinates, confidence level, and other information into a structured inference result in JSON format. This approach has the advantage of standardizing and operationalizing the output of underlying computations, transforming them into valuable information that can be directly used for reporting and analysis.

[0086] Optionally, in a certain embodiment, the S43 includes the following sub-steps S431-S433.

[0087] S431. Preprocess the read image data so that its format and size meet the input specifications of the inference engine.

[0088] S432. Input the pre-processed image data into the inference engine to perform forward calculation.

[0089] S433. Output the original inference data including the location and category confidence of the detected target.

[0090] In order to illustrate the execution process of step S43 in more detail, the embodiment of the present application also discloses an optional embodiment S431-S433.

[0091] In step S431, the edge end preprocesses the read image data to standardize and normalize the input data, because the deep learning model has strict requirements on the format, size and value range of its input data. For example, a damage recognition model may be trained to process 640x640 pixel images, and requires pixel values ​​to be in the floating point range of [0, 1]. The original image obtained from the camera may be 1920x1080 pixels, and the pixel values ​​are in the integer range of [0, 255]. Therefore, the preprocessing step will be responsible for scaling and padding the 1920x1080 image to 640x640, and normalizing all pixel values.

[0092] After preprocessing, the standardized image data is fed into the inference engine in step S432 for forward computation. This step is based on the forward propagation of a neural network. The preprocessed 640x640 image data (now a standardized multidimensional array or tensor) is fed into the inference engine, which drives this data through dozens or even hundreds of computational layers defined within the model, performing massively parallel matrix multiplications and activation function calculations.

[0093] In step S433, the inference engine outputs the raw inference data containing the location and category confidence of the detected target, and performs structured encoding of the calculation results. The output of the engine is not a direct conclusion, but a set of raw tensors with a specific data structure. For example, it may output a tensor with a shape of [1, 25200, 6], where "25200" represents that the model predicts 25,200 possible target boxes on the image, and "6" represents the 4 position coordinates, 1 target confidence, and 1 category score of each box.

[0094] S5. The edge reports the inference result to the cloud.

[0095] Optionally, in a certain embodiment, the S5 includes the following steps S51-S52.

[0096] S51. The edge end encapsulates the structured reasoning result into a data transmission object containing metadata.

[0097] S52. Send the data transmission object to the cloud via the communication link.

[0098] In step S51, the edge encapsulates the structured inference results generated in S4 into a data transmission object containing metadata. A simple inference result (e.g., a damage finding) is of limited significance without the context in which it was generated. Therefore, it needs to be bound to metadata. Metadata here refers to data that describes data attributes, such as the timestamp of the event and the device identifier that generated the data (e.g., edge-node-03). The encapsulation process integrates this information into a uniformly formatted, self-explanatory data package, such as a rich JSON object. This ensures that every piece of data reported to the cloud is complete, facilitating subsequent processing by the cloud.

[0099] After the encapsulation is completed, step S52 executes the sending of the data, that is, the data transmission object is sent to the cloud through the communication link. For example, the edge end uses the JSON object generated in S51 as the request body, and initiates a POST request to the business analysis service interface (such as a URL address) on the cloud through the public network or SD-WAN communication link using a secure HTTP / S protocol. Compared with transmitting the original video stream, transmitting a lightweight JSON object can reduce bandwidth consumption by several orders of magnitude. At the end of the entire interaction, the cloud receives and verifies the reported data, and if successful, stores it in the database and triggers the corresponding data analysis process, while returning a receipt of successful processing to the edge end. This response from the cloud ensures the reliability of communication. After receiving the successful receipt, the edge end can consider that the reporting task is completed.

[0100] The following will be combined Figure 3 , the method of the present invention is illustrated through a complete scenario of real-time monitoring of package damage in a logistics transit center.

[0101] In this example, the cloud (media-cloud) operation and maintenance personnel want to perform real-time package damage detection on conveyor belt 3 (camera address is rtsp: / / 192.168.3.103 / stream). The operation and maintenance personnel send an inference task instruction to the edge computing node deployed in the center through the cloud management platform. This is step S1, which occurs Figure 1 Before the process shown begins.

[0102] After the media-edge process of the edge node receives the task, it starts executing Figure 3 The loop shown in the figure begins with the timer module in the process triggering a task execution at a predetermined time (for example, 11:45:30.100) based on the preset 10fps screenshot frequency. After the task starts, it undergoes task verification to confirm that it is valid and enters the task execution phase. The media-edge process first obtains the shm client and requests shared memory, creating a shared memory area named / cam03_buffer.

[0103] Next, the screenshot operation module begins. The media-edge process performs a stream capture, pulling a frame of H.264 compressed video data from the rtsp: / / 192.168.3.103 / stream address. Next, during the transcoding step, the GPU hardware decoder decodes this frame into a 1920x1080 pixel raw image and copies its contents to the / cam03_buffer shared memory area. This completes steps S2 and S3.

[0104] After the data is written, the media-edge process immediately executes the shared memory send, publishes a "data ready" notification, and enters the waiting for response state.

[0105] At this point, the independent media-process process on the edge node is awakened and responds to the notification via the shared memory receiving mechanism. It first constructs a cvMat object using a shallow copy, efficiently loading the image data from shared memory. The data is then fed into the algorithm inference module. In this example, the inference algorithm identifies a package in the image with obvious signs of tearing. After internal processing, the inference result confirms that the decision is correct, resulting in a "yes" result.

[0106] Because the judgment result is "yes," the media-process process triggers the reporting process. It first sends a shared memory response signal back to the media-edge process, informing it that the data has been used up. Then, it performs asynchronous reporting: on the one hand, it reports the result to media-cloud as structured JSON data containing information such as "damage detected, confidence level 0.97"; on the other hand, it sends the original image containing the damaged package to the media-image process in the form of a raw image stream. After receiving the data, the media-image process independently performs image upload and also sends the live image to media-cloud. At this point, steps S4 and S5 are completed.

[0107] Finally, after receiving the response from media-process, the media-edge process processes the response result, releases the shared memory and releases the client in succession, completing this cycle and waiting for the next trigger of the time wheel.

[0108] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0109] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration, characterized in that: The following steps are involved: S1. The cloud sends an inference task instruction to the edge, wherein the inference task instruction includes at least one inference algorithm and at least one video stream address; S2. The edge responds to the inference task instruction, uses a time wheel mechanism to schedule the video stream address, and decodes the video stream to obtain image data; S3. The edge end writes the image data into the shared memory area through the shared memory mechanism to achieve cross-process sharing; S4. The edge terminal reads the image data from the shared memory area and executes the inference algorithm specified in the inference task instruction to generate an inference result; S5. The edge reports the inference result to the cloud.

2. The method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration according to claim 1 is characterized in that: Said S1 comprises the following steps: S11. The cloud generates the inference task instruction including the algorithm identifier corresponding to the inference algorithm and the video stream address according to the preset business logic or the received user request; S12. The cloud sends the inference task instruction to the designated edge device through a preset communication link.

3. The method for improving multi-machine algorithm reasoning efficiency based on cloud-edge collaboration according to claim 2 is characterized in that: The S2 comprises the following steps: S21. The edge parses the inference task instruction, extracts the target video stream address, and registers the stream pulling task to the specified scale of the time wheel data structure according to the preset screenshot frequency; S22. The edge periodically advances the pointer, and when the time wheel reaches the specified scale, the stream pulling task is triggered, and the streaming media processing module is connected to the corresponding video stream address and pulls a frame of compressed video data; S23. The edge decodes the pulled compressed video data to generate raw image data in a specific format that can be used for algorithm reasoning.

4. The method for improving multi-machine algorithm reasoning efficiency based on cloud-edge collaboration according to claim 3 is characterized in that: The S3 includes the following steps: S31. The edge end creates a named shared memory area by executing the first process of decoding, wherein the storage space size of the named shared memory area is set to be sufficient to accommodate one or more frames of the image data; S32. The edge end maps the named shared memory area to the virtual address space of the first process, thereby obtaining a memory pointer that can be directly read and written; S33 edge end of the image data, through a memory copy operation is written to the named shared memory area pointed to by the memory pointer; S34. After the image data is written, the edge end publishes a data ready notification through the publish / subscribe function.

5. The method for improving the efficiency of multi-machine algorithm reasoning based on cloud-edge collaboration according to claim 4 is characterized in that: The S4 comprises the following steps: S41. The edge end, in response to the data ready notification, maps the named shared memory area to the virtual address space of the second process executing the inference algorithm to obtain a read-only access pointer to the named shared memory area; S42. The edge end reads the image data from the named shared memory area to the local memory of the second process through the read-only access pointer; S43. The edge end performs reasoning on the read image data using the corresponding reasoning engine according to the reasoning task instruction to obtain the original reasoning data; S44. The edge processes the raw inference data to generate structured inference results.

6. The method for improving multi-machine algorithm reasoning efficiency based on cloud-edge collaboration according to claim 5 is characterized in that: The S5 comprises the following steps: S51. The edge encapsulates the structured reasoning results into a data transmission object containing metadata; S52. Send the data transmission object to the cloud via the communication link.

7. The method for improving multi-machine algorithm reasoning efficiency based on cloud-edge collaboration according to claim 5 is characterized in that: The S43 includes the following sub-steps: S431. Preprocessing the read image data so that its format and size meet the input specifications of the inference engine; S432. The pre-processed image data is input into the inference engine to perform forward calculations; S433. Output the original inference data including the location and category confidence of the detected target.

Images