Multi-process video data acquisition method and related device

Abstract

The invention provides a multi-process video data acquisition method and a related device, and the method comprises the steps: controlling m video nodes to carry out real-time acquisition according to a preset frequency, and obtaining m pieces of first video data; acquiring a preset multi-process acquisition task; the multi-process collection task comprises a first process collection tasks; determining a clients corresponding to the a first process collection tasks; receiving a processing instructions sent by a clients for the reference video node; acquiring first video data corresponding to a reference video node in the m pieces of first video data to obtain reference first video data; and performing data processing and data reading on the reference first video data according to the a processing instructions and the a clients to complete a multi-process acquisition task. Data processing and multi-process sharing reading are carried out on the video data, so that the efficiency and the flexibility of multi-process video data acquisition are improved.

Description

Technical Field

[0001] This application relates to the field of video data acquisition technology, and in particular to a multi-process video data acquisition method and related apparatus. Background Technology

[0002] With the widespread adoption of smart tablets in the automotive field, multi-channel video processing has become an essential function for core operations such as AI visual monitoring, video streaming, and real-time display. Current automotive video data acquisition and processing largely rely on the V4L2 video driver framework, but its video nodes employ an exclusive access mechanism, supporting only one client process at a time, which cannot meet the needs of multiple processes using video streams in parallel. Existing solutions often use data copy-forward mechanisms, which suffer from high transmission latency, high system resource consumption, and poor compatibility with multiple systems. Furthermore, the graphics processing operations of video data are executed independently by each process, resulting in resource waste and insufficient processing flexibility.

[0003] Therefore, how to improve the efficiency and flexibility of multi-process video data acquisition is an urgent problem to be solved. Summary of the Invention

[0004] This application provides a multi-process video data acquisition method and related apparatus. By exclusively managing multiple video nodes through a server and uniformly receiving processing instructions from multiple clients for the same video node, the method processes and shares the video data of the video node across multiple processes, thereby improving the efficiency and flexibility of multi-process video data acquisition.

[0005] In a first aspect, embodiments of this application provide a multi-process video data acquisition method applied to a server; the multi-process video data acquisition system includes the server, m video nodes exclusively managed by the server, and several clients; m is an integer greater than 1; the method includes: The m video nodes are controlled to collect data in real time at a preset frequency to obtain m first video data. Obtain a preset multi-process acquisition task; the multi-process acquisition task includes a first-process acquisition task; a is a positive integer; Determine the a clients corresponding to the a first-process acquisition tasks in the multi-process video data acquisition system; Receive a processing instructions sent by the a clients to a reference video node; the reference video node is any one of the m video nodes; Obtain the first video data corresponding to the reference video node from the m first video data to obtain the reference first video data; Based on the a processing instructions and the a clients, the reference first video data is processed and read to complete the multi-process acquisition task.

[0006] Secondly, embodiments of this application provide a multi-process video data acquisition device applied to a server; the multi-process video data acquisition system includes the server, m video nodes exclusively controlled by the server, and several clients; m is an integer greater than 1; the device includes a control module, a first acquisition module, a determination module, a receiving module, a second acquisition module, and a processing module, wherein: The control module is used to control the m video nodes to collect data in real time at a preset frequency to obtain m first video data. The first acquisition module is used to acquire a preset multi-process acquisition task; the multi-process acquisition task includes a first process acquisition task; a is a positive integer; The determining module is used to determine the a clients corresponding to the a first process acquisition tasks in the multi-process video data acquisition system; The receiving module is used to receive a processing instructions sent by the a clients to the reference video node; the reference video node is any one of the m video nodes; The second acquisition module is used to acquire the first video data corresponding to the reference video node among the m first video data, and obtain the reference first video data; The processing module is used to process and read the reference first video data according to the a processing instructions and the a clients, so as to complete the multi-process acquisition task.

[0007] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing steps in any method of the first aspect of this application.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in any method of the first aspect of this application.

[0009] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in any method of the first aspect of this application. The computer program product may be a software installation package.

[0010] By implementing the embodiments of this application, multiple video nodes can be exclusively managed by the server, and processing instructions from multiple clients for the same video node can be received in a unified manner. The video data of the video node can be processed and shared by multiple processes, thereby improving the efficiency and flexibility of multi-process video data acquisition. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a system architecture diagram of a multi-process video data acquisition system provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the composition of a server provided in an embodiment of this application; Figure 3 This is a schematic diagram of a multi-process video acquisition task processing flow provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 5 This is a flowchart illustrating a multi-process video data acquisition method provided in an embodiment of this application; Figure 6 This is a schematic diagram of a data reading process provided in an embodiment of this application; Figure 7 This is a schematic diagram of an anomaly detection process provided in an embodiment of this application; Figure 8 This is a block diagram of the functional modules of a multi-process video data acquisition device provided in an embodiment of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0014] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0015] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, "multiple" refers to two or more.

[0016] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0017] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0018] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0019] The following is an explanation of the relevant terms used in this application: Video node: refers to V4L2 video node. V4L2 video node is a device file node defined by the Video for Linux 2 (V4L2) video device driver framework under the Linux system. It is the core interface for user-space programs to interact with underlying video acquisition hardware (such as camera modules, video capture cards, etc.).

[0020] 2D hardware accelerator: refers to a dedicated hardware module (or chip unit) specifically designed to efficiently perform fast processing operations of 2D graphics and images. It aims to replace the CPU in completing computationally intensive 2D image processing tasks, reducing CPU load and improving processing efficiency.

[0021] Neural Processing Unit (NPU): This refers to a dedicated hardware module designed specifically for accelerating artificial intelligence (AI) algorithms. It adopts a parallel architecture adapted to neural network computing, excels at handling large-scale matrix operations and convolution operations, and can efficiently execute inference tasks of deep learning models. It features high computing power density and low power consumption.

[0022] Graphics Processing Unit (GPU): It has a large number of lightweight computing cores and supports large-scale parallel data processing. Its performance far exceeds that of traditional CPUs in scenarios such as graphics rendering, video encoding and decoding, and general computing.

[0023] With the widespread adoption of smart tablets in the automotive field, multi-channel video processing has become an essential function for core operations such as AI visual monitoring, video streaming, and real-time display. Current automotive video data acquisition and processing largely rely on the V4L2 video driver framework, but its video nodes employ an exclusive access mechanism, supporting only one client process at a time, which cannot meet the needs of multiple processes using video streams in parallel. Existing solutions often use data copy-forward mechanisms, which suffer from high transmission latency, high system resource consumption, and poor compatibility with multiple systems. Furthermore, the graphics processing operations of video data are executed independently by each process, resulting in resource waste and insufficient processing flexibility.

[0024] Therefore, how to improve the efficiency and flexibility of multi-process video data acquisition is an urgent problem to be solved.

[0025] To address the aforementioned issues, this application provides a multi-process video data acquisition method and related apparatus, applied to a server. The multi-process video data acquisition system includes the server, m video nodes exclusively managed by the server, and several clients; where m is an integer greater than 1. First, the m video nodes are controlled to acquire data in real-time at a preset frequency, resulting in m first video data. Then, a preset multi-process acquisition task is acquired; the multi-process acquisition task includes a first-process acquisition tasks; where a is a positive integer. Next, a clients corresponding to the a first-process acquisition tasks in the multi-process video data acquisition system are determined. a processing instructions sent by the a clients to a reference video node are received; the reference video node is any one of the m video nodes. Then, the first video data corresponding to the reference video node among the m first video data is acquired, resulting in reference first video data. Finally, according to the a processing instructions and the a clients, the reference first video data is processed and read to complete the multi-process acquisition task. By exclusively managing multiple video nodes on the server side and uniformly receiving processing instructions from multiple clients for the same video node, the system processes the video data of that video node and shares it across multiple processes, thereby improving the efficiency and flexibility of multi-process video data acquisition.

[0026] For easier understanding, please refer to Figure 1 , Figure 1 This is a system architecture diagram of a multi-process video data acquisition system provided in an embodiment of this application. The multi-process video data acquisition system includes a server, m video nodes, and several clients.

[0027] The server is responsible for coordinating the acquisition and scheduling of video nodes, receiving and responding to client commands, managing the hardware acceleration process, and allocating permissions to memory blocks.

[0028] Among them, m video nodes are used to collect real-time video data in vehicle scenarios, including video frames from multiple perspectives such as front view, rear view, and surround view. The collected raw video data is stored in its corresponding first memory block, providing a basic data source for subsequent data reading and processing by multiple clients. Each video node is configured with a set of memory blocks, including a first memory block for storing the raw collected data and a second memory block for storing data processed by hardware acceleration. All video nodes are exclusively managed by the server to ensure the security of data collection and transmission.

[0029] Among them, there are several clients, including but not limited to: AI recognition client, rendering client, and network streaming client. Each client runs as an independent process and can send hardware acceleration enable / disable commands to the server according to its business needs, and read video data from the corresponding memory block.

[0030] It is evident that this multi-process video data acquisition system achieves high efficiency, stability, and flexibility in multi-client parallel video data processing and acquisition through on-demand hardware acceleration, shared memory access under permission control, chip load balancing scheduling, and device status pre-detection.

[0031] For easier understanding, please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the composition of a server provided in an embodiment of this application. Deployed on an in-vehicle smart tablet, the server serves as the core control hub of a multi-process video acquisition system. Internally, it integrates a video node management unit, a client management unit, a shared memory management unit, and a system adaptation unit. These units interact and transmit commands via the internal bus of the in-vehicle smart tablet, collaboratively completing the entire process of video acquisition, hardware acceleration, data sharing, and client scheduling. The in-vehicle smart tablet can run multiple operating systems, including Android, Debian, Buildroot Linux, and OpenHarmony, and features a high-performance in-vehicle chip integrating a 2D hardware accelerator, NPU, and GPU, providing hardware computing power support for the efficient operation of the server.

[0032] The video node management unit is responsible for the centralized management and control of multiple V4L2 video nodes in the system, enabling exclusive access to video nodes, configuration of acquisition parameters, and reception and caching of raw video data. At the same time, the video node management unit is deeply integrated with the 2D hardware accelerator built into the vehicle-mounted smart tablet. It can trigger the hardware acceleration process according to the client's processing instructions to complete operations such as format transcoding, cropping, scaling, and rotation of video data, and output processed video data that meets the client's business requirements.

[0033] The client management unit is responsible for receiving hardware acceleration enable / disable commands and data access requests from multiple clients; verifying the legitimacy of client identities; maintaining the association mapping relationship between clients, video nodes, and shared memory blocks; and providing real-time feedback to clients on hardware acceleration processing status, memory permission verification results, device anomaly prompts, and other information, thereby achieving unified scheduling and management of multiple clients.

[0034] The shared memory management unit is responsible for configuring two sets of dedicated shared memory blocks for each video node, which are used to store the original video data in NV12 / UYVY format and the processed video data in RGB / YUV422 format, respectively. The shared memory management unit can realize the creation, dynamic expansion, real-time data synchronization and access control of shared memory blocks. Through the permission verification mechanism, it ensures that the client can only access the memory block with the corresponding permission, and at the same time monitors the data writing status of the memory block to ensure that the client reads complete and undamaged video frame data.

[0035] The system adaptation unit is used to shield the differences in underlying interfaces between different operating systems such as Android and Debian, ensuring that each functional unit of the server can run stably in different system environments. At the same time, it completes the driver adaptation between the server and the integrated chip (2D hardware accelerator, NPU, GPU) of the vehicle-mounted smart tablet, optimizes the hardware computing power calling logic, and improves the execution efficiency of multi-process video acquisition tasks.

[0036] It is evident that this server ensures efficient collaboration and stable operation of multi-process video acquisition tasks by centrally managing video nodes, hardware accelerators, and shared memory, combined with dynamic chip load scheduling and multi-client permission verification.

[0037] For easier understanding, please refer to Figure 3 , Figure 3 This is a schematic diagram of the processing flow of a multi-process video acquisition task provided in an embodiment of this application. The multi-process video acquisition task includes a first video acquisition task. Each first process acquisition task corresponds to an independently running business client (such as an AI recognition client, a rendering client, or a network streaming client). The tasks do not interfere with each other and can be executed in parallel.

[0038] For any first-process acquisition task in a multi-process video acquisition task, the server first performs a task type determination, classifying the task into three categories based on the corresponding client's business type: AI inference / recognition tasks, 2D graphics manipulation tasks, and rendering / visualization tasks. If it's an AI inference / recognition task (such as driver behavior recognition or vehicle surrounding object detection), it's directly scheduled to the NPU for neural network inference computation. If it's a 2D graphics manipulation task (such as image cropping, scaling, rotation, or format transcoding), it's scheduled to the 2D hardware accelerator for hardware acceleration processing. If it's a rendering / visualization task (such as 360-degree surround view stitching or monitoring image overlay), it's directly scheduled to the GPU for graphics rendering operations. It should be noted that processing chips include, but are not limited to, 2D hardware accelerators, NPUs, and GPUs; specific limitations are not specified here.

[0039] Then, the 2D hardware accelerator, NPU, and GPU all interact with each other through a pre-defined zero-copy memory sharing mechanism. It's important to note that the zero-copy memory sharing mechanism allows the data source and data consumer to directly access the same physical memory region. Through memory mapping and shared memory block allocation, redundant copying steps are omitted, and data interaction is completed only by passing memory addresses or permissions. Specifically, the video data processed by the 2D hardware accelerator is directly written to the shared memory block, allowing the NPU / GPU to read it without additional data copying. The raw / processed data required for NPU / GPU processing is read from the shared memory block, and the processing results are also written back to the shared memory block for access by the corresponding clients, significantly reducing data transmission latency and ensuring real-time requirements in automotive scenarios.

[0040] Next, the real-time chip load monitoring step is executed, which involves collecting operating parameters such as the computing power utilization, task queue length, and core temperature of the 2D hardware accelerator, NPU, and GPU, and performing chip load determination based on these parameters. If the determination result for the processing chip corresponding to the first process acquisition task is that the load is too high, a load balancing scheduling strategy is implemented for that processing chip, alleviating the load pressure through task migration, priority adjustment, and resource expansion. If the determination result is that the load is normal, the current first process acquisition task continues to be executed until processing is completed. After each processing chip completes processing, it stores the results in a shared memory block, the system marks the current first process acquisition task status as completed, and outputs the processed data or inference results to the corresponding client. At the same time, the load balancing scheduling result is fed back to the task entry point to optimize the allocation strategy of subsequent first process acquisition tasks, forming a closed-loop scheduling.

[0041] The following is combined Figure 4 The electronic devices in the embodiments of this application will be described. Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 4 As shown, the electronic device includes one or more processors, a memory, a communication interface, and one or more programs. The processor is connected to the memory and the communication interface via an internal communication bus.

[0042] The processor can be used for: Control m video nodes to collect data in real time at a preset frequency to obtain m first video data; Obtain the preset multi-process acquisition task; the multi-process acquisition task includes a first-process acquisition task; a is a positive integer; In a multi-process video data acquisition system, identify the a clients corresponding to the a first-process acquisition tasks. Receive a processing instructions sent by a clients to a reference video node; the reference video node is any one of m video nodes; Obtain the reference first video data from the reference video node among the m first video data; Based on a processing instruction and a client, data processing and data reading are performed on the reference first video data to complete the multi-process acquisition task.

[0043] The one or more programs are stored in the aforementioned memory and configured to be executed by the aforementioned processor, and the one or more programs include instructions for performing any step in the above method embodiments.

[0044] The processor can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, cells, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The communication unit can be a communication interface, transceiver, transceiver circuit, etc., and the storage unit can be a memory.

[0045] The memory can be volatile or non-volatile, or a combination of both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0046] It is understood that the electronic device may include more or fewer structural elements than those shown in the block diagram above, such as a power module, physical buttons, a Wi-Fi module, a speaker, a Bluetooth module, sensors, a display module, etc., without limitation. It is understood that the electronic device may be as follows: Figure 1 The aforementioned server.

[0047] After understanding the software and hardware architecture of this application, the following will be combined with... Figure 5 This application describes a multi-process video data acquisition method according to an embodiment. Figure 5 This is a flowchart illustrating a multi-process video data acquisition method provided in an embodiment of this application, applied to a server. The multi-process video data acquisition system includes the server, m video nodes exclusively managed by the server, and several clients; m is an integer greater than 1. Specifically, it includes the following steps: Step S501: Control the m video nodes to collect data in real time at a preset frequency to obtain m first video data.

[0048] Specifically, the server distributes unified acquisition configuration parameters to the m video nodes (i.e., V4L2 video nodes) under its exclusive control. These parameters include a preset frequency (e.g., 25-30fps), video resolution, and data format. Based on these parameters, the server can synchronously control the m video nodes to start real-time video acquisition in parallel. Each video node writes the acquired raw video data into its corresponding first memory block for caching in real time, ultimately resulting in m sets of first video data stored in their respective first memory blocks, achieving continuous, stable, and low-latency acquisition of multiple video streams.

[0049] Step S502: Obtain the preset multi-process acquisition task.

[0050] The multi-process acquisition task includes *a* first-process acquisition tasks, where *a* is a positive integer. The server retrieves a preset multi-process acquisition task from the system task queue or client requests. This multi-process acquisition task supports multiple independent process clients simultaneously accessing the same or multiple V4L2 video nodes. Each first-process acquisition task corresponds to the video acquisition requirements of an independent process client, including at least one of the following: video node selection, data reading method, whether hardware acceleration processing is enabled, target timeliness requirements, and acquisition duration.

[0051] Step S503: Determine the a clients corresponding to the a first process acquisition tasks in the multi-process video data acquisition system.

[0052] Specifically, the server parses and matches each of the *a* first-process data collection tasks, determining the independent client process to which each task belongs. This results in *a* independently running clients, each corresponding to one of the *a* first-process data collection tasks. These clients include, but are not limited to, AI visual recognition clients, real-time rendering clients, and network streaming clients; no specific limitations are set here. Each client runs as an independent process and does not interfere with the others.

[0053] Step S504: Receive a processing instructions sent by the a clients to the reference video node.

[0054] The reference video node is any one of the m video nodes. The server receives processing instructions from a separate client processes through a unified data access interface. Each processing instruction carries a video node identifier, pointing to the same video node. The server uniformly caches and manages all processing instructions, forming a set of concurrent requests from multiple clients to a single video node, providing a basis for subsequent centralized data processing and shared access.

[0055] Step S505: Obtain the first video data corresponding to the reference video node among the m first video data to obtain the reference first video data.

[0056] Specifically, the server parses the video node identifier carried in each of the 'a' processing instructions and locates the reference video node from the 'm' video nodes under its exclusive control. Then, it accesses the first memory block corresponding to the reference video node, reads the first video data cached in the first memory block and collected in real time at a preset frequency, and identifies it as the reference first video data for subsequent data processing operations based on client instructions.

[0057] Step S506: Based on the a processing instructions and the a clients, perform data processing and data reading on the reference first video data to complete the multi-process acquisition task.

[0058] Each of the m video nodes includes a memory block group, and each memory block group includes a first memory block and a second memory block. The first memory block is used to store the first video data collected by its corresponding video node, and the second memory block is used to store the second video data obtained after hardware acceleration processing of the first video data. The processing instructions include any one of the following: hardware acceleration enable instruction and hardware acceleration disable instruction. The hardware acceleration enable instruction carries a target timeliness requirement, which is used to constrain the hardware acceleration processing time of the reference first video data.

[0059] In one possible embodiment, the server supports dynamic expansion and reclamation of shared memory blocks (i.e., the first memory block and the second memory block). The storage space size of the shared memory blocks can be adjusted according to the video resolution. When the client stops accessing for more than a preset time, the corresponding memory resources are automatically released, and the 2D hardware acceleration processing of the V4L2 video node is stopped to optimize system resource usage.

[0060] In one possible embodiment, both the first and second memory blocks employ contiguous physical memory allocation to reduce data copy latency and ensure the real-time performance of video stream transmission. Specifically, the space of the first memory block corresponding to 720P video data is no less than 1.5MB, and the space of the first memory block corresponding to 1080P video data is no less than 3MB. The space of the second memory block matches the space of the corresponding first memory block to ensure data storage integrity. Then, a shared memory data synchronization mechanism can be established. When the server writes raw video data or data processed by 2D hardware acceleration, it marks the data update timestamp. Each client verifies this timestamp before reading data to ensure the acquisition of the latest frame data, avoiding acquisition errors caused by multiple-process clients reading expired data. Furthermore, integrity checks can be performed on the raw video data in NV12 / UYVY format and the processed data in RGB / YUV422 and other formats in the shared memory block to remove broken frames that occur during transmission or processing, ensuring the accuracy of business client processing and improving the reliability of multi-process video acquisition.

[0061] The step of processing and reading the reference first video data according to the a processing instructions and the a clients to complete the multi-process acquisition task includes the following steps: A1. If all of the a processing instructions are hardware acceleration disable instructions, then hardware acceleration processing of the reference first video data is stopped; the reference first video data is stored in the first memory block of the reference video node. A2. Control the a clients to read the reference first video data to complete the multi-process acquisition task; or, A3. If at least one of the processing instructions in the a processing instructions is the hardware acceleration enable instruction, then the reference first video data is subjected to hardware acceleration processing to obtain reference second video data; the reference second video data is stored in the second memory block of the reference video node; A4. Control the a clients to read the reference first video data or the reference second video data to complete the multi-process acquisition task.

[0062] If all 'a' processing instructions are hardware acceleration disabled, the server controls the 2D hardware accelerator to remain in standby mode, ceasing any hardware acceleration processing on the reference first video data. At this time, the reference first video data stored in the first memory block of the reference video node remains in its original state, without any format conversion, scaling, rotation, or other operations. Then, the server grants parallel read permissions to the first memory block of the reference video node to 'a' clients, controlling each client to retrieve the reference first video data from the first memory block according to its own business cycle (e.g., a 40Hz read frequency). After each client completes its read, it executes the corresponding business operation (e.g., AI recognition, video streaming), completing its respective first-process acquisition task, thereby achieving the overall multi-process acquisition task goal.

[0063] If at least one hardware acceleration enable instruction exists among the *a* processing instructions, the server performs hardware acceleration processing on the reference first video data to obtain hardware-accelerated reference second video data. This reference second video data is stored in the second memory block of the reference video node. Then, *a* clients are controlled to read either the reference first video data or the reference second video data according to their respective business needs to complete the multi-process acquisition task. Each client's business need corresponds to its processing instruction: when the business need is to read video data from the first memory block, the corresponding processing instruction is a hardware acceleration disable instruction; when the business need is to read video data from the second memory block, the corresponding processing instruction is a hardware acceleration enable instruction.

[0064] It is evident that by triggering hardware acceleration processing on demand, the waste of computing power in scenarios without demand can be avoided. At the same time, it supports multiple clients to read raw or processed video data differently, thereby improving the resource utilization and execution flexibility of multi-process acquisition tasks.

[0065] The specific steps of performing hardware acceleration processing on the reference first video data to obtain reference second video data include: B1. Invoke the 2D hardware accelerator corresponding to the server, and assign reference read permission and reference write permission to the 2D hardware accelerator; the reference read permission is used to read the reference first video data in the first memory block of the reference video node; the reference write permission is used to write the reference second video data into the second memory block of the reference video node; B2. Control the 2D hardware accelerator to read the reference first video data according to the reference reading permission; B3. Determine the target processing speed based on the target timeliness requirements; B4. Control the 2D hardware accelerator to perform 2D graphics operations on the reference first video data according to the target processing speed to obtain the reference second video data; the 2D graphics operations include at least one of the following: point drawing, line drawing, image cropping, scaling, and rotation. B5. Control the 2D hardware accelerator to write the reference second video data into the second memory block of the reference video node according to the reference write permission, and release the reference read permission and the reference write permission after the writing is completed.

[0066] In a specific embodiment, after detecting a hardware acceleration enable command sent by at least one client, the server first invokes its integrated independent 2D hardware accelerator. This 2D hardware accelerator includes, but is not limited to, 2D graphics acceleration modules such as RGA, G2D, and VIVante, possessing hardware acceleration processing capabilities for point drawing, line drawing, image cropping, scaling, and rotation. Next, the server assigns the 2D hardware accelerator reference read permissions and reference write permissions corresponding to the reference video node. The reference read permission authorizes the 2D hardware accelerator to access the first memory block of the reference video node to read the reference first video data stored therein; the reference write permission authorizes the 2D hardware accelerator to write the processed reference second video data into the second memory block of the reference video node, achieving independent storage of the original data and the processed data.

[0067] Then, the server controls the 2D hardware accelerator with assigned permissions. Based on the reference read permission, it directly accesses the first memory block of the reference video node, retrieves the complete reference first video data from the first memory block, and copies it to the dedicated memory area of ​​the 2D hardware accelerator. This dedicated memory is the on-chip cache or dedicated physical memory of the 2D hardware accelerator. The copy is completed using a high-speed data transmission channel, reducing data migration time. This reference first video data can be raw acquisition data in NV12 or UYVY format, and the reference first video data in the first memory block is not modified during the reading process, ensuring that clients without hardware acceleration enabled can read the reference first video data normally.

[0068] Next, the server parses the target timeliness requirement carried in the hardware acceleration enable command. This target timeliness requirement constrains the hardware acceleration processing time for a single frame of the reference first video data. For example, in a multi-process acquisition scenario in a vehicle, the processing time for a single frame should not exceed 20ms. Then, combining the resolution (e.g., 720P, 1080P), data format (e.g., NV12, UYVY), and the type of 2D graphics operation to be executed of the reference first video data, a preset computing power matching algorithm is used to calculate and determine the target processing speed that meets the target timeliness requirement. For example, if the target timeliness requirement is a single frame processing time ≤ 20ms, the corresponding target processing speed is no less than 50 frames / second, thus ensuring that the hardware acceleration processing efficiency can adapt to the real-time data reading needs of multi-process clients.

[0069] Then, the server controls the 2D hardware accelerator to perform specified 2D graphics operations on the reference first video data read into dedicated memory according to the target processing speed, obtaining reference second video data. These 2D graphics operations include at least one of the following: point drawing, line drawing, image cropping, scaling, and rotation. It should be noted that 2D graphics operations include, but are not limited to, point / line drawing, image cropping, scaling, rotation, bitBlt, alpha blending, etc., and are not specifically limited here. During the operation execution, the 2D hardware accelerator independently completes graphics transformation and data processing based on its own hardware parallel processing architecture, without occupying CPU core resources. This ensures that the processing speed meets the target requirements while avoiding impact on the operation of other system processes. After processing, reference second video data that meets the client's business needs is generated, for example, video data after operations such as transcoding from NV12 format to RGB format, scaling from 1080P to 720P, and 90-degree clockwise rotation.

[0070] Finally, the server controls the 2D hardware accelerator to write the processed reference second video data completely into the second memory block of the reference video node, based on the reference write permission. After the write is complete, the server controls the 2D hardware accelerator to generate a data write completion identifier. This identifier includes information such as data format, processing timestamp, and integrity checksum, which is used for verification when the client reads the data. Then, the server immediately releases the reference read and reference write permissions allocated to the 2D hardware accelerator and updates the permission status of the first and second memory blocks to "client readable," preventing the hardware accelerator from occupying memory resources for an extended period and ensuring the parallel access requirements of multi-process clients.

[0071] As can be seen, by assigning dedicated read and write permissions to 2D hardware accelerators and binding them to target processing speeds for execution, precise processing of video data and closed-loop permission management can be achieved, ensuring the efficiency of the hardware acceleration processing flow and the security of video data.

[0072] For easier understanding, please refer to Figure 6 , Figure 6 This is a schematic diagram of a data reading process provided in an embodiment of this application, wherein controlling the a clients to read the reference first video data or the reference second video data includes the following specific steps: C1. Obtain the processing instruction corresponding to the reference client from the a processing instructions to obtain the reference processing instruction; the reference client is any one of the a clients. C2. If the reference processing instruction is the hardware acceleration disable instruction, then the reference first video data is read from the first memory block of the reference video node; C3. If the reference processing instruction is the hardware acceleration enable instruction, then the reference second video data is read from the second memory block of the reference video node.

[0073] In a specific embodiment, firstly, any one of the a clients is taken as a reference client, and the processing instructions sent by the reference client to the server are extracted and determined as reference processing instructions.

[0074] If the reference processing instruction is a hardware acceleration disable instruction, it indicates that the business requirement of this reference client is to acquire raw video data, without the need for 2D graphics operations. The server grants the reference client read access to the first memory block of the reference video node, controlling the reference client to directly read the unprocessed reference first video data from the first memory block. This reference first video data maintains the original NV12 or UYVY format, meeting the business scenario requirements of the client that do not require hardware acceleration.

[0075] If the reference processing instruction is a hardware acceleration enable instruction, it indicates that the business requirement of the reference client is to obtain video data processed by 2D hardware acceleration. The server grants the reference client read permission to the second memory block of the reference video node, controlling the reference client to read the reference second video data processed by the 2D hardware accelerator from the second memory block. This reference second video data has completed the corresponding 2D graphics operations to match the client's customized business requirements.

[0076] As can be seen, by matching a differentiated data reading path with corresponding processing instructions for each client, multiple clients can accurately access the original or processed video data, ensuring the orderliness and flexibility of multi-process parallel reading.

[0077] The specific steps of reading the reference first video data from the first memory block of the reference video node include: D1. Receive a first access request sent by the reference client; the first access request includes a first identifier of the reference client and a first number of the first memory block of the reference video node; D2. Perform permission verification on the preset data read permission database according to the first identifier and the first number, and obtain the permission verification result; D3. If the permission verification result is that the verification fails, the reading process is terminated and a permission exception message is sent to the reference client. D4. If the permission verification result is successful, then obtain the first access address and data writing status of the first memory block of the reference video node; the data writing status includes a writing in progress state or a paused writing state. D5. If the data writing status is the writing state, then wait for the data writing status to change to the paused writing state, and then read the reference first video data according to the first access address. D6. If the data writing status is the paused writing status, then the reference first video data is read directly according to the first access address.

[0078] In a specific embodiment, firstly, the server receives a first access request sent by the reference client for reading the original video data. The first access request carries the first identifier of the reference client and the first number of the first memory block of the reference video node, which are used to uniquely identify the client and the memory resource to be accessed, respectively.

[0079] Then, the server performs a matching query in the preset data read permission database based on the first identifier and the first number to verify the access legitimacy of the reference client and obtain the corresponding permission verification result. If the permission verification result is that the verification fails, it indicates that the current reference client does not have the legal permission to access the first memory block. The server immediately terminates the current data read process and returns a permission exception message to the reference client.

[0080] If the permission verification passes, the server obtains the first access address of the first memory block of the reference video node and queries the current data writing status of the first memory block, which includes either a writing in progress state or a paused writing state. If the data writing status is in progress, the server instructs the reference client to wait until the data writing status of the first memory block changes to a paused writing state before reading the reference first video data from the first memory block according to the first access address. If the data writing status is paused writing, the server directly and stably reads the reference first video data from the first memory block of the reference video node according to the first access address, ensuring that the client obtains complete, valid, and undamaged video frame data.

[0081] It is evident that by using dual permission verification of identity identifier and memory number, combined with the timing control of reading data based on the data writing status, the legitimacy and integrity of video data reading by multi-process clients are ensured.

[0082] After reading the reference second video data from the second memory block of the reference video node, the method further includes the following steps: E1. Obtain the service type of the reference client and the collection requirements of the reference process collection task; the reference process collection task is the first process collection task corresponding to the reference client among the a first process collection tasks; E2. Determine the processing chip corresponding to the service type in the server; the processing chip includes an NPU chip or a GPU chip; E3. Obtain the load status of the processing chip; the load status includes at least one of the following: computing power utilization, task queue length, and chip temperature; E4. If the load state meets the preset conditions, control the processing chip to process the reference second video data based on the acquisition requirements to obtain the target second video data. E5. If the load status does not meet the preset conditions, a load balancing scheduling strategy is initiated; the load balancing scheduling strategy is used to adjust the allocation of chip resources and reduce the load pressure on the processing chip.

[0083] In a specific embodiment, firstly, the server reads the attribute information of the reference client that has completed reading the second reference video data, and extracts the business type of the reference client. This business type includes, but is not limited to, AI recognition and rendering. Simultaneously, the server parses the reference process acquisition task corresponding to the reference client to obtain the acquisition requirements corresponding to the reference process acquisition task. These acquisition requirements include, but are not limited to, the accuracy threshold for AI recognition, the resolution requirements for the rendered image, and the upper limit for data processing latency, which are not specifically limited here.

[0084] Then, based on the preset mapping relationship between service types and processing chips, the server matches a dedicated processing chip that is compatible with the service type of the reference client. For example, if the service type of the reference client is AI recognition, the corresponding processing chip is an NPU chip, which is used to perform AI inference tasks such as driver behavior recognition and detection of objects around the vehicle; if the service type of the reference client is rendering, the corresponding processing chip is a GPU chip, which is used to perform graphics rendering tasks such as 360-degree surround view stitching and monitoring screen overlay.

[0085] Next, the server monitors the load of the processing chip to obtain its load status, which includes at least one of the following: computing power utilization, task queue length, and chip temperature. The server compares the real-time collected load status of the processing chip with preset conditions, which may include: computing power utilization ≤ 70%, task queue length ≤ 10, and chip temperature ≤ 85℃, without specific limitations. If the load status meets the preset conditions, it indicates that the processing chip has sufficient computing power resources and a stable operating state. The server issues processing instructions to the processing chip, controlling it to perform targeted processing on the reference second video data based on the acquisition requirements to obtain the target second video data. If the load status of the processing chip does not meet the preset conditions, it indicates that the chip's current computing power is strained or its operating state is unstable. Directly processing tasks will lead to increased latency, decreased accuracy, or even task failure. Therefore, the server immediately initiates a load balancing scheduling strategy to reduce the load pressure on the processing chip. The load balancing scheduling strategy includes at least one of the following: task migration, priority adjustment, resource expansion, and task splitting.

[0086] It is evident that by precisely matching the service type with the processing chip and dynamically scheduling the load status of the processing chip, efficient processing of video data can be achieved, overloading of the processing chip can be avoided, and the stability and real-time performance of multi-process acquisition tasks can be guaranteed.

[0087] For easier understanding, please refer to Figure 7 , Figure 7 This is a flowchart illustrating an anomaly detection method provided in an embodiment of this application. Before receiving the a processing instructions sent by the a clients to the reference video node, the method further includes the following steps: F1. Obtain the first working state of the reference video node and the second working state of the 2D hardware accelerator; F2. If both the first working state and the second working state are normal, then execute the step of receiving the a processing instructions sent by the a clients to the reference video node; F3. If the first working state and / or the second working state is an abnormal state, then the step of receiving the a processing instructions sent by the a clients for the reference video node is not executed, and a working abnormality prompt is fed back to each of the a clients.

[0088] In a specific embodiment, firstly, the server acquires the first working state of the reference video node and the second working state of the 2D hardware accelerator in real time. The first working state characterizes the acquisition and operation status of the reference video node, including normal and abnormal states; the second working state characterizes the hardware processing status of the 2D hardware accelerator, including normal and abnormal states.

[0089] If both the first and second working states are normal, it indicates that the reference video node can execute the video acquisition task normally, and the 2D hardware accelerator can respond to hardware acceleration processing requests normally. At this time, the server continues to execute the subsequent steps, that is, receiving a processing instructions sent by a clients to the reference video node.

[0090] If the first working state is abnormal, and / or the second working state is abnormal, it indicates that there is a fault in the video data acquisition link or the hardware acceleration processing link, and the multi-process acquisition task cannot be executed normally. At this time, the server refuses to receive a processing instructions sent by a clients to the reference video node, and uniformly reports a working abnormality to each of the a clients.

[0091] It is evident that by pre-detecting the working status of video nodes and 2D hardware accelerators, invalid instruction processing can be avoided when the equipment malfunctions, thus ensuring the reliability and execution efficiency of multi-process video acquisition tasks.

[0092] In one possible embodiment, when the reference client is an AI recognition client, it can autonomously choose whether to send a hardware acceleration enable command to the server based on the AI ​​inference task requirements. Specifically, the reference client can perform preprocessing operations on the read video frame data and then input the preprocessed data into a preset NPU neural network model to complete AI inference tasks such as driver behavior recognition, vehicle surrounding object detection, and facial recognition. After inference is completed, it outputs a structured recognition result and synchronizes this result to the vehicle monitoring platform to support business scenarios such as driving safety warnings and personnel identity verification.

[0093] In one possible embodiment, when the reference client is a rendering client, it can choose whether to send a hardware acceleration enable command to the server according to the visualization requirements of the screen, and perform real-time rendering operations through the OpenGL graphical interface to realize visualization display functions such as 360-degree surround view splicing, monitoring screen overlay, and warning information annotation, and output high-definition video screens that meet the requirements of vehicle display terminals.

[0094] In one possible embodiment, when the reference client is a network streaming client, it can autonomously choose whether to send a hardware acceleration enable command to the server based on remote transmission requirements. After reading the target format video data, it performs H.264 / H.265 standard compression encoding and then pushes the stream to the remote monitoring server via RTSP / RTMP streaming media protocols. This network streaming client has a streaming latency of no more than 100ms and supports remote access from multiple terminals such as mobile phones, computers, and monitoring screens, meeting the remote monitoring needs of vehicle-mounted videos.

[0095] It should be noted that the data access and hardware acceleration commands between clients are uniformly coordinated through the server. Specifically, when multiple clients send hardware acceleration commands to the same video node, the server executes unified hardware acceleration processing according to the command reception order and grants shared memory access permissions to each client. Each client can read complete data frames between video frame refresh intervals without setting access priorities, ensuring stable multi-process video data acquisition and avoiding process conflicts and data loss.

[0096] In one possible embodiment, the multi-process video acquisition method also includes an intelligent frame caching and predictive reading mechanism. This mechanism can further improve the efficiency of concurrent access across multiple processes through caching strategy optimization, predictive reading timing, and differential data updates. Specifically, the server maintains a cache of the most recent N frames of video data in a shared memory block (i.e., the first memory block or the second memory block) (N is a positive integer, flexibly configurable based on the server's memory resources). Clients can choose to read the latest frame data or historical frame data within a specified time interval according to their business needs. For example, an AI recognition client can backtrack and read the previous 3 frames for behavioral trajectory analysis, while a rendering client can directly read the latest frame data for real-time video display, significantly improving the flexibility and scene adaptability of multi-process data reading.

[0097] The server can predict the next read time point based on the client's historical read patterns (including read frequency, frame data type preferences, access time intervals, etc.) using a preset algorithm. Before the predicted time point, the server preloads the corresponding video data into the high-speed cache area of ​​the shared memory block and performs data integrity verification, reducing the waiting latency of multi-process clients and ensuring the real-time performance of AI inference, image rendering, and other services. The server can also perform pixel-level difference detection on consecutive video frames and calculate the content change rate between adjacent frames. New frame data is only written to the shared memory block when the change rate exceeds a preset threshold, avoiding meaningless full-frame data update operations. This design can significantly reduce the write frequency of the shared memory block, reduce the computing power consumption and memory bandwidth usage of the automotive chip, and adapt to high-concurrency access scenarios from multiple clients.

[0098] In one possible embodiment, the server further includes a frame difference detection module. After receiving the raw video data collected by the video nodes, the frame difference detection module uses an inter-frame difference or hash comparison algorithm to calculate the pixel difference between the current frame and the previous frame. If the pixel difference is greater than or equal to a preset threshold (the threshold ranges from 5% to 10% pixel change), the current frame is determined to be a valid new frame, and the server immediately updates the shared memory block of the video nodes and writes the valid new frame into the smart frame buffer. If the pixel difference is less than the preset threshold, the current frame is determined to be a static duplicate frame, and the server directly discards the duplicate frame to avoid invalid data occupying memory bandwidth. The smart frame buffer adopts a circular queue structure to maintain the most recent N frames of video data (N is usually 5 to 10 frames), which can support the historical frame backtracking reading needs of multi-process clients without the need for additional storage of the full video stream.

[0099] The multi-process client initiates data read requests to the server at a frequency of 40Hz. The server performs differentiated data distribution based on the business type of the request: if the client is a business type that requires time-series analysis, such as driver behavior recognition, and needs to obtain historical action trajectory data, then it extracts historical frame data for a specified time interval from the historical cache queue of the intelligent frame buffer; if the client is a business type that requires low-latency processing, such as real-time rendering or remote streaming, then it directly accesses the current frame data in the shared memory block to ensure the real-time requirements of the business.

[0100] The server-side built-in predictive reading engine continuously collects and analyzes the historical reading timestamps of each client, calculating the average reading interval Δt (e.g., 25ms corresponds to a reading frequency of 40Hz). Based on linear regression or moving average algorithms, it predicts the client's next reading time point T+Δt. 5ms before the prediction time point, the server proactively preloads the target video data (i.e., the video data needed by the client) into the CPU cache. When a client read request arrives: if the prediction is correct, the data is returned directly from the CPU cache with a data acquisition latency of <5ms; if the prediction is incorrect (e.g., due to sudden client frequency reduction or reading frequency fluctuations), it automatically reverts to the real-time acquisition and processing flow, with a data acquisition latency of 15~20ms. After each read request is processed, the predictive reading engine automatically updates the parameters of the prediction model, forming an adaptive optimization loop to continuously improve prediction accuracy.

[0101] As can be seen, by predictively reading video data, invalid memory write operations can be reduced by 30% to 50%, and the average data acquisition latency of multi-process clients can be reduced from 15ms to <5ms. At the same time, it supports the function of historical frame backtracking, adapts to business scenarios that require time-series data support, such as AI behavior analysis, and takes into account both system resource utilization and adaptability to multiple business scenarios.

[0102] In one possible embodiment, the entire 2D hardware-accelerated processing flow can be broken down into four independent functional stages: cropping, scaling, format conversion, and effects overlay. Each stage can be configured with its own processing parameters and execution logic. For example, the cropping stage supports setting the ROI (Region of Interest) coordinates according to client requirements, and the format conversion stage supports bidirectional conversion between NV12 / UYVY and RGB / YUV422 to meet the differentiated processing needs of multi-process clients. A pipelined parallel processing mechanism is employed, breaking the timing limitations of traditional serial processing: post-processing of the current video frame (such as format conversion and effects overlay) and pre-processing of the next video frame (such as cropping and scaling) can be performed simultaneously. Through the pipelined flow of frame data, the throughput of the 2D hardware accelerator is significantly improved, supporting the high-concurrency video acquisition needs of multi-process clients and reducing the overall processing time of a single frame. The server supports dynamically adjusting the combination of pipeline stages according to the actual needs of the business client: for network streaming clients that only need format conversion, the cropping, scaling, and effects overlay stages can be skipped directly; for rendering clients that need high-definition rendering, the cropping, scaling, and format conversion stages can be combined for processing; by skipping unnecessary processing stages, the ineffective consumption of computing power is reduced, and the collaborative execution efficiency of multi-process tasks is optimized.

[0103] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0104] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0105] When dividing each function into modules according to its corresponding function. Figure 8This is a functional module block diagram of a multi-process video data acquisition device 800 provided in an embodiment of this application. The multi-process video data acquisition device 800 is applied to a server. The multi-process video data acquisition system includes the server, m video nodes exclusively controlled by the server, and several clients; m is an integer greater than 1. The multi-process video data acquisition device 800 includes a control module 810, a first acquisition module 820, a determination module 830, a receiving module 840, a second acquisition module 850, and a processing module 860, wherein: Optionally, each of the m video nodes includes a memory block group, and each memory block group includes a first memory block and a second memory block; the first memory block is used to store the first video data collected by its corresponding video node, and the second memory block is used to store the second video data obtained after hardware acceleration processing of the first video data; the processing instruction includes any one of the following: hardware acceleration enable instruction and hardware acceleration disable instruction, wherein the hardware acceleration enable instruction carries a target timeliness requirement, and the target timeliness requirement is used to constrain the hardware acceleration processing time of the reference first video data; In the process of performing data processing and data reading on the reference first video data according to the a processing instructions and the a clients to complete the multi-process acquisition task, the processing module 860 is specifically used for: If all of the a processing instructions are hardware acceleration disable instructions, then hardware acceleration processing of the reference first video data is stopped; the reference first video data is stored in the first memory block of the reference video node. Control the a clients to read the reference first video data to complete the multi-process acquisition task; or, If at least one of the a processing instructions is the hardware acceleration enable instruction, then the reference first video data is subjected to hardware acceleration processing to obtain reference second video data; the reference second video data is stored in the second memory block of the reference video node; The a clients are controlled to read data from the reference first video data or the reference second video data to complete the multi-process acquisition task.

[0106] Optionally, in the process of performing hardware acceleration processing on the reference first video data to obtain reference second video data, the processing module 860 is further specifically used for: The server calls the corresponding 2D hardware accelerator and assigns reference read permission and reference write permission to the 2D hardware accelerator; the reference read permission is used to read the reference first video data in the first memory block of the reference video node; the reference write permission is used to write the reference second video data into the second memory block of the reference video node; Control the 2D hardware accelerator to read the reference first video data according to the reference reading permission; Determine the target processing speed based on the target timeliness requirements; The 2D hardware accelerator is controlled to perform 2D graphics operations on the reference first video data according to the target processing speed to obtain the reference second video data; the 2D graphics operations include at least one of the following: point drawing, line drawing, image cropping, scaling, and rotation. The 2D hardware accelerator is controlled to write the reference second video data into the second memory block of the reference video node according to the reference write permission, and the reference read permission and the reference write permission are released after the writing is completed.

[0107] Optionally, in controlling the a clients to read the reference first video data or the reference second video data, the processing module 860 is further specifically used for: Obtain the processing instruction corresponding to the reference client from the a processing instructions to obtain the reference processing instruction; the reference client is any one of the a clients. If the reference processing instruction is the hardware acceleration disable instruction, then the reference first video data is read from the first memory block of the reference video node; If the reference processing instruction is the hardware acceleration enable instruction, then the reference second video data is read from the second memory block of the reference video node.

[0108] Optionally, in reading the reference first video data from the first memory block of the reference video node, the processing module 860 is further specifically configured to: Receive a first access request sent by the reference client; the first access request includes a first identifier of the reference client and a first number of a first memory block of the reference video node; Based on the first identifier and the first number, the preset data read permission database is verified to obtain the permission verification result. If the permission verification result is that the verification fails, the reading process is terminated and a permission error message is sent to the reference client. If the permission verification result is successful, then the first access address and data writing status of the first memory block of the reference video node are obtained; the data writing status includes a writing in progress state or a paused writing state. If the data writing status is the writing state, then wait for the data writing status to change to the paused writing state, and then read the reference first video data according to the first access address; If the data writing status is the paused writing status, then the reference first video data is read directly according to the first access address.

[0109] Optionally, after reading the reference second video data from the second memory block of the reference video node, the processing module 860 is further configured to: Obtain the service type of the reference client and the collection requirements of the reference process collection task; the reference process collection task is the first process collection task corresponding to the reference client among the a first process collection tasks; Determine the processing chip corresponding to the service type in the server; the processing chip includes an NPU chip or a GPU chip. Obtain the load status of the processing chip; the load status includes at least one of the following: computing power utilization, task queue length, and chip temperature; If the load state meets the preset conditions, the processing chip is controlled to process the reference second video data based on the acquisition requirements to obtain the target second video data. If the load status does not meet the preset conditions, a load balancing scheduling strategy is initiated; the load balancing scheduling strategy is used to adjust the allocation of chip resources and reduce the load pressure on the processing chip.

[0110] Optionally, before receiving the a processing instructions sent by the a clients for the reference video node, the processing module 860 is further specifically used for: Obtain the first working state of the reference video node and the second working state of the 2D hardware accelerator; If both the first working state and the second working state are normal, then the step of receiving the a processing instructions sent by the a clients to the reference video node is executed; If the first working state and / or the second working state is an abnormal state, then the step of receiving the a processing instructions sent by the a clients for the reference video node will not be executed, and a working abnormality prompt will be fed back to each of the a clients.

[0111] It is evident that by exclusively managing multiple video nodes on the server side and uniformly receiving processing instructions from multiple clients for the same video node, the video data of that video node can be processed and shared across multiple processes, thereby improving the efficiency and flexibility of multi-process video data acquisition.

[0112] It should be noted that the specific implementation of each operation can be described in the corresponding description of the method embodiments shown above. The multi-process video data acquisition device 800 can be used to execute the method embodiments of this application, and will not be described again here.

[0113] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0114] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0115] It should be noted that, for the sake of simplicity, the above embodiments are all described as a series of actions. Those skilled in the art should understand that this application is not limited to the described order of actions, as some steps in the embodiments of this application can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions, steps, modules, or units involved are not necessarily essential to the embodiments of this application.

[0116] In the above embodiments, the descriptions of each embodiment in this application have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0117] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0118] The steps of the methods or algorithms described in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (EEPROM), registers, hard disk, portable hard disk, read-only optical disk (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Furthermore, the ASIC can reside in a terminal device or management device. Alternatively, the processor and storage medium can exist as discrete components in the terminal device or management device.

[0119] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in the embodiments of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0120] The modules / units included in the various devices and products described in the above embodiments can be software modules / units, hardware modules / units, or a combination of both. For example, for devices and products applied to or integrated into a chip, all modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For devices and products applied to or integrated into a chip module, all modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on the processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into terminal equipment, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal equipment. Alternatively, at least some modules / units can be implemented through a software program that runs on the processor integrated within the terminal equipment, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0121] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above descriptions are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A multi-process video data acquisition method, characterized in that, Applied to the server side; the multi-process video data acquisition system includes the server, m video nodes exclusively managed by the server, and several clients; m is an integer greater than 1; the method includes: The m video nodes are controlled to collect data in real time at a preset frequency to obtain m first video data. Obtain a preset multi-process acquisition task; the multi-process acquisition task includes a first-process acquisition task; a is a positive integer; Determine the a clients corresponding to the a first-process acquisition tasks in the multi-process video data acquisition system; Receive a processing instructions sent by the a clients to a reference video node; the reference video node is any one of the m video nodes; Obtain the first video data corresponding to the reference video node from the m first video data to obtain the reference first video data; Based on the a processing instructions and the a clients, the reference first video data is processed and read to complete the multi-process acquisition task.

2. The method as described in claim 1, characterized in that, Each of the m video nodes includes a memory block group, and each memory block group includes a first memory block and a second memory block. The first memory block is used to store the first video data collected by its corresponding video node, and the second memory block is used to store the second video data obtained after hardware acceleration processing of the first video data. The processing instructions include any one of the following: hardware acceleration enable instruction and hardware acceleration disable instruction. The hardware acceleration enable instruction carries a target timeliness requirement, and the target timeliness requirement is used to constrain the hardware acceleration processing time of the reference first video data. The step of processing and reading the reference first video data according to the a processing instructions and the a clients to complete the multi-process acquisition task includes: If all of the a processing instructions are hardware acceleration disable instructions, then hardware acceleration processing of the reference first video data is stopped; the reference first video data is stored in the first memory block of the reference video node. Control the a clients to read the reference first video data to complete the multi-process acquisition task; or, If at least one of the a processing instructions is the hardware acceleration enable instruction, then the reference first video data is subjected to hardware acceleration processing to obtain reference second video data; the reference second video data is stored in the second memory block of the reference video node; The a clients are controlled to read data from the reference first video data or the reference second video data to complete the multi-process acquisition task.

3. The method as described in claim 2, characterized in that, The step of performing hardware acceleration processing on the reference first video data to obtain reference second video data includes: The server calls the corresponding 2D hardware accelerator and assigns reference read permission and reference write permission to the 2D hardware accelerator; the reference read permission is used to read the reference first video data in the first memory block of the reference video node; the reference write permission is used to write the reference second video data into the second memory block of the reference video node; Control the 2D hardware accelerator to read the reference first video data according to the reference reading permission; Determine the target processing speed based on the target timeliness requirements; The 2D hardware accelerator is controlled to perform 2D graphics operations on the reference first video data according to the target processing speed to obtain the reference second video data; the 2D graphics operations include at least one of the following: point drawing, line drawing, image cropping, scaling, and rotation. The 2D hardware accelerator is controlled to write the reference second video data into the second memory block of the reference video node according to the reference write permission, and the reference read permission and the reference write permission are released after the writing is completed.

4. The method as described in claim 2, characterized in that, The control of the a clients to read data from the reference first video data or the reference second video data includes: Obtain the processing instruction corresponding to the reference client from the a processing instructions to obtain the reference processing instruction; the reference client is any one of the a clients. If the reference processing instruction is the hardware acceleration disable instruction, then the reference first video data is read from the first memory block of the reference video node; If the reference processing instruction is the hardware acceleration enable instruction, then the reference second video data is read from the second memory block of the reference video node.

5. The method as described in claim 4, characterized in that, The step of reading the reference first video data from the first memory block of the reference video node includes: Receive a first access request sent by the reference client; the first access request includes a first identifier of the reference client and a first number of a first memory block of the reference video node; Based on the first identifier and the first number, the preset data read permission database is verified to obtain the permission verification result. If the permission verification result is that the verification fails, the reading process is terminated and a permission error message is sent to the reference client. If the permission verification result is successful, then the first access address and data writing status of the first memory block of the reference video node are obtained; the data writing status includes a writing in progress state or a paused writing state. If the data writing status is the writing state, then wait for the data writing status to change to the paused writing state, and then read the reference first video data according to the first access address; If the data writing status is the paused writing status, then the reference first video data is read directly according to the first access address.

6. The method as described in claim 4, characterized in that, After reading the reference second video data from the second memory block of the reference video node, the method further includes: Obtain the service type of the reference client and the collection requirements of the reference process collection task; the reference process collection task is the first process collection task corresponding to the reference client among the a first process collection tasks; Determine the processing chip corresponding to the service type in the server; the processing chip includes an NPU chip or a GPU chip. Obtain the load status of the processing chip; the load status includes at least one of the following: computing power utilization, task queue length, and chip temperature; If the load state meets the preset conditions, the processing chip is controlled to process the reference second video data based on the acquisition requirements to obtain the target second video data. If the load status does not meet the preset conditions, a load balancing scheduling strategy is initiated; the load balancing scheduling strategy is used to adjust the allocation of chip resources and reduce the load pressure on the processing chip.

7. The method according to any one of claims 3-6, characterized in that, Before receiving the a processing instructions sent by the a clients to the reference video node, the method further includes: Obtain the first working state of the reference video node and the second working state of the 2D hardware accelerator; If both the first working state and the second working state are normal, then the step of receiving the a processing instructions sent by the a clients to the reference video node is executed; If the first working state and / or the second working state is an abnormal state, then the step of receiving the a processing instructions sent by the a clients for the reference video node will not be executed, and a working abnormality prompt will be fed back to each of the a clients.

8. A multi-process video data acquisition device, characterized in that, Applied to the server side; the multi-process video data acquisition system includes the server, m video nodes exclusively controlled by the server, and several clients; m is an integer greater than 1; the device includes a control module, a first acquisition module, a determination module, a receiving module, a second acquisition module, and a processing module, wherein: The control module is used to control the m video nodes to collect data in real time at a preset frequency to obtain m first video data. The first acquisition module is used to acquire a preset multi-process acquisition task; the multi-process acquisition task includes a first process acquisition task; a is a positive integer; The determining module is used to determine the a clients corresponding to the a first process acquisition tasks in the multi-process video data acquisition system; The receiving module is used to receive a processing instructions sent by the a clients to the reference video node; the reference video node is any one of the m video nodes; The second acquisition module is used to acquire the first video data corresponding to the reference video node among the m first video data, and obtain the reference first video data; The processing module is used to process and read the reference first video data according to the a processing instructions and the a clients, so as to complete the multi-process acquisition task.

9. An electronic device, characterized in that, include: Processor, memory, communication interface, and one or more programs; The one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-7.

Images