Cross-platform data processing method and device, framework, electronic device, and storage medium

Abstract

The application provides a cross-platform data processing method and device, framework, electronic equipment and storage medium, the cross-platform visual data processing method comprises the following steps: obtaining an entrance; obtaining a custom device; creating a device task based on the custom device; executing the device task; stopping all devices after the device task is executed. The above method is carried out in the cross-platform visual computing framework of the application, the cross-platform visual computing framework of the application is provided to application developers in the form of header files and binary libraries, and multiple programs can call underlying computing modules through a cross-platform unified API interface to complete CV operation calculation, so as to reduce the learning cost and development difficulty of application developers using hardware acceleration units, and improve the development efficiency and iteration speed of the application.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, specifically to a cross-platform data processing method and apparatus, frame, electronic device, and storage medium. Background Technology

[0002] On various computing platforms for Computer Vision (CV), there are different types of heterogeneous hard-core accelerators, programmable processors, or relatively general-purpose computing modules such as DSPs (Digital Signal Processors), CPUs (Central Processing Units), and GPUs (Graphics Processing Units). For the same CV operation, different computing modules will differ in terms of the size and arrangement of the data being processed, the range of data they can process, and the acceleration methods. Their respective SDKs (Software Development Kits) also provide different software interfaces and usage conventions. This presents a significant challenge to developing cross-platform, portable applications: it requires learning the functional characteristics, limitations, and usage methods of different computing modules on different platforms; adapting to the interfaces provided by the SDKs of different computing modules; and manually configuring which computing module to use for computation on different platforms.

[0003] In addition to the CPU and BPU (Branch Processing Unit), CV computing platforms also include other hardware acceleration units such as GDC (Graphic Display Controller), Stitch, optical flow, and DSP. However, when developing AI (Artificial Intelligence) applications, besides deep learning-based model inference, a large number of non-AI basic CV operations are required, such as image remapping, blurring, stitching, and resizing. These basic CV operations either need to be executed on the CPU, consuming significant CPU computing power, or require code adaptation for the specific chip, calling low-level interfaces provided by the system software.

[0004] Therefore, there is an urgent need to develop a universal computing framework to manage the different computing modules on the chip in a unified manner, and to realize a cross-platform execution framework, so as to reduce the learning cost and development difficulty of application developers using hardware acceleration units, and improve the development efficiency and iteration speed of applications. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the present invention provides a cross-platform visual operation method and operation framework, electronic device, and storage medium to solve the above-mentioned technical problems.

[0006] This invention provides a cross-platform visual data processing method, comprising: obtaining an entry point, wherein the entry point is the starting point of data processing; obtaining a custom device; creating a device task based on the custom device; executing the device task; and stopping all devices after the device task has been executed.

[0007] In one embodiment of the present invention, the process of obtaining the entry point further includes pre-launch preparation, which includes: customizing the device; customizing the visual computing logic; registering the custom device and the visual computing logic into the computing framework; and compiling the computing framework to generate a binary library to obtain the visual computing framework.

[0008] In one embodiment of the present invention, based on the device, a device task is created, including: obtaining the visual computing logic of the device; creating a runtime environment (IContext), framework operation and related parameters (ICVOP), system-allocated resource container (CVDevSession), and context options (option) for executing the visual computing logic based on the visual computing logic of the device, and setting custom parameters; and creating a device task (CVDevTask) using the runtime environment, framework operation and related parameters, system-allocated resource container, and option parameters.

[0009] In one embodiment of the present invention, executing the device task includes: allocating input media memory based on the device task; allocating output media memory based on the device task; and running the task based on the input media memory and the output media memory.

[0010] In one embodiment of the present invention, after the device task is completed, all devices are stopped, the device task is destroyed, the input and output media memory is released, the resource containers allocated by the system are destroyed, and all devices are stopped.

[0011] The cross-platform visual data processing device of the present invention includes: an entry acquisition module, a device acquisition module, a task creation module, a task execution module, and a device stop module. The entry acquisition module is configured to acquire a visual framework entry; the device acquisition module is configured to acquire a device; the task creation module is configured to create a device task based on the device; the task execution module is configured to execute the device task; and the device stop module is configured to stop all devices.

[0012] The cross-platform visual computing framework of the present invention includes a C++ interface layer, a front-end access layer, a core implementation layer, a device interface layer, and a device layer. The device layer is communicatively connected to the core implementation layer through the device connection layer. The C++ interface layer is used by developers for development. The front-end access layer is used to connect the C++ interface layer and the core implementation layer. The core implementation layer includes the cross-platform visual data processing device of the present invention.

[0013] In one embodiment of the present invention, the C++ interface layer includes the interface classes ICVOP, IImage, IContext, ISession, ITask, and IDevice. ICVOP represents the CV operation to be performed and related parameters. IImage is an abstraction of image data. IContext represents the runtime environment for executing CVOP. ISession represents a container of resources allocated by the system for CVOP execution. ITask is used to record the state of a CVOP execution. IDevice is an abstraction of the underlying device, used for querying / profiling and providing the required interface to an external scheduler.

[0014] The electronic device of the present invention includes one or more processors and a storage device, the storage device being used to store one or more programs, which, when executed by the one or more processors, enable the electronic device to implement the cross-platform visual data processing method of the present invention.

[0015] The present invention has a computer program stored on a computer-readable storage medium, which, when executed by a computer's processor, causes the computer to perform the cross-platform visual data processing method of the present invention.

[0016] The beneficial effects of this invention are as follows: The cross-platform vision algorithm framework in this invention is provided to application developers as middleware in the form of header files and binary libraries. Multiple programs can call the underlying computing modules through a unified cross-platform API interface to complete the calculation of CV operations, thereby reducing the learning cost and development difficulty for application developers to use hardware acceleration units, and improving the development efficiency and iteration speed of applications.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0019] Figure 1 This is a flowchart illustrating a cross-platform visual data processing method in an exemplary embodiment of this application;

[0020] Figure 2 This is a flowchart illustrating the development process of a visual computing framework, as shown in an exemplary embodiment of this application.

[0021] Figure 3 yes Figure 2 The flowchart of step S230 in the illustrated embodiment is shown in an exemplary embodiment;

[0022] Figure 4 yes Figure 1 The flowchart of step S130 in the illustrated embodiment is shown in an exemplary embodiment;

[0023] Figure 5 yes Figure 4 The flowchart of step S420 in the illustrated embodiment is shown in an exemplary embodiment.

[0024] Figure 6 yes Figure 1 The flowchart of step S140 in the illustrated embodiment is shown in an exemplary embodiment;

[0025] Figure 7 yes Figure 1 The flowchart of step S150 in the illustrated embodiment is shown in an exemplary embodiment;

[0026] Figure 8 This is a block diagram illustrating a cross-platform visual data processing apparatus in an exemplary embodiment of this application;

[0027] Figure 9 This is an exemplary embodiment of the cross-platform visual computing framework shown in this application.

[0028] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0029] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0030] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0031] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0032] First, it's important to clarify that computer vision refers to using computers to achieve human visual functions—the perception, recognition, and understanding of three-dimensional scenes in the objective world. This means that the research goal of computer vision technology is to enable computers to recognize three-dimensional environmental information through two-dimensional images. Therefore, it's not only necessary for machines to perceive the geometric information (shape, position, pose, motion, etc.) of objects in the three-dimensional environment, but also to describe, store, recognize, and understand them. Computer vision uses geometric, physical, and learning techniques to construct models and statistical methods to process data.

[0033] The main functions of the algorithm integration framework are: (1) to provide a standardized algorithm integration platform to facilitate the rapid integration and verification of various functional algorithms; and (2) to serve as a functional platform that can connect clients, hardware devices, and algorithms to realize the analysis and transmission of various data. Existing algorithm integration frameworks generally include four major processes: algorithm library import, template creation, algorithm execution, and result display. Algorithm import refers to designing parameter configurations and developing dynamic libraries that meet the requirements of the corresponding algorithm framework. By creating templates, algorithm task flows are assigned to each data. The framework calls and executes each algorithm according to the task flow and sends the results of the algorithm execution to the client for display.

[0034] Currently, various computing platforms for computer vision employ different types of heterogeneous hard-core accelerators, programmable processors, and other acceleration modules, or utilize relatively general-purpose computing modules such as DSPs, CPUs, and GPUs. For the same computer vision operation, different computing modules will differ in terms of the size and arrangement of the data being processed, the range of data they can handle, and the acceleration methods. Furthermore, the software interfaces and usage conventions provided by their respective SDKs also differ, severely impacting the development efficiency and iteration speed of application development.

[0035] To address these issues, embodiments of this application propose a cross-platform visual data processing method, a cross-platform visual data processing device, a cross-platform visual computing framework, an electronic device, and a computer-readable storage medium, which will be described in detail below.

[0036] Please see Figure 1 , Figure 1 This is a flowchart illustrating a cross-platform visual data processing method in an exemplary embodiment of this application.

[0037] like Figure 1 As shown, in an exemplary embodiment, the cross-platform visual data processing method includes at least steps S310 to S350, which are described in detail below:

[0038] Step S110: Obtain the entry point, which is the starting position of data processing.

[0039] First, it should be noted that the data processing in this application is performed within a visual computing framework. This framework is a general-purpose computing framework that unifies the management of different computing modules on a chip and enables unified execution, API calls, and usage conventions across platforms. This reduces the learning curve and development difficulty for application developers using hardware acceleration units, improving application development efficiency and iteration speed. Accessing the visual computing framework is equivalent to starting the framework.

[0040] All interfaces within the framework must be obtained directly or indirectly from CVFramework::Instance(). When CVFramework::Instance() is called for the first time, the framework is initialized, calling the initialization functions of each module. These initialization functions are registered with the framework initialization mechanism through the constructor of the global object of the Dummy.

[0041] Step 120: Obtain the custom device.

[0042] Custom devices, or user-defined devices, refer to heterogeneous computing modules, such as DSPs, GPUs, CPUs, BPUs, and GDCs. Each module differs in the size and arrangement of data it processes, the range of data it can handle, and its acceleration methods. During the development of a visual computing framework, developers first register the user-defined devices and custom visual computing logic into the framework. The framework is then compiled to generate a binary library. In other words, the visual computing framework contains the visual computing logic for the corresponding devices. Users can obtain the corresponding computer vision operating devices (CVDevice) through the Device Manager.

[0043] CVDevice*dev=DeviceManager::Instance()->GetDevice("HelloDevice").

[0044] Step 130: Create a device task based on the custom device.

[0045] The device creation task involves creating a container for the system's operating environment, related parameters, and allocated resources based on the acquired custom device and visual computing logic.

[0046] Step S140: Execute the device task.

[0047] Executing device tasks means performing visual logic calculations for each device within the visual algorithm framework. When executing tasks, the visual logic calculation tasks for each device are performed based on the operating environment, relevant parameters, and system-allocated resource containers configured in step S130.

[0048] Step S150: Stop the operation of all devices, that is, stop all devices after each device has completed its task.

[0049] Please see Figure 2 , Figure 2 This is a flowchart illustrating the development process of a visual algorithm framework, as shown in an exemplary embodiment of this application. Figure 2 As shown, before obtaining the entry point of the visual algorithm framework, there is also a pre-launch preparation, which is the development of the visual computing framework, and may include steps S210 to S240, which are described in detail below:

[0050] S210, a custom device.

[0051] Custom devices, also known as user-defined devices, refer to computing modules such as DSP, GPU, CPU, BPU, Gdc, etc. Each module has different data processing sizes and arrangements, data ranges that can be processed, and acceleration methods. Users can choose from a variety of different computing modules according to their needs.

[0052] S220, custom visual computing logic.

[0053] Different devices correspond to different visual computing logics. Custom visual computing logic means customizing the corresponding visual computing logic based on the selected device.

[0054] S230, the custom device and the visual computing logic are registered into the computing framework.

[0055] Custom devices and visual computing logic are registered into the computing framework, so that the computing framework stores the corresponding devices and visual computing logic, which users can directly retrieve from the computing framework when using it.

[0056] S240, Compile the computing framework to generate a binary library to obtain the visual computing framework.

[0057] Figure 3 yes Figure 2 The flowchart of step S230 in the illustrated embodiment is shown in an exemplary embodiment. (See attached flowchart.) Figure 3 As shown, registering the device and the visual computing logic into the computing framework may include steps S310 to S340, which are described in detail below:

[0058] S310, Registered Device:

[0059] REGISTER_SYS_INIT_FUNCTION(MOD_DEVICE_LEVEL,hello_device,hello_register_function,false);

[0060] The `hello_register_function` method registers the user-implemented `HelloDevice` class with `DeviceManager`. `HelloDevice` inherits from `IDevBackend`.

[0061] S320, Start the device:

[0062] REGISTER_SYS_INIT_FUNCTION(MOD_MID_LEVEL,start_all_device,register_device_start_function,true).

[0063] S330, Register Device Scheduler:

[0064] The application sets the string attribute "device_scheduler" in the IContext to the new scheduling policy name to use the new policy.

[0065] REGISTER_SYS_INIT_FUNCTION(MOD_DEVICE_LEVEL,sequence_scheduler,register_sequence_scheduler,false);

[0066] S340, Register a CVOP factory:

[0067] REGISTER_SYS_INIT_FUNCTION(MOD_OP_LEVEL,HelloOP,op_register_function,false).

[0068] Figure 4 yes Figure 1 The flowchart of step S130 in the illustrated embodiment is shown in an exemplary embodiment. Figure 5 yes Figure 4 The flowchart of step S420 in the illustrated embodiment is shown in an exemplary embodiment. Figure 4 As shown, based on the custom device, creating a device task may include steps S410 to S430, which are described in detail below:

[0069] Step S410: Obtain the custom visual computing logic.

[0070] Each device in the visual computing framework has corresponding visual computing logic; therefore, obtaining a custom device allows you to obtain the visual computing logic.

[0071] Step S420: Based on the custom device and visual computing logic, create IContext, ICVOP, CVDevSession, and opinion, and set custom parameters. The creation process is as follows: Figure 5 As shown:

[0072] S510, Create IContext

[0073] IContext*context=new CVContext();

[0074] S520, Create ICVOP

[0075] ICVOP*op=CreateOP();→OPRegistry::Instance()->Create("HelloOP",1)

[0076] S530, Create CVDevSession

[0077] CVDevSession*session=dev->CreateSession(context,op,input_shapes,MEM_PIX_FMT_NV12);

[0078] S540, Create an option and set custom parameters.

[0079] CVOption*option=new CVOption("param");

[0080] option->SetParamMem("vio_out",&result_addr,sizeof(uint8_t*));

[0081] option->SetParamPointer("int_pointer",&demo_data).

[0082] Step S430: After all parameters in step S220 are configured, create a CVDevTask, as follows:

[0083] CVDevTask*sync_task=session->CreateTask(true,inputs,outputs,kTaskPrioMiddle,option).

[0084] Figure 6 yes Figure 1 The flowchart of step S140 in the illustrated embodiment is shown in an exemplary embodiment. (See attached flowchart.) Figure 6 As shown, the process of executing the device task may include steps S610 to S630, which are described in detail below:

[0085] Step S610: Allocate input media memory based on the device task:

[0086] session->AllocInputMem(inputs);

[0087] Step S620: Allocate output media memory based on the device task:

[0088] session->AllocOutputMem(outputs);

[0089] Step S630: Based on the input media memory and the output media memory, run the task:

[0090] session->RunTask(sync_task);

[0091] The tasks are executed sequentially as follows:

[0092] CVDevSession::RunTask(CVDevTask*cv_task)

[0093] TaskScheduler::Schedule(CVDevTask*cv_task)

[0094] TaskScheduler::Run(CVDevTask*cv_task)

[0095] TaskScheduler::RunAppSyncTask(CVDevTask*cv_task)

[0096] TaskScheduler::RunDevSyncTask(CVDevTask*cv_task)

[0097] CVDevSession::RunDevSyncTask(CVDevTask*cv_task)

[0098] IDevSession(HelloSession)::ExecSyncTask(IDevTask*task)

[0099] IDevSession(HelloSession)::ExecuteOP(HelloTask*task)

[0100] IDevOP(HelloDevOP)::Run(IDevTask*task).

[0101] Figure 7 yes Figure 1 The flowchart of step S150 in the illustrated embodiment is shown in an exemplary embodiment. (See attached flowchart.) Figure 7 As shown, the process of stopping all equipment may include steps S710 to S730, which are described in detail below:

[0102] S710, Destroy the device task

[0103] session->DestroyTask(sync_task);

[0104] S720, release input / output media memory

[0105] session->FreeMem(input_image);

[0106] session->FreeMem(output_image);

[0107] S730, destroy session

[0108] dev->DestroySession(session);

[0109] S740, stop all devices

[0110] REGISTER_SYS_RELEASE_FUNCTION(MOD_MID_LEVEL,stop_all_device,register_device_stop_function).

[0111] Figure 8 This is a block diagram illustrating a cross-platform visual data processing apparatus as shown in an exemplary embodiment of this application. Figure 8 As shown, this exemplary cross-platform data processing apparatus includes:

[0112] Entry point acquisition module 801 is configured to acquire an entry point, wherein the entry point is the starting point of the program; device acquisition module 802 is configured to acquire a custom device by acquiring the custom device through the device manager; task creation module 803 is configured to create a device task based on the custom device; task execution module 804 is configured to execute the device task; and device stop module is configured to terminate the operation of all devices.

[0113] Figure 9 This is a schematic diagram illustrating the architecture of a cross-platform visual computing framework, as shown in an exemplary embodiment of this application. Figure 9 As shown, this exemplary cross-platform visual computing framework includes: a C++ interface layer 901, a front-end access layer 902, a core implementation layer 903, a device interface layer 904, and a device layer 905. The C++ interface layer 901 is used by developers to complete the development of the visual computing framework during the development process. The front-end access layer 902 is a concrete implementation class of the interface class. The CVFramework class provides the entry point for the entire framework, and all interfaces are obtained directly or indirectly from the CVFramework class. The core implementation layer 903 is the core layer of the visual computing framework, which includes the cross-platform visual data processing device of this application. The front-end access layer 902 implements the communication connection between the C++ interface layer 901 and the core implementation layer 903. The device layer 905 is a custom device, which includes various different devices. The device interface layer 904 implements the communication connection between the device layer 905 and the core implementation layer 903.

[0114] In an exemplary embodiment, the C++ interface layer 901 includes the following interface classes: ICVOP, IImage, IContext, ISession, ITask, and IDevice. ICVOP represents the CV operation to be performed and its related parameters; IImage is an abstraction of image data; IContext represents the runtime environment for executing CVOP, such as a list of available devices and their priorities; ISession represents a container of resources allocated by the system for CVOP execution; ITask records the state of a single CVOP execution; and IDevice is an abstraction of the underlying devices, primarily providing interfaces for querying / profiling and for external schedulers to implement the necessary functionality. The design process for each interface is detailed in Tables 1 to 4.

[0115] Table 1: CVOP Class Design

[0116] Main functions interface illustrate Name + Version Number op_name+op_version Register various types of parameters RegisterParamFloat Set various types of parameters SetParamFloat

[0117] Table 2: Iimage Class Design

[0118]

[0119] Table 3: Icontext Class Design

[0120]

[0121]

[0122] Table 4: Isession Class Design

[0123]

[0124]

[0125] In an exemplary embodiment, the core implementation layer 903 implements the core functions of the framework, including device scheduling, task scheduling, etc. The core implementation layer 903 includes DevManager, DeviceScheduler, TaskScheduler, ExecDevice, CVDevice, and PluginManager. CVDevice is the entity representative of the underlying device in the system, ExecDevice is used to indicate that a certain CVOP is bound to a certain CVDevice for execution, and DevManager, DeviceScheduler, and TaskScheduler assist the core implementation layer in completing core functions such as device scheduling and task scheduling.

[0126] In an exemplary embodiment, the device interface layer 904 defines the interfaces that the CACV device needs to implement. Within the framework, the operation describing the computation logic is called a CVOP, and the operation implementing the CVOP computation process within the device is called a DevOP. Each device needs to derive its own class from the following five classes. IDevBackend is the device abstract interface, providing the outermost interface for the device. IDevSession defines a container for resources allocated when the device executes an operation. IDevTask is used to store the state during an operation execution and serves as a container for temporarily allocated resources, containing input and output data. IDevOP is implemented when the device executes an operation. Since an operation may be called by multiple threads simultaneously, it is generally recommended that the operation be implemented stateless, with the relevant state stored in the Session and Task. ICVBuffer, derived from this class, is used to store device-private memory allocated by the device.

[0127] The cross-platform visual computing framework of this invention can realize automatic device scheduling: when the parameters of the CV operation exceed the limits of the hardware acceleration module, it automatically selects a CPU to execute; when the hardware acceleration module is overloaded, it automatically selects a CPU to execute. In addition to running on various x86 and Arm (Horizon Robotics, Nvidia, etc.) platforms, it also supports Linux / Windows OS to maximize support for the development of cross-platform applications.

[0128] To support the needs of specific scenarios, the framework also provides applications with the ability to specify computing modules and set the priority of CV operations: multiple CV operations scheduled to the same device will be executed in order of priority.

[0129] It should be noted that the cross-platform visual computing framework and the cross-platform visual data processing device and the cross-platform visual data processing method provided in the above embodiments belong to the same concept. The specific way each module performs its operation has been described in detail in the method embodiments and will not be repeated here. In practical applications, the cross-platform visual computing framework and cross-platform visual data processing device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0130] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the cross-platform visual data processing method provided in the above embodiments.

[0131] Figure 10 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0132] like Figure 10 As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0133] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. Drive 1210 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0134] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0135] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0137] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0138] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the cross-platform visual data processing method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0139] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the cross-platform visual data processing method provided in the various embodiments described above.

[0140] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A cross-platform visual data processing method, characterized in that, The method comprises the following steps: acquiring an entry, which is a starting point of data processing; acquiring a customized device; creating a device task based on the customized device; executing the device task; stopping all devices after the device task is executed; wherein the entry is an entry of a visual computing framework, the visual computing framework stores visual computing logic of corresponding devices, and corresponding computer vision operation devices can be acquired through a device manager, the visual computing framework uniformly manages different devices, and realizes unified execution, calling interface and usage convention across platforms; creating a device task based on the customized device comprises the following steps: acquiring visual computing logic of the customized device; creating a running environment for executing visual computing logic, framework operation and related parameters, a system-allocated resource container and context options based on the customized device and the visual computing logic, and setting customized parameters; creating a device task based on the running environment, framework operation and related parameters, the system-allocated resource container and context option parameters; executing the device task comprises the following steps: allocating input media memory based on the device task; allocating output media memory based on the device task; running the task based on the input media memory and the output media memory.

2. The cross-platform visual data processing method of claim 1, wherein, The method further comprises the following steps before acquiring the entry: customizing a device; customizing visual computing logic; registering the customized device and the visual computing logic in the computing framework; compiling the computing framework to generate a binary library to obtain a visual computing framework.

3. The cross-platform visual data processing method of claim 1, wherein, stopping all devices after the device task is executed comprises the following steps: destroying the device task; releasing input and output media memory; destroying a system-allocated resource container; stopping all devices.

4. A cross-platform visual data processing apparatus, characterized by comprising: The method comprises the following steps: an entry acquiring module configured to acquire an entry, which is a starting point of data processing; a device acquiring module configured to acquire a customized device; a task creating module configured to create a device task based on the customized device; a task executing module configured to execute the device task; a device stopping module configured to stop all devices; the entry is an entry of a visual computing framework, the visual computing framework stores visual computing logic of corresponding devices, and corresponding computer vision operation devices can be acquired through a device manager, the visual computing framework uniformly manages different devices, and realizes unified execution, calling interface and usage convention across platforms; creating a device task based on the customized device comprises the following steps: acquiring visual computing logic of the customized device; creating a running environment for executing visual computing logic, framework operation and related parameters, a system-allocated resource container and context options based on the customized device and the visual computing logic, and setting customized parameters; creating a device task based on the running environment, framework operation and related parameters, the system-allocated resource container and context option parameters; executing the device task comprises the following steps: allocating input media memory based on the device task; allocating output media memory based on the device task; running the task based on the input media memory and the output media memory.

5. A cross-platform visual computing framework, characterized by The cross-platform visual computing framework stores visual computing logic of multiple devices and uniformly manages the multiple devices, and realizes unified execution, calling interface and usage convention across platforms, and the cross-platform visual computing framework comprises a C++ interface layer, a front-end access layer, a core implementation layer, a device interface layer and a device layer, the C++ interface layer is in communication connection with the core implementation layer through the front-end access layer, the device layer is in communication connection with the core implementation layer through the device interface layer, and the core implementation layer comprises the cross-platform visual data processing device in claim 4; and the cross-platform visual computing framework is used for executing the cross-platform visual data processing method in claim 1.

6. The cross-platform visual computing framework of claim 5, wherein, The C++ interface layer comprises ICVOP, IImage, IContext, ISession, ITask and IDevice interface classes, the ICVOP represents a CV operation to be performed and related parameters, the IImage is an abstraction of image data, the IContext represents a running environment for performing the CVOP, the ISession represents a container of resources allocated by the system for CVOP execution, the ITask is used for representing the state of one-time CVOP execution, and the IDevice is an abstraction of a bottom device, and is used for querying / profiling and an interface required by an external scheduler.

7. An electronic device, comprising: The electronic device comprises: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the electronic device is caused to implement the cross-platform visual data processing method in any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that, A computer program is stored thereon, when the computer program is executed by a processor of a computer, the computer is caused to execute the cross-platform visual data processing method in any one of claims 1 to 3.

Images