Image data operation method and device, electronic equipment and computer program product
By introducing a matrix processor into the target device and utilizing the collaborative operation of the central processing unit and the matrix processor, the problem of insufficient computing power of the central processing unit is solved, and the efficiency of image data processing is improved.
Patent Information
- Application Number
- CN202511284611.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-09
- Publication Date
- 2026-01-13
AI Technical Summary
Existing target devices suffer from low image data processing efficiency due to low central processing unit computing power, which fails to meet user needs.
By introducing a matrix processor into the target device, the central processing unit acquires image data and sends the matrix data to be processed to the matrix processor for matrix operations. The matrix processor then returns the results to the central processing unit to determine the final image processing result.
It improves the efficiency of the target device in processing image data. By using a matrix processor to assist the central processing unit in image data processing, it enhances the overall processing capability.
Smart Images
Figure CN121329752A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image processing technology, and in particular relates to a method, apparatus, electronic device and computer program product for processing image data. Background Technology
[0002] Image data processing technology is being used more and more widely in some target devices (such as embedded devices, including but not limited to smart cameras, drones and robots), which usually have high requirements for image data processing efficiency.
[0003] Currently, these target devices typically process image data using their central processing unit (CPU). However, the computing power of the CPU in these target devices is usually low, resulting in low efficiency in processing image data and thus failing to meet user needs. Summary of the Invention
[0004] In view of this, embodiments of this application provide a method, apparatus, electronic device, and computer program product for processing image data, in order to solve the technical problem of low efficiency in processing image data in existing target devices.
[0005] In a first aspect, embodiments of this application provide a method for processing image data, applied to a target device, the target device including a central processing unit and a matrix processor; the method includes:
[0006] The central processing unit acquires the target image data to be processed and sends the target matrix data to be processed in the target image data to the matrix processor.
[0007] The matrix processor performs matrix operations on the target matrix data and sends the matrix operation results to the central processing unit.
[0008] The central processing unit determines the image processing result of the target image data based on the matrix operation result.
[0009] Optionally, before acquiring the target image data to be processed through the central processing unit and sending the target matrix data to be processed in the target image data to the matrix processor, the method further includes:
[0010] The central processing unit loads the service firmware into the matrix processor. The service firmware includes basic service firmware and acceleration operator firmware.
[0011] After the basic service firmware is loaded, the matrix processor is initialized through the basic service firmware, and the service environment required for matrix operations is set up.
[0012] After the acceleration operator firmware is loaded, the matrix processor is given the ability to perform matrix operations through the acceleration operator firmware.
[0013] Optionally, the target device further includes memory; the initialization of the matrix processor via the basic service firmware includes:
[0014] The matrix processor sends a dedicated memory pool creation instruction to the memory through the basic service firmware, instructing the memory to create a matrix operation operator memory pool, a matrix operation parameter memory pool, and a matrix operation data memory pool. The matrix operation operator memory pool is used to store the target operators used in the matrix operation process, the matrix operation parameter memory pool is used to store the target parameters of the matrix operation, and the matrix operation data memory pool is used to store the target data used in the matrix operation process. The target operators, the target parameters, and the target data are all sent to the memory by the central processing unit.
[0015] Optionally, sending the target matrix data to be processed in the target image data to the matrix processor includes:
[0016] Based on the target matrix data, a matrix operation task with task information is generated, and the matrix operation task is sent to the matrix processor to instruct the matrix processor to retrieve the target matrix data from the memory based on the matrix operation task; the task information includes any one or more of the following: task number, information on the operators required by the task, task parameter information, and task memory information.
[0017] Optionally, the target device further includes a direct memory accessor; the matrix operation on the target matrix data via the matrix processor includes:
[0018] The matrix processor, based on the task information in the matrix operation task, uses the direct memory accessor to retrieve the target operator, the target parameter, and the target data corresponding to the target matrix data from the memory, and performs matrix operations on the target matrix data based on the target operator, the target parameter, and the target data.
[0019] Optionally, sending the matrix operation result to the central processing unit includes:
[0020] The matrix operation result is sent to the memory via the direct memory accessor;
[0021] The matrix operation result is obtained from the memory by the central processing unit.
[0022] Optionally, before performing matrix operations on the target matrix data through the matrix processor, the method further includes:
[0023] The central processing unit sends the target operator corresponding to the target matrix data to the matrix operation operator memory pool, sends the target parameter corresponding to the target matrix data to the matrix operation parameter memory pool, and sends the target data corresponding to the target matrix data to the matrix operation data memory pool.
[0024] Secondly, embodiments of this application provide an image data processing device applied to a target device, the target device including a central processing unit and a matrix processor; the device includes:
[0025] A matrix data sending unit is used to acquire target image data to be processed through the central processing unit, and send the target matrix data to be processed in the target image data to the matrix processor.
[0026] A matrix data processing unit is used to perform matrix operations on the target matrix data through the matrix processor, and send the matrix operation results to the central processing unit.
[0027] The image processing result determination unit is used to determine the image processing result of the target image data based on the matrix processing result through the central processing unit.
[0028] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the image data processing method as described in any of the first aspects above.
[0029] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the image data processing method as described in any of the first aspects above.
[0030] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the steps of the image data processing method as described in any of the first aspects above.
[0031] The image data processing method, apparatus, electronic device, and computer program product provided in this application have the following beneficial effects:
[0032] The image data processing method provided in this application can be applied to a target device, wherein the target device includes a central processing unit (CPU) and a matrix processor. The image data processing method may include: acquiring target image data to be processed through the CPU, and sending target matrix data to be processed in the target image data to the matrix processor; performing matrix operations on the target matrix data through the matrix processor, and sending the matrix operation result to the CPU; and determining the image processing result of the target image data through the CPU based on the matrix operation result. This method allows the matrix processor in the target device to assist the CPU in processing the target matrix data in the target image data, thereby improving the efficiency of the target device in processing image data. Attached Figure Description
[0033] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0034] Figure 1 A flowchart illustrating the implementation of the image data processing method provided in this application embodiment;
[0035] Figure 2 This is a schematic diagram of the structure of a target device provided in an embodiment of this application;
[0036] Figure 3 This is a schematic diagram of the structure of an image data processing device provided in an embodiment of this application;
[0037] Figure 4 This is a schematic diagram of the structure of a target device provided in an embodiment of this application. Detailed Implementation
[0038] It should be noted that the terminology used in the embodiments of this application is only for explaining specific embodiments of this application and is not intended to limit this application. In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, "at least one" or "one or more" means one, two or more. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.
[0039] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0040] The execution entity of the image data processing method provided in this application embodiment can be a target device including a central processing unit and a matrix processor. For example, the target device can be an embedded device, which may include, but is not limited to, smart cameras, drones, and robots.
[0041] The image data processing method provided in this application can be applied to any scenario where a target device needs to perform image data processing. For example, when a robot needs to perform image data processing, it can execute the various steps of the image data processing method provided in this application, thereby improving the robot's efficiency in processing image data.
[0042] Please see Figure 1 , Figure 1 This is a flowchart illustrating the implementation of an image data processing method provided in an embodiment of this application. This image data processing method can be applied to a target device, which includes a central processing unit and a matrix processor. The image data processing method may include steps S101 to S103, detailed below:
[0043] In S101, the target image data to be processed is obtained through the central processing unit, and the target matrix data to be processed in the target image data is sent to the matrix processor.
[0044] In this embodiment of the application, before executing S101, the target device can first enable the matrix processor to perform matrix operations in the following way:
[0045] The central processing unit loads service firmware into the matrix processor. The service firmware includes basic service firmware and acceleration operator firmware. After the basic service firmware is loaded, the matrix processor is initialized and the service environment required for matrix operations is set up. After the acceleration operator firmware is loaded, the matrix processor is enabled to perform matrix operations.
[0046] In practical applications, the central processing unit (CPU) can load preset service firmware from its file system and submit it to the matrix processor as a BOOT task. The CPU automatically initiates the preset service firmware loading operation when the target device is first turned on, and provides an application layer interface for applications to freely load specified service firmware, enabling flexible customization of acceleration algorithms. The application program can be a program stored in the CPU used to control both the CPU and the matrix processor.
[0047] The basic service firmware is mainly used to initialize the microcontroller in the matrix processor and to build the service environment required for matrix operations. Building the service environment required for matrix operations can be used to establish a communication link between the matrix processor and the central processing unit.
[0048] Specifically, the basic service firmware can be used to initialize the matrix processor's stack, initialize the matrix processor's interrupts, register the matrix processor's interrupt handlers and exception handlers, and enable the matrix processor to enter a communication state to wait for task interrupts sent by the central processing unit.
[0049] The acceleration operator firmware can include various types of matrix multiplication acceleration operators. For example, matrix multiplication acceleration operators can include, but are not limited to, single-precision floating-point operators, double-precision floating-point operators, single-precision complex operators, double-precision complex operators, and matrix transpose / non-transpose operators.
[0050] In one possible implementation, the target device may also include memory, and the matrix processor may be initialized via the underlying service firmware in the following ways:
[0051] The matrix processor sends dedicated memory pool creation instructions to memory via the basic service firmware, instructing memory to create a matrix operation operator memory pool, a matrix operation parameter memory pool, and a matrix operation data memory pool. The matrix operation operator memory pool stores the target operators used in matrix operations, the matrix operation parameter memory pool stores the target parameters, and the matrix operation data memory pool stores the target data used in matrix operations. In practical applications, the target operators, target parameters, and target data can all be sent to memory by the central processing unit (CPU). Based on this, the matrix processor can retrieve the target operators, target parameters, and target data from the corresponding sub-memory pools in memory.
[0052] Optionally, to further improve the efficiency of the target device in processing image data, memory can create a matrix operation operator memory pool, a matrix operation parameter memory pool, and a matrix operation data memory pool through a memory mapping (mmap) mechanism.
[0053] In practical applications, the initialization of the matrix processor can include, but is not limited to, clock enabling, reset release, and power-on. After completing the initialization steps, the matrix processor can have functions including, but not limited to, service firmware loading, task-specific memory pool management, task management, and task exception and recovery management. It can also provide the target device's application layer with functional interfaces such as requesting / releasing task memory, requesting task IDs, submitting tasks, obtaining task results, and loading specified firmware.
[0054] In addition, the matrix processor can store a set of preset custom instructions for matrix operations, enabling a single instruction to perform multiple floating-point operations, thereby further improving the efficiency of matrix operations in image data processing.
[0055] In practical applications, a matrix processor can include modules such as a microprocessor, task scheduler, direct memory accessor, local random access memory, and matrix arithmetic unit. The task scheduler is primarily responsible for task interaction with the central processing unit (CPU) and is the core object for implementing heterogeneous interaction in software. Internally, the task scheduler can include multiple registers related to task submission and completion information to store the information sent by the CPU. Furthermore, the matrix processor can use two FIFO (First-In-First-Out) data buffers to cache task submission and completion information respectively, facilitating asynchronous task processing.
[0056] The central processing unit (CPU) manages task submission and completion information in the matrix processor using the following methods: The CPU first instructs the matrix processor to build and maintain two task queues: one for submitted tasks and one for completed tasks. When an application in the CPU submits a task, the CPU sends key task information to the registers related to the task submission information in the matrix processor's task scheduler, instructing the matrix processor's matrix arithmetic unit to process the task. The CPU then sends the task to the submitted task queue. After the matrix processor completes its matrix processing task, it sends task completion information to the CPU via a task interrupt function. The CPU matches the task in the submitted task queue, updates the completion information based on the matched task, and moves the task from the submitted task queue to the completed task queue. This method ensures that the task submission and result retrieval processes do not interfere with each other in a multi-threaded scenario, achieving efficient task management. Furthermore, when the matrix processor encounters an exception and needs to be reset, an efficient reset function can be implemented through the submitted task queue.
[0057] The above methods enable the matrix processor to perform matrix operations. Once the matrix processor is capable of performing matrix operations, the central processing unit can acquire the target image data to be processed, determine the target matrix data to be processed in the target image data, and send the target matrix data to be processed in the target image data to the matrix processor.
[0058] Specifically, the central processing unit can send the target matrix data to be processed from the target image data to the matrix processor in the following ways:
[0059] Based on the target matrix data, a matrix operation task with task information is generated and sent to the matrix processor to instruct the matrix processor to retrieve the target matrix data from memory based on the matrix operation task. The task information includes any one or more of the following: task number, information on the operators required by the task, task parameter information, and task memory information.
[0060] The task number can be used to distinguish different tasks within the task's validity period.
[0061] The information of the operators required by the task may include the operator source address and the operator destination address. The operator source address is needed when the matrix processor uses dynamic operators, and the operator destination address is the operator address inherent in the matrix processor's local random access memory or the dynamic operator storage address.
[0062] The task parameter information may include the source address of the parameter list used by the operator and the destination address of the parameter list used by the operator. The source address of the parameter list used by the operator can be the address of the operator stored in memory, and the destination address of the parameter list used by the operator can be the address of the parameter list stored in local random access memory.
[0063] The task memory information may include the dedicated memory block number used, the size of each matrix block in the memory block, the input and output attributes of each matrix in the memory block, and the attributes of the memory block (such as whether the mmap mechanism is used).
[0064] In addition, the target operator corresponding to the target matrix data can be sent to the matrix operation operator memory pool by the central processing unit, the target parameter corresponding to the target matrix data can be sent to the matrix operation parameter memory pool, and the target data corresponding to the target matrix data can be sent to the matrix operation data memory pool.
[0065] Based on this, after receiving a matrix operation task with task information sent by the central processing unit, the matrix processor can retrieve the target matrix data from memory according to the matrix operation task. Specifically, the matrix processor can retrieve the target operator corresponding to the target matrix data from the matrix operation operator memory pool, retrieve the target parameters corresponding to the target matrix data from the matrix operation parameter memory pool, and retrieve the target data corresponding to the target matrix data from the matrix operation data memory pool.
[0066] In addition, before sending the matrix operation task to the matrix processor, the central processing unit can send a task interrupt function to the matrix processor. After receiving the task interrupt function, the matrix processor can receive the matrix operation task sent by the central processing unit, and then obtain the target matrix data from memory according to the matrix operation task and in the manner described above.
[0067] Optionally, before the matrix processor receives the task interrupt function, the matrix processor can enter a communication low-power mode, thereby reducing the power consumption of the target device.
[0068] In S102, the matrix processor performs matrix operations on the target matrix data and sends the matrix operation results to the central processing unit.
[0069] In this embodiment, the target device may further include a direct memory accessor, which can be integrated into the matrix processor as one of its modules. Based on this, matrix operations on the target matrix data can be performed by the matrix processor in the following way:
[0070] The matrix processor retrieves the target operator, target parameters, and target data from memory using the direct memory accessor based on the task information in the matrix operation task. Then, it performs matrix operations on the target matrix data based on the target operator, target parameters, and target data.
[0071] In practical applications, when performing matrix operations on target matrix data using a matrix processor, matrix multiplication operations can be integrated into the matrix arithmetic unit of the matrix processor using preset instructions. The preset instructions can then be called within the operator to accelerate the calculation of the target matrix data in the matrix processor, thereby speeding up the various matrix multiplication operations.
[0072] Furthermore, during matrix operations on the target matrix data corresponding to a certain frame of image, the matrix processor can use multiple different operators to perform matrix operations together, thereby realizing the operation on the image data of that frame of image.
[0073] After performing matrix operations on the target matrix data, if the matrix processor determines that the matrix operation process is normal and the matrix operation result is normal, then the matrix processor can send the matrix operation result to the central processing unit.
[0074] Specifically, the matrix processor can send the matrix operation results to the central processing unit in the following ways: send the matrix operation results to memory through the direct memory accessor, and retrieve the matrix operation results from memory through the central processing unit.
[0075] Furthermore, if the matrix processor determines that the matrix operation process is normal and the matrix operation result is normal, the matrix processor can send the task completion information to the central processing unit in the form of an interrupt task function to trigger the central processing unit's interrupt handling, thereby instructing the central processing unit to retrieve the matrix operation result from memory.
[0076] After performing matrix operations on the target matrix data, if the matrix processor determines that the matrix operation process is abnormal or the matrix operation result is abnormal, the matrix processor can send the task exception information to the central processing unit in the form of an interrupt task function to trigger the central processing unit's interrupt handling, thereby instructing the central processing unit to handle the task exception information.
[0077] In S103, the central processing unit determines the image processing result of the target image data based on the matrix operation result.
[0078] In this embodiment, after obtaining the matrix operation result, the central processing unit (CPU) can determine the image operation result of the target image data based on the matrix operation result. The specific method by which the CPU determines the image operation result of the target image data based on the matrix operation result can be set according to actual needs and is not limited here.
[0079] As can be seen from the above, the image data processing method provided in this application embodiment can be applied to a target device. The target device includes a central processing unit (CPU) and a matrix processor. The image data processing method may include: acquiring target image data to be processed through the CPU, and sending the target matrix data to be processed from the target image data to the matrix processor; performing matrix operations on the target matrix data through the matrix processor, and sending the matrix operation result to the CPU; and determining the image processing result of the target image data based on the matrix operation result through the CPU. This method allows the matrix processor in the target device to assist the CPU in processing the target matrix data in the target image data, thereby improving the efficiency of the target device in processing image data.
[0080] Based on the image data processing method provided above, this application embodiment provides a schematic diagram of the structure of a target device for implementing the image data processing method. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of a target device provided in an embodiment of this application. Figure 2 As shown, the target device may include a central processing unit, a matrix processor, a bus, and memory. For example, the bus may use an on-chip bus protocol, and the memory may be a double data rate synchronous dynamic random access memory.
[0081] Based on the image data processing method provided in the above embodiments, this application further provides an image data processing apparatus for implementing the above method embodiments. This image data processing apparatus is applied to a target device, wherein the target device includes a central processing unit and a matrix processor. Please refer to... Figure 3 , Figure 3 This is a schematic diagram of the structure of an image data processing device provided in an embodiment of this application. Figure 3 As shown, the image data processing device 30 may include a matrix data transmitting unit 31, a matrix data processing unit 32, and an image processing result determining unit 33. Wherein:
[0082] The matrix data sending unit 31 is used to acquire the target image data to be processed through the central processing unit, and send the target matrix data to be processed in the target image data to the matrix processor.
[0083] The matrix data processing unit 32 is used to perform matrix operations on the target matrix data through the matrix processor, and send the matrix operation results to the central processing unit.
[0084] The image processing result determination unit 33 is used to determine the image processing result of the target image data based on the matrix processing result through the central processing unit.
[0085] Optionally, the image data processing unit 30 may further include a matrix processor debugging unit. Wherein:
[0086] The matrix processor debugging unit is specifically used for:
[0087] The central processing unit loads the service firmware into the matrix processor. The service firmware includes basic service firmware and acceleration operator firmware.
[0088] After the basic service firmware is loaded, the matrix processor is initialized through the basic service firmware, and the service environment required for matrix operations is set up.
[0089] After the acceleration operator firmware is loaded, the matrix processor is given the ability to perform matrix operations through the acceleration operator firmware.
[0090] Optionally, the target device may also include memory; the matrix processor debugging unit is specifically used for:
[0091] The matrix processor sends a dedicated memory pool creation instruction to the memory through the basic service firmware, instructing the memory to create a matrix operation operator memory pool, a matrix operation parameter memory pool, and a matrix operation data memory pool. The matrix operation operator memory pool is used to store the target operators used in the matrix operation process, the matrix operation parameter memory pool is used to store the target parameters of the matrix operation, and the matrix operation data memory pool is used to store the target data used in the matrix operation process. The target operators, the target parameters, and the target data are all sent to the memory by the central processing unit.
[0092] Optionally, the matrix data transmission unit 31 is specifically used for:
[0093] Based on the target matrix data, a matrix operation task with task information is generated, and the matrix operation task is sent to the matrix processor to instruct the matrix processor to retrieve the target matrix data from the memory based on the matrix operation task; the task information includes any one or more of the following: task number, information on the operators required by the task, task parameter information, and task memory information.
[0094] Optionally, the target device also includes a direct memory accessor; the matrix data processing unit 32 is specifically used for:
[0095] The matrix processor, based on the task information in the matrix operation task, uses the direct memory accessor to retrieve the target operator, the target parameter, and the target data corresponding to the target matrix data from the memory, and performs matrix operations on the target matrix data based on the target operator, the target parameter, and the target data.
[0096] Optionally, the matrix data operation unit 32 is also specifically used for:
[0097] The matrix operation result is sent to the memory via the direct memory accessor;
[0098] The matrix operation result is obtained from the memory by the central processing unit.
[0099] Optionally, the image data processing unit 30 may further include a memory data transmission unit, wherein:
[0100] The memory data sending unit is specifically used for:
[0101] The central processing unit sends the target operator corresponding to the target matrix data to the matrix operation operator memory pool, sends the target parameter corresponding to the target matrix data to the matrix operation parameter memory pool, and sends the target data corresponding to the target matrix data to the matrix operation data memory pool.
[0102] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of this application. Their specific functions and technical effects can be referred to the method embodiments section, and will not be repeated here.
[0103] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a target device provided in an embodiment of this application. Figure 4 As shown, the target device 4 provided in this embodiment may include: a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. For example, a program corresponding to an image data processing method. When the processor 40 executes the computer program 42, it implements the steps described above in the embodiment of the image data processing method, for example... Figure 1 S101 to S103 are shown. Alternatively, when the processor 40 executes the computer program 42, it implements the functions of each module / unit in the embodiment corresponding to the image data processing device described above, for example... Figure 3 The functions of units 31 to 33 shown.
[0104] For example, computer program 42 can be divided into one or more modules / units, one or more of which are stored in memory 41 and executed by processor 40 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 42 in target device 4. For example, computer program 42 can be divided into matrix data transmission unit 31, matrix data operation unit 32, and image operation result determination unit 33. For the specific functions of each unit, please refer to... Figure 3 The relevant descriptions in the corresponding embodiments are not repeated here.
[0105] Those skilled in the art will understand that Figure 4 This is merely an example of target device 4 and does not constitute a limitation on target device 4. It may include more or fewer components than shown, or combine certain components, or use different components.
[0106] The processor 40 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0107] The memory 41 can be an internal storage unit of the target device 4, such as a hard disk or RAM of the target device 4. The memory 41 can also be an external storage device of the target device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card equipped on the target device 4. Furthermore, the memory 41 can include both internal storage units and external storage devices of the target device 4. The memory 41 is used to store computer programs and other programs and data required by the target device. The memory 41 can also be used to temporarily store data that has been output or will be output.
[0108] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units is merely an example. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the image data processing device can be divided into different functional units to complete all or part of the functions described above. The functional units in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0109] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.
[0110] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.
[0111] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.
[0112] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0113] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for processing image data, characterized in that, Applied to a target device, the target device including a central processing unit and a matrix processor; the method includes: The central processing unit acquires the target image data to be processed and sends the target matrix data to be processed in the target image data to the matrix processor. The matrix processor performs matrix operations on the target matrix data and sends the matrix operation results to the central processing unit. The central processing unit determines the image processing result of the target image data based on the matrix operation result.
2. The method according to claim 1, characterized in that, Before acquiring the target image data to be processed through the central processing unit and sending the target matrix data to be processed from the target image data to the matrix processor, the method further includes: The central processing unit loads the service firmware into the matrix processor. The service firmware includes basic service firmware and acceleration operator firmware. After the basic service firmware is loaded, the matrix processor is initialized through the basic service firmware, and the service environment required for matrix operations is set up. After the acceleration operator firmware is loaded, the matrix processor is given the ability to perform matrix operations through the acceleration operator firmware.
3. The method according to claim 2, characterized in that, The target device also includes memory; the initialization of the matrix processor via the basic service firmware includes: The matrix processor sends a dedicated memory pool creation instruction to the memory through the basic service firmware, instructing the memory to create a matrix operation operator memory pool, a matrix operation parameter memory pool, and a matrix operation data memory pool. The matrix operation operator memory pool is used to store the target operators used in the matrix operation process, the matrix operation parameter memory pool is used to store the target parameters of the matrix operation, and the matrix operation data memory pool is used to store the target data used in the matrix operation process. The target operators, the target parameters, and the target data are all sent to the memory by the central processing unit.
4. The method according to claim 3, characterized in that, Sending the target matrix data to be processed in the target image data to the matrix processor includes: Based on the target matrix data, a matrix operation task with task information is generated, and the matrix operation task is sent to the matrix processor to instruct the matrix processor to retrieve the target matrix data from the memory based on the matrix operation task; the task information includes any one or more of the following: task number, information on the operators required by the task, task parameter information, and task memory information.
5. The method according to claim 4, characterized in that, The target device further includes a direct memory accessor; the matrix operation on the target matrix data via the matrix processor includes: The matrix processor, based on the task information in the matrix operation task, uses the direct memory accessor to retrieve the target operator, the target parameter, and the target data corresponding to the target matrix data from the memory, and performs matrix operations on the target matrix data based on the target operator, the target parameter, and the target data.
6. The method according to claim 5, characterized in that, Sending the matrix operation result to the central processing unit includes: The matrix operation result is sent to the memory via the direct memory accessor; The matrix operation result is obtained from the memory by the central processing unit.
7. The method according to any one of claims 3 to 6, characterized in that, Before performing matrix operations on the target matrix data through the matrix processor, the method further includes: The central processing unit sends the target operator corresponding to the target matrix data to the matrix operation operator memory pool, sends the target parameter corresponding to the target matrix data to the matrix operation parameter memory pool, and sends the target data corresponding to the target matrix data to the matrix operation data memory pool.
8. A processing device for image data, characterized in that, Applied to a target device, the target device including a central processing unit and a matrix processor; the device includes: A matrix data sending unit is used to acquire target image data to be processed through the central processing unit, and send the target matrix data to be processed in the target image data to the matrix processor. A matrix data processing unit is used to perform matrix operations on the target matrix data through the matrix processor, and send the matrix operation results to the central processing unit. The image processing result determination unit is used to determine the image processing result of the target image data based on the matrix processing result through the central processing unit.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the image data processing method as described in any one of claims 1 to 7.
10. A computer program product, characterized in that, When the computer program product is executed by a processor, it implements each step of the image data processing method as described in any one of claims 1 to 7.