Image data processing device and method, equipment and storage medium

Abstract

The invention relates to the technical field of image processing, and provides an image data processing device and method, equipment and a storage medium, and the device comprises a first data path and a second data path; the first data path comprises a first reading module, a first bit width conversion module and an address module; the second data path comprises a second reading module, an address module and a writing module; the first reading module is used for reading the address block from the first memory in the first operation mode; the first bit width conversion module is used for segmenting the address blocks to obtain a plurality of addresses; the address module is used for generating an absolute address corresponding to each address based on each address and a preset read parameter; the second reading module is used for reading the to-be-processed image data of each absolute address from the second memory; and the writing module is used for writing the plurality of pieces of to-be-processed image data into the first preset address block of the second memory. According to the technical scheme, the data bit width of the data path when the to-be-processed image data is stored in the OCM is increased.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image data processing apparatus, method, device, and storage medium. Background Technology

[0002] Currently, in image data processing, address modules are often used for non-contiguous or irregular memory access. Address modules include gathering and scattering modules. The gathering module can "converge" image data from multiple non-contiguous addresses into a single vector, while the scattering module can write a vector to multiple non-contiguous addresses.

[0003] In related technologies, the address module is mainly used in the following two scenarios: First, image data is stored in Double Data Rate Synchronous Dynamic Random Memory (DDR), and the address is stored in On-Chip Memory (OCM); Second, image data is stored in OCM, and the address is stored in DDR / OCM.

[0004] However, the two scenarios mentioned above need to share a data path, which causes the data width of the second scenario to be reduced from OCM to DDR, thus reducing the output efficiency of the second scenario. Summary of the Invention

[0005] This application provides an image data processing apparatus, method, device, and storage medium. By configuring the data path according to the operating mode, the data path's bit width is maximized, thereby increasing the data bit width of the data path when the image data to be processed is stored in the OCM, and consequently improving the apparatus's output efficiency. The technical solution is as follows: According to a first aspect of the embodiments of this application, an image data processing apparatus is provided, comprising: First data path and second data path; The first data path includes a first read module, a first bit width conversion module, and an address module; The second data path includes a second read module, the address module, and a write module; The first read module is used to read an address block from a first memory in a first operating mode; the first operating mode indicates that the address block is stored in the first memory or the second memory, and the image data to be processed is stored in the second memory; the data bit width of the second memory is greater than the data bit width of the first memory; The first bit-width conversion module is used to segment the address block to obtain multiple addresses; The address module is used to generate an absolute address corresponding to each address based on each address and a preset read parameter; The second read module is used to read the image data to be processed from each of the absolute addresses in the second memory; The write module is used to write multiple images to be processed into a first preset address block of the second memory.

[0006] In some embodiments, the address module includes an aggregation module and a distribution module; the aggregation module and the distribution module do not operate simultaneously. The aggregation module is used to generate the absolute address corresponding to each address based on each address and the preset read parameters.

[0007] In some embodiments, the first read module is further configured to read an address block from the first memory; The first bit-width conversion module is also used to segment the address block to obtain multiple addresses; The distributed module is also used to generate an absolute address corresponding to each address based on each address and a preset write parameter; The second read module is further configured to read multiple image data to be processed from the second memory at a second preset address block; The write module is further configured to write each of the image data to be processed to the corresponding absolute address in the second memory.

[0008] In some embodiments, the first data path further includes a first selector; The second data path also includes a second selector, the third selector, and the fourth selector; The first read module is also used to transmit the address block to the first selector; The first selector is used to transmit the address block to the first bit-width conversion module; The second reading module is also used to transmit the image data to be processed to the second selector; The second selector is used to transmit the image data to be processed to the aggregation module; The aggregation module is also used to transmit the image data to be processed to the third selector; The third selector is used to transmit the image data to be processed to the fourth selector; The fourth selector is used to transmit the image data to be processed to the write module.

[0009] In some embodiments, the apparatus further includes a third data path and a fourth data path; The third data path includes the second read module, the first bit-width conversion module, and the aggregation module; The fourth data path includes the first read module, the second bit-width conversion module, the aggregation module, and the write module; The second read module is further configured to read the address block from the second memory in a second operating mode; the second operating mode indicates that the address block is stored in the second memory and the image data to be processed is stored in the first memory; The first bit-width conversion module is used to segment the address block to obtain multiple addresses; The aggregation module is used to generate the absolute address corresponding to each address based on each address and the preset read parameters; The first read module is used to read the image data to be processed from each of the absolute addresses in the first memory; The second bit-width conversion module is used to combine multiple image data to be processed to obtain combined image data; The write module is used to write the combined image data into a first preset address block of the second memory.

[0010] In some embodiments, the third data path further includes the first selector; The fourth data path further includes the second selector, the third selector, and the fourth selector; The second read module is also used to transmit the address block to the first selector; The first selector is also used to transmit the address block to the first bit-width conversion module; The second bit-width conversion module is also used to transmit the combined image data to the second selector; The second selector is also used to transmit the combined image data to the aggregation module; The aggregation module is also used to transmit the combined image data to the third selector; The third selector is also used to transmit the combined image data to the fourth selector; The fourth selector is also used to transmit the combined image data to the write module.

[0011] In some embodiments, the apparatus further includes a fifth data path; The fifth data path includes the first read module, the second bit-width conversion module, and the write module; The first read module is used to read multiple images to be processed from the second preset address block in the third operating mode; the third operating mode indicates that the images to be processed are stored in the first memory or the second memory; The second bit-width conversion module is used to combine multiple image data to be processed to obtain the combined image data; The write module is used to write the combined image data into the first preset address block of the second memory.

[0012] In some embodiments, the fifth data path further includes the third selector and the fourth selector; The second bit-width conversion module is also used to transmit the combined image data to the third selector; The third selector is also used to transmit the combined image data to the fourth selector; The fourth selector is also used to transmit the combined image data to the write module.

[0013] In some embodiments, the apparatus further includes a sixth data path; The sixth data path includes the second read module and the write module; The second read module is used to read multiple images to be processed from the second preset address block in the fourth operating mode; the fourth operating mode indicates that the images to be processed are stored in the second memory. The write module is used to write multiple images to be processed into the first preset address block of the second memory.

[0014] In some embodiments, the sixth data path further includes the fourth selector; The second reading module is also used to transmit multiple image data to be processed to the fourth selector; The fourth selector is also used to transmit multiple images to be processed to the write module.

[0015] In some embodiments, the first memory is DDR and the second memory is OCM.

[0016] According to a second aspect of the embodiments of this application, an image data processing method is provided, comprising: reading an address block from a first memory in a first operating mode; the first operating mode indicating that the address block is stored in the first memory or a second memory, and the image data to be processed is stored in the second memory; the data bit width of the second memory is greater than the data bit width of the first memory; The address block is divided to obtain multiple addresses; Based on each address and preset read parameters, generate the absolute address corresponding to each address; Read the image data to be processed from each of the absolute addresses in the second memory; The multiple images to be processed are written into the first preset address block of the second memory.

[0017] According to a third aspect of the present application, a computer device is provided, the computer device including a processor and a memory, the memory being used to store at least one program, the at least one program being loaded by the processor and executed by the image data processing method.

[0018] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein at least one program is stored in the computer-readable storage medium, the at least one program being loaded and executed by a processor to implement the image data processing method described herein.

[0019] In this embodiment, the device includes a first data path and a second data path. The first data path includes a first read module, a first bit width conversion module, and an address module. The second data path includes a second read module, an address module, and a write module. In a first operating mode, the first read module reads an address block from a first memory. The data bit width of the second memory is greater than that of the first memory. The first bit width conversion module divides the address block to obtain multiple addresses. The address module generates an absolute address corresponding to each address based on each address and preset read parameters. The second read module reads the image data to be processed from each absolute address in the second memory. The write module writes the multiple image data to be processed into a first preset address block in the second memory. In the above technical solution, the first read module and the second read module do not need to share a data path, ensuring that the data bit width of the second data path is maximized, improving the data bit width of the second read module and the address module, thereby improving the output efficiency of the second read module and the address module, and thus improving the output efficiency of the device. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an implementation environment provided according to an embodiment of this application; Figure 2This is a schematic diagram of the structure of the first image data processing apparatus provided according to the embodiments of this application; Figure 3 This is a schematic diagram of the structure of a second image data processing apparatus provided according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a third image data processing apparatus provided according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a fifth data path according to an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a sixth data path according to an embodiment of this application; Figure 7 This is a flowchart illustrating an image data processing method according to an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a terminal according to an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a server according to an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0024] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items that have essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or execution order. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms.

[0025] These terms are simply used to distinguish one element from another. For example, without departing from the scope of various examples, the first action can be called the second action, and similarly, the second action can be called the first action. Both the first and second actions can be actions, and in some cases, they can be separate and distinct actions.

[0026] "At least one" refers to one or more actions. For example, at least one action can be one action, two actions, three actions, or any integer number of actions greater than or equal to one. "Multiple" refers to two or more actions. For example, multiple actions can be two actions, three actions, or any integer number of actions greater than or equal to two.

[0027] Figure 1 This is a schematic diagram of an implementation environment provided according to an embodiment of this application. The implementation environment may include a terminal 101 and a server 102.

[0028] The terminal 101 is equipped with a Neural Processing Unit (NPU) and a DDR. The NPU is equipped with an OCM and an image data processing device. The image data processing device includes a first read module, a second read module, a first bit width conversion module, a second bit width conversion module, and an address module.

[0029] For example, terminal 101 can be a wearable device, personal computer, laptop computer, tablet computer, smart TV, and vehicle terminal, etc.

[0030] Server 102 can be a single server, a server cluster consisting of multiple servers, or a cloud processing center.

[0031] Terminal 101 is connected to server 102 via wired or wireless network.

[0032] In some embodiments, the wireless or wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but can be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats, including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies may be used to replace or supplement the aforementioned data communication technologies.

[0033] In related technologies, address modules are commonly used in Single Instruction, Multiple Data (SIMD) instruction sets, Graphics Processing Unit (GPU) programming, and sparse data processing. They are primarily applied in two scenarios: first, image data is stored in DDR and the address is stored in on-chip memory (OCM); second, image data is stored in OCM and the address is stored in DDR / OCM.

[0034] Because DDR has a small data bit width and OCM has a large data bit width, and typically one Direct Memory Access (DMA) is configured to read addresses only and the other DMA is configured to read image data only, the data bit width of the DMA that reads image data in DDR is usually smaller, while the data bit width of the DMA that reads image data in OCM is larger.

[0035] However, the two scenarios mentioned above need to share a data path, which causes the data width of the second scenario to be reduced from OCM to DDR, thus reducing the output efficiency of the second scenario.

[0036] To address the aforementioned technical problems, embodiments of this application provide an image data processing apparatus. It supports multiple operating modes, each with its own data path. The first and second read modules function differently in different operating modes and do not need to share a single data path. This maximizes the data width of each data path, thereby improving the data width and output efficiency of the second read module and the address module, and ultimately enhancing the device's output efficiency. Furthermore, embodiments of this application achieve hardware reuse by reusing some hardware configurations for multiple data paths, significantly reducing the hardware area.

[0037] Figure 2 This is a schematic diagram of the structure of a first image data processing apparatus 200 provided according to an embodiment of this application. (In conjunction with...) Figure 2 A detailed description of the image data processing apparatus is provided. The apparatus includes a first data path and a second data path.

[0038] In some examples, the first data path includes a first read module 201, a first bit width conversion module 203, and an address module 204. The second data path includes a second read module 202, an address module 204, and a write module 205.

[0039] For example, the first operating mode indicates that the address block is stored in a first memory or a second memory, and the image data to be processed is stored in the second memory. The data bit width of the second memory is greater than the data bit width of the first memory.

[0040] For example, in a first operating mode, the first read module 201 reads an address block from the first memory; the first width conversion module 203 divides the address block into multiple addresses; the address module 204 generates an absolute address corresponding to each address based on each address and preset read parameters; the second read module 202 reads the image data to be processed from each absolute address in the second memory; and the write module 205 writes the multiple image data to be processed into the first preset address block of the second memory. The address block includes multiple non-contiguous addresses.

[0041] For example, since the address module 204 can only receive one address at a time, in this embodiment of the application, the first read module 201 reads one address block from the first memory each clock cycle. The first bit width conversion module 203 divides the address block to obtain multiple addresses, and transmits one address to the address module 204 each time. Each address corresponds one-to-one with the image data to be processed.

[0042] For example, if an address block contains multiple addresses, then after splitting the address block, you get 4 addresses. Each address has a bit width of 32 bits.

[0043] For example, when the image data to be processed is stored in the first memory, the address module 204 outputs n bits of image data to be processed per clock cycle without changing its operation; that is, the output efficiency of the address module 204 remains unchanged. n is the data bit width of the first memory. A data bit width of n bits in the first memory indicates that n bits of image data to be processed can be requested from the first memory per clock cycle. n is an integer greater than or equal to 1.

[0044] For example, when the image data to be processed is stored in the second memory, the address module 204 outputs n bits of image data to be processed per clock cycle, increasing the output to m bits per clock cycle. This improves the output efficiency of the address module 204 by a factor of m / n. m is the data width of the second memory. A data width of m bits in the second memory indicates that m bits of image data to be processed can be requested from the second memory per clock cycle. m is an integer greater than or equal to 1.

[0045] For example, the first memory is DDR, and the data width of DDR is 64 bits, that is, n is 64; the second memory is OCM, and the data width of OCM is 256 bits, that is, m is 256.

[0046] Understandably, the first read module 201 can access both DDR and OCM, so its data width can only be 64 bits. The second read module 202 can only access OCM, so its data width is 256 bits.

[0047] For example, the first preset address block includes multiple consecutive addresses. Each address corresponds one-to-one with the image data to be processed. The first preset address block is stored in a pre-configured first register.

[0048] As can be seen from the above analysis, in the first operating mode of the embodiments of this application, if the aggregation module 2041 is started, the second read module 202 reads multiple image data to be processed from multiple non-contiguous absolute addresses in the second memory; the write module 205 writes the multiple image data to be processed to multiple contiguous addresses in the second memory.

[0049] For example, after the device is powered on, the first register is configured according to the first operating mode. After the configuration of the first register is completed, the device responds to the enable signal, determines the activation of multiple modules corresponding to the first operating mode based on the value of the first register, and executes the first operating mode. The specific operating mode to be executed is determined by the preceding device.

[0050] For example, activating address module 204 indicates that the subsequent device needs to use the image data output by the device for convolution calculation, which means it needs to read the image data to be processed stored in multiple non-contiguous addresses. Conversely, not activating address module 204 indicates that the subsequent device needs to use the image data output by the device for non-convolution calculation, which means it needs to sequentially read the image data to be processed stored in multiple contiguous addresses. Therefore, the first operating mode is applied to scenarios where the subsequent module performs convolution calculations.

[0051] It should be noted that, since the front-end devices and the back-end devices are not within the scope of protection of this application, the embodiments of this application will not describe the front-end devices and the back-end devices in detail.

[0052] In some examples, address module 204 includes aggregation module 2041 and dispersion module 2042; aggregation module 2041 and dispersion module 2042 do not operate simultaneously. Aggregation module 2041 generates the absolute address corresponding to each address based on each address and preset read parameters.

[0053] For example, after the device is powered on, when the aggregation module 2041 is started, the dispersion module 2042 is turned off; or when the aggregation module 2041 is turned off, the dispersion module 2042 is started.

[0054] For example, the preset read parameters include a read base address and a read offset address; the aggregation module 2041 calculates the read base address, the read offset address, and the address to obtain the absolute address. The read base address and the read offset address are obtained through the values ​​of a pre-configured first register.

[0055] In some examples, under one operating mode, the first read module 201 reads an address block from the first memory; the first width conversion module 203 divides the address block to obtain multiple addresses; the scattering module 2042 generates the absolute address corresponding to each address based on each address and preset write parameters; the second read module 202 reads multiple image data to be processed from the second memory of the second preset address block; and the write module 205 writes each image data to be processed to the corresponding absolute address in the second memory.

[0056] For example, the preset write parameters include the write base address and the write offset address; the scattering module 2042 calculates the write base address, the write offset address and the address to obtain the absolute address; wherein, the write base address and the write offset address are obtained through the value of the pre-configured first register.

[0057] For example, the second preset address block is stored in a pre-configured first register.

[0058] As can be seen from the above analysis, in the first operating mode of the embodiments of this application, if the scattering module 2042 is started, the second read module 202 reads multiple image data to be processed from multiple consecutive addresses in the second memory; the write module 205 writes the multiple image data to be processed to multiple non-contiguous absolute addresses in the second memory.

[0059] In the embodiments of this application, in a first operating mode, the second reading module 202 reads the image data to be processed from the second memory through the second data path, thus eliminating the need to share the first data path with the first reading module 201. Since the data bit width of the second memory is larger, the data bit width of the second reading module 202 is also larger. That is, compared to the related technologies where the first reading module 201 and the second reading module 202 share a single data path, the embodiments of this application increase the data bit width of both the second reading module 202 and the aggregation module 2041, thereby improving the output efficiency of both the second reading module 202 and the aggregation module 2041.

[0060] Figure 3 This is a schematic diagram of the structure of a second image data processing apparatus 200 provided according to an embodiment of this application. (In conjunction with...) Figure 3 A detailed description of the image data processing apparatus is provided. The apparatus includes: In some examples, the first data path also includes a first selector 206; the second data path also includes a second selector 207, a third selector 208, and a fourth selector 209.

[0061] For example, the first read module 201 transmits the address block to the first selector 206; the first selector 206 transmits the address block to the first width conversion module 203; the second read module 202 transmits multiple image data to be processed to the second selector 207; the second selector 207 transmits multiple image data to be processed to the aggregation module 2041; the aggregation module 2041 transmits multiple image data to be processed to the third selector 208; the third selector 208 transmits multiple image data to be processed to the fourth selector 209; and the fourth selector 209 transmits multiple image data to be processed to the write module 205.

[0062] For example, the first read module 201 transmits the address block to the first selector 206; the first selector 206 transmits the address block to the first width conversion module 203; the second read module 202 transmits multiple image data to be processed to the second selector 207; the second selector 207 transmits multiple image data to be processed to the distribution module 2042; the distribution module 2042 transmits multiple image data to be processed to the third selector 208; the third selector 208 transmits multiple image data to be processed to the fourth selector 209; and the fourth selector 209 transmits multiple image data to be processed to the write module 205.

[0063] For example, the first selector 206 includes a first input terminal, a second input terminal, and a first output terminal. The second selector 207 includes a third input terminal, a fourth input terminal, and a second output terminal. The third selector 208 includes a fifth input terminal, a sixth input terminal, and a third output terminal. The fourth selector 209 includes a seventh input terminal, an eighth input terminal, and a fourth output terminal.

[0064] For example, the conduction state of the first selector 206, the second selector 207, the third selector 208 and the fourth selector 209 is determined by the value of the second register corresponding to the first selector 206, the second selector 207, the third selector 208 and the fourth selector 209.

[0065] For example, when the value of the second register is 2 (==2), the first input of the first selector and the third input of the second selector are enabled; when the value of the second register is not 2 (!=2), the second input of the first selector and the fourth input of the second selector are enabled. When the value of the second register is 0 (==0), the fifth input of the third selector is enabled; when the value of the second register is not 0 (!=0), the sixth input of the third selector is enabled. When the value of the second register is not 1 (!=1), the seventh input of the fourth selector is enabled; when the value of the second register is 1 (==1), the eighth input of the fourth selector is enabled.

[0066] For example, when the value of the second register is 3, the second input terminal of the first selector 206, the fourth input terminal of the second selector 207, the sixth input terminal of the third selector 208, and the seventh input terminal of the fourth selector 209 are turned on.

[0067] For example, the first read module 201 transmits the address block to the second input terminal of the first selector 206, and the first selector 206 transmits the address block to the first width conversion module 203 through the first output terminal; the second read module 202 transmits multiple image data to be processed to the fourth input terminal of the second selector 207; the second selector 207 transmits multiple image data to be processed to the aggregation module 2041 through the second output terminal; the aggregation module 2041 transmits multiple image data to be processed to the sixth input terminal of the third selector 208; the third selector 208 transmits multiple image data to be processed to the seventh input terminal of the fourth selector 209 through the third output terminal; and the fourth selector 209 transmits the image data to be processed to the write module 205 through the fourth output terminal.

[0068] For example, the first selector 206, the second selector 207, the third selector 208, and the fourth selector 209 are all implemented based on a multiplexer.

[0069] Figure 4 This is a structural schematic diagram of a third image data processing apparatus 200 provided according to an embodiment of this application. (In conjunction with...) Figure 4 A detailed description of the image data processing apparatus is provided. The apparatus includes: In some examples, the device further includes a third data path and a fourth data path; the third data path includes a second read module 202, a first bit width conversion module 203, and an aggregation module 2041; the fourth data path includes a first read module 201, a second bit width conversion module 210, an aggregation module 2041, and a write module 205.

[0070] For example, the second operating mode indicates that the address block is stored in the second memory, and the image data to be processed is stored in the first memory. This second operating mode is applied to scenarios where subsequent modules perform convolution calculations.

[0071] For example, in the second operating mode, the second read module 202 reads an address block from the second memory; the first bit-width conversion module 203 divides the address block into multiple addresses; the aggregation module 2041 generates an absolute address corresponding to each address based on each address and preset read parameters; the first read module 201 reads the image data to be processed from each absolute address in the first memory; the second bit-width conversion module 210 combines multiple image data to be processed to obtain combined image data; and the write module 205 writes the combined image data into a first preset address block in the second memory. The address block includes multiple consecutive addresses.

[0072] For example, in the second operating mode, the second read module 202 reads address blocks from the second memory; the first bit width conversion module 203 divides the address blocks to obtain multiple addresses; the scattering module 2042 generates the absolute address corresponding to each address based on each address and preset write parameters; the first read module 201 reads the image data to be processed from each second preset address block of the first memory; the second bit width conversion module 210 combines multiple image data to be processed to obtain combined image data; and the write module 205 writes the combined image data into the first preset address block of the second memory.

[0073] For example, the second bit-width conversion module 210 receives image data to be processed from multiple clock cycles, combines the image data from multiple clock cycles to obtain a combined image data, and outputs the combined image data. In this embodiment of the application, by combining and transmitting the image data to be processed through the second bit-width conversion module 210, the transmission cycle is shortened and the data bit width of the aggregation module 2041 or the dispersion module 2042 is increased without changing the amount of image data transmitted.

[0074] For example, the second bit-width conversion module 210 receives four clock cycles of image data to be processed, combines these four clock cycles of image data into a single combined image data, and outputs this combined image data in the fourth clock cycle. Where the bit width of each image data to be processed is 64 bits, the bit width of the combined image data is 256 bits.

[0075] In some examples, the third data path also includes a first selector 206; the fourth data path also includes a second selector 207, a third selector 208, and a fourth selector 209.

[0076] For example, the second read module 202 transmits the address block to the first selector 206; the first selector 206 transmits the address block to the first bit width conversion module 203; the second bit width conversion module 210 transmits the combined image data to the second selector 207; the second selector 207 transmits the combined image data to the aggregation module 2041; the aggregation module 2041 transmits the combined image data to the third selector 208; the third selector 208 transmits the combined image data to the fourth selector 209; and the fourth selector 209 transmits the combined image data to the write module 205.

[0077] For example, when the value of the second register is 2, the first input terminal of the first selector 206, the third input terminal of the second selector 207, the sixth input terminal of the third selector 208, and the seventh input terminal of the fourth selector 209 are turned on.

[0078] For example, the second read module 202 transmits the address block to the first input terminal of the first selector 206; the first selector 206 transmits the address block to the first bit width conversion module 203 through the first output terminal; the second bit width conversion module 210 transmits the combined image data to the third input terminal of the second selector 207; the second selector 207 transmits the combined image data to the aggregation module 2041 through the second output terminal; the aggregation module 2041 transmits the combined image data to the sixth input terminal of the third selector 208; the third selector 208 transmits the combined image data to the seventh input terminal of the fourth selector 209 through the third output terminal; and the fourth selector 209 transmits the combined image data to the write module 205 through the fourth output terminal.

[0079] Figure 5 This is a schematic diagram of the structure of a fifth data path according to an embodiment of this application. (Combined with...) Figure 5 The fifth data path is described in detail. The fifth data path includes a first read module 201, a second bit-width conversion module 210, and a write module 205.

[0080] For example, the third operating mode indicates that the image data to be processed is stored in the first memory or the second memory. The third operating mode is applied to scenarios where subsequent devices perform non-convolutional computations.

[0081] For example, in the third operating mode, the first read module 201 reads multiple image data to be processed from the second preset address block of the first memory; the second bit width conversion module 210 combines the multiple image data to be processed to obtain combined image data; and the write module 205 writes the combined image data into the first preset address block of the second memory.

[0082] In some examples, the fifth data path also includes a third selector 208 and a fourth selector 209. Exemplarily, the second bit-width conversion module 210 transmits the combined image data to the third selector 208; the third selector 208 transmits the combined image data to the fourth selector 209; and the fourth selector 209 transmits the combined image data to the write module 205.

[0083] For example, when the value of the second register is 0, the fifth input of the third selector 208 and the seventh input of the fourth selector 209 are turned on.

[0084] For example, the second bit-width conversion module 210 transmits the combined image data to the fifth input terminal of the third selector 208; the third selector 208 transmits the combined image data to the seventh input terminal of the fourth selector 209 through the third output terminal; and the fourth selector 209 transmits the combined image data to the write module 205 through the fourth output terminal.

[0085] Figure 6 This is a schematic diagram of the structure of a sixth data path according to an embodiment of this application. (Combined with...) Figure 6 The fifth data path is described in detail. The sixth data path includes the second read module 202 and the write module 205.

[0086] For example, the fourth operating mode indicates that the image data to be processed is stored in the second memory; the fourth operating mode is applied to scenarios where the subsequent device performs non-convolutional calculations.

[0087] For example, the second read module 202 is used to read multiple image data to be processed from the second memory in the second preset address block in the fourth operating mode; the write module 205 is used to write the multiple image data to be processed into the first preset address block of the second memory.

[0088] In some examples, the sixth data path also includes a fourth selector 209.

[0089] For example, the second read module 202 transmits multiple image data to be processed to the fourth selector 209; the fourth selector 209 transmits multiple image data to be processed to the write module 205.

[0090] For example, when the value of the second register is 1, the eighth input of the fourth selector 209 is turned on.

[0091] For example, the second read module 202 transmits multiple image data to be processed to the eighth input terminal of the fourth selector 209, and the output terminal of the fourth selector 209 transmits multiple image data to be processed to the write module 205.

[0092] It should be noted that the image data processing device provided in the above embodiments is only illustrated by the division of the above functional modules when performing the corresponding steps. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0093] The embodiments of this application configure multiple data paths through a first read module, a second read module, an address module, a first bit width conversion module, a second bit width conversion module, and multiple selectors. This eliminates the need for the first read module and the second read module to share a single data path, thereby maximizing the data bit width of each data path. This increases the data bit width of the second read module and the address module, thus improving their output efficiency and consequently, the overall output efficiency of the device. Furthermore, the multiple data paths enable the execution of various operating modes, making it suitable for diverse scenarios and expanding its applicability. In addition, the embodiments of this application, by reusing data paths and multiple modules, not only integrate multiple functions but also reduce the hardware area.

[0094] Figure 7 This is a schematic flowchart of an image data processing method provided according to an embodiment of this application. Combined with... Figure 7 The image data processing method is described in detail. The method includes steps S701 to S705: In step S701, in the first operating mode, an address block is read from the first memory.

[0095] In the first operating mode, the address block is stored in either the first or second memory, and the image data to be processed is stored in the second memory; the data bit width of the second memory is greater than that of the first memory. In step S702, the address block is divided to obtain multiple addresses.

[0096] In step S703, the absolute address corresponding to each address is generated based on each address and the preset read parameters.

[0097] In step S704, the image data to be processed at each absolute address is read from the second memory.

[0098] In step S705, multiple image data to be processed are written into the first preset address block of the second memory.

[0099] In some embodiments, the address module includes an aggregation module and a distribution module; the aggregation module and the distribution module do not operate simultaneously. The aggregation module is used to generate the absolute address corresponding to each address based on each address and preset read parameters.

[0100] In some embodiments, the first read module is further configured to read an address block from the first memory; The first wide conversion module is also used to segment address blocks to obtain multiple addresses; The distributed module is also used to generate the absolute address corresponding to each address based on each address and preset write parameters; The second read module is also used to read multiple image data to be processed from the second memory at the second preset address block; The write module is also used to write each image data to be processed to the corresponding absolute address in the second memory.

[0101] In some embodiments, the first data path further includes a first selector; The second data path also includes a second selector, a third selector, and a fourth selector; The first read module is also used to transmit address blocks to the first selector; The first selector is used to transfer the address block to the first wide conversion module; The second reading module is also used to transmit the image data to be processed to the second selector; The second selector is used to transmit the image data to be processed to the aggregation module; The aggregation module is also used to transfer the image data to be processed to a third selector; The third selector is used to transfer the image data to be processed to the fourth selector; The fourth selector is used to transfer the image data to be processed to the write module.

[0102] In some embodiments, a third data path and a fourth data path are also included; The third data path includes a second read module, a first width conversion module, and an aggregation module; The fourth data path includes a first read module, a second bit-width conversion module, an aggregation module, and a write module; The second read module is also used to read address blocks from the second memory in the second operating mode; the second operating mode indicates that the address blocks are stored in the second memory and the image data to be processed is stored in the first memory. The first wide conversion module is used to segment address blocks and obtain multiple addresses; The aggregation module is used to generate the absolute address corresponding to each address based on each address and preset read parameters; The first read module is used to read the image data to be processed from each absolute address of the first memory; The second bit-width conversion module is used to combine multiple image data to be processed to obtain combined image data; The write module is used to write the combined image data into the first preset address block of the second memory.

[0103] In some embodiments, the third data path further includes a first selector; The fourth data path also includes a second selector, a third selector, and a fourth selector; The second read module is also used to transmit address blocks to the first selector; The first selector is also used to transfer address blocks to the first wide translation module; The second bit-width conversion module is also used to transmit the combined image data to the second selector; The second selector is also used to transmit the combined image data to the aggregation module; The aggregation module is also used to transmit combined image data to a third selector; The third selector is also used to transfer the combined image data to the fourth selector; The fourth selector is also used to transfer combined image data to the write module.

[0104] In some embodiments, a fifth data path is also included; The fifth data path includes a first read module, a second bit-width conversion module, and a write module; The first read module is used to read multiple image data to be processed from the second preset address block in the third operating mode; the third operating mode indicates that the image data to be processed is stored in the first memory or the second memory. The second bit-width conversion module is used to combine multiple image data to be processed to obtain combined image data; The write module is used to write the combined image data into the first preset address block of the second memory.

[0105] In some embodiments, the fifth data path further includes a third selector and a fourth selector; The second bit-width conversion module is also used to transmit the combined image data to the third selector; The third selector is also used to transfer the combined image data to the fourth selector; The fourth selector is also used to transfer combined image data to the write module.

[0106] In some embodiments, a sixth data path is also included; The sixth data path includes a second read module and a write module; The second read module is used to read multiple image data to be processed from the second memory in the fourth operating mode; the fourth operating mode means that the image data to be processed is stored in the second memory. The write module is used to write multiple image data to be processed into the first preset address block of the second memory.

[0107] In some embodiments, the sixth data path further includes a fourth selector; The second reading module is also used to transmit multiple image data to be processed to the fourth selector; The fourth selector is also used to transfer multiple image data to be processed to the write module.

[0108] In some embodiments, the first memory is DDR and the second memory is OCM.

[0109] Furthermore, the image data processing apparatus and image data processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the apparatus embodiments, which will not be repeated here.

[0110] In an embodiment of this application, the first read module reads an address block from the first memory in a first operating mode; the data bit width of the second memory is greater than that of the first memory; a first bit width conversion module divides the address block to obtain multiple addresses; the address module generates an absolute address corresponding to each address based on each address and preset read parameters; the second read module reads the image data to be processed from each absolute address in the second memory; and the write module writes the multiple image data to be processed into the first preset address block of the second memory. In the above technical solution, the first read module and the second read module do not need to share a data path, ensuring that the data bit width of the second data path is maximized, improving the data bit width of the second read module and the address module, thereby improving the output efficiency of the second read module and the address module, and consequently improving the output efficiency of the device.

[0111] An embodiment of this application also provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the method described above.

[0112] Taking computer devices as terminals as an example, Figure 8 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. See also... Figure 8 Terminal 800 can be: a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. Terminal 800 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0113] Typically, terminal 800 includes a processor 801 and a memory 802.

[0114] Processor 801 may include one or more processing cores, such as a quad-core processor, a penta-core processor, etc. Processor 801 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 801 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 801 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 801 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0115] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 802 are used to store at least one program code, which is executed by the processor 801 to implement the process of terminal execution in the method embodiments of this application.

[0116] In some embodiments, the terminal 800 may also optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 803 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a display screen 804, a camera assembly 805, an audio circuit 806, and a power supply 807.

[0117] Peripheral device interface 803 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 801 and memory 802. In some embodiments, processor 801, memory 802 and peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 801, memory 802 and peripheral device interface 803 can be implemented on separate chips or circuit boards, and this application embodiment does not limit this.

[0118] Display screen 804 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 804 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 801 for processing. In this case, display screen 804 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 804, disposed on the front panel of terminal 800; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal 800 or in a folded design; in other embodiments, display screen 804 may be a flexible display screen, disposed on a curved or folded surface of terminal 800. Furthermore, display screen 804 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 804 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0119] The camera assembly 805 is used to acquire images or videos. In some embodiments, the camera assembly 805 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 805 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash is a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0120] The audio circuit 806 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals which are then input to the processor 801 for processing. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 800. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 801 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 806 may also include a headphone jack.

[0121] Power supply 807 is used to power the various components in terminal 800. Power supply 807 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 807 includes a rechargeable battery, the rechargeable battery can support wired or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0122] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on terminal 800 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0123] Taking computer equipment as a server as an example, Figure 9 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 900 can vary significantly due to different configurations or performance. It may include one or more processors 901 (Central Processing Units, CPUs) and one or more memories 902. The one or more memories 902 store at least one computer program, which is loaded and executed by the one or more processors 901 to implement the aforementioned image data processing method. Of course, the server 900 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 900 may also include other components for implementing device functions, which will not be elaborated upon here.

[0124] Embodiments of this application also provide a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the method described above. Optionally, the computer-readable storage medium may be read-only memory (ROM), random access memory (RAM), compact-disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0125] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0126] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image data processing apparatus characterized by comprising: include: First data path and second data path; The first data path includes a first read module, a first bit width conversion module, and an address module; The second data path includes a second read module, the address module, and a write module; The first read module is used to read an address block from a first memory in a first operating mode; the first operating mode indicates that the address block is stored in the first memory or the second memory, and the image data to be processed is stored in the second memory; The data bit width of the second memory is greater than that of the first memory; The first bit-width conversion module is used to segment the address block to obtain multiple addresses; The address module is used to generate an absolute address corresponding to each address based on each address and a preset read parameter; The second read module is used to read the image data to be processed from each of the absolute addresses in the second memory; The write module is used to write multiple images to be processed into a first preset address block of the second memory.

2. The apparatus of claim 1, wherein, The address module includes an aggregation module and a distribution module; the aggregation module and the distribution module do not operate simultaneously. The aggregation module is used to generate the absolute address corresponding to each address based on each address and the preset read parameters.

3. The apparatus of claim 2, wherein, The first read module is also used to read an address block from the first memory; The first bit-width conversion module is also used to segment the address block to obtain multiple addresses; The distributed module is also used to generate an absolute address corresponding to each address based on each address and a preset write parameter; The second read module is further configured to read multiple image data to be processed from the second memory at a second preset address block; The write module is further configured to write each of the image data to be processed to the corresponding absolute address in the second memory.

4. The apparatus of claim 2, wherein, The first data path also includes a first selector; The second data path also includes a second selector, a third selector, and a fourth selector; The first read module is also used to transmit the address block to the first selector; The first selector is used to transmit the address block to the first bit-width conversion module; The second reading module is also used to transmit the image data to be processed to the second selector; The second selector is used to transmit the image data to be processed to the aggregation module; The aggregation module is also used to transmit the image data to be processed to the third selector; The third selector is used to transmit the image data to be processed to the fourth selector; The fourth selector is used to transmit the image data to be processed to the write module.

5. The apparatus of claim 2, wherein, The device further includes a third data path and a fourth data path; The third data path includes the second read module, the first bit-width conversion module, and the aggregation module; The fourth data path includes the first read module, the second bit-width conversion module, the aggregation module, and the write module; The second read module is also used to read the address block from the second memory in the second operating mode; The second operation mode indicates that the address block is stored in the second memory, and the image data to be processed is stored in the first memory; The first bit width conversion module is configured to divide the address block to obtain a plurality of addresses; The aggregation module is configured to generate the absolute address corresponding to each address based on each address and the preset reading parameter; The first reading module is configured to read the image data to be processed of each absolute address from the first memory; The second bit width conversion module is configured to combine a plurality of the image data to be processed to obtain combined image data; The writing module is configured to write the combined image data into the first preset address block of the second memory.

6. The apparatus of claim 5, wherein, The third data path further comprises the first selector; The fourth data path further comprises the second selector, the third selector, and the fourth selector; The second reading module is further configured to transmit the address block to the first selector; The first selector is further configured to transmit the address block to the first bit width conversion module; The second bit width conversion module is further configured to transmit the combined image data to the second selector; The second selector is further configured to transmit the combined image data to the aggregation module; The aggregation module is further configured to transmit the combined image data to the third selector; The third selector is further configured to transmit the combined image data to the fourth selector; The fourth selector is further configured to transmit the combined image data to the writing module.

7. The apparatus of claim 1, wherein, The device further comprises a fifth data path; The fifth data path comprises the first reading module, the second bit width conversion module, and the writing module; The first reading module is configured to read a plurality of the image data to be processed of the second preset address block from the first memory in a third operation mode; The third operation mode indicates that the image data to be processed is stored in the first memory or the second memory; The second bit width conversion module is configured to combine a plurality of the image data to be processed to obtain the combined image data; The writing module is configured to write the combined image data into the first preset address block of the second memory.

8. The apparatus of claim 7, wherein, The fifth data path further comprises a third selector and a fourth selector; The second bit width conversion module is further configured to transmit the combined image data to the third selector; The third selector is further configured to transmit the combined image data to the fourth selector; The fourth selector is further configured to transmit the combined image data to the writing module.

9. The apparatus of claim 1, wherein, The device further comprises a sixth data path; The sixth data path comprises the second reading module and the writing module; The second reading module is configured to read a plurality of the image data to be processed of the second preset address block from the second memory in a fourth operation mode; The fourth operation mode indicates that the image data to be processed is stored in the second memory; The writing module is configured to write the plurality of pieces of image data to be processed into the first preset address block of the second memory.

10. The apparatus of claim 9, wherein, The sixth data path further includes a fourth selector. The second reading module is further configured to transmit the plurality of pieces of image data to be processed to the fourth selector. The fourth selector is further configured to transmit the plurality of pieces of image data to be processed to the writing module.

11. The apparatus of any one of claims 1 to 10, wherein, The first memory is a DDR, and the second memory is an OCM.

12. An image data processing method, characterized by, Comprise: In the first operation mode, read an address block from the first memory; The first operation mode indicates that the address block is stored in the first memory or the second memory, and the image data to be processed is stored in the second memory; The data bit width of the second memory is greater than the data bit width of the first memory; Divide the address block to obtain a plurality of addresses; Generate an absolute address corresponding to each address based on each address and a preset reading parameter; Read the image data to be processed of each absolute address from the second memory; Write the plurality of pieces of image data to be processed into the first preset address block of the second memory.

13. A computer device, comprising: The computer device comprises a processor and a memory, and the memory is configured to store at least one program, and the at least one program is loaded and executed by the processor to implement the image data processing method of claim 12.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the image data processing method of claim 12.

Images