Image processing methods and apparatus, devices and media

Abstract

The present disclosure provides an image processing method, apparatus, device, and medium, and relates to the computer technology field, particularly to chip technology, artificial intelligence technology, and image processing technology field. The implementation includes reading a plurality of images arranged in a predetermined order from a first memory and writing them into a second memory readable and writable by a processing unit, reading the plurality of images from the second memory using the processing unit, determining a plurality of first feature data corresponding to the plurality of images, determining filler data for each of the first feature data based on the data amount of the first feature data and the predetermined data amount, splicing the first feature data and the filler data to obtain second feature data having a predetermined data amount, sequentially writing the plurality of second feature data into the second memory in the predetermined order, and transferring the second feature data stored in the second memory to the first memory in response to determining that the data amount of the second feature data stored in the second memory has reached a predetermined threshold.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, particularly to the fields of chip technology, artificial intelligence technology, and image processing technology. Specifically, it relates to an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product.

Background Art

[0002] Artificial intelligence is a subject that studies how to simulate some human thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.) on a computer. There are both hardware technologies and software technologies. The hardware technologies of artificial intelligence generally include technologies such as sensors, artificial intelligence dedicated chips, cloud computing, distributed storage, and big data processing. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0003] With the development of artificial intelligence technology, more and more applications are obtaining effects far exceeding the conventional algorithms based on artificial intelligence technology. Deep learning is a data-intensive algorithm and a computing-intensive algorithm. In order to improve the efficiency of performing image processing based on artificial intelligence technology, it is necessary to rationally utilize the hardware resources of the image processing apparatus and reduce the processing delay.

[0004] The methods described in this section are not necessarily the previously assumed or adopted methods. Unless otherwise specified, none of the methods described in this section should be considered as prior art just because they are included in this section. Similarly, unless otherwise specified, the problems mentioned in this section should not be considered as those approved by any prior art.

Summary of the Invention

[0005] [Cross-reference to Related Applications] This application claims priority to Chinese Patent Application No. 202311170382.8, filed on 11 September 2023, the entirety of which is incorporated herein by reference.

[0006] This disclosure provides image processing methods and apparatus, chips, electronic devices, computer-readable storage media, and computer program products.

[0007] According to one aspect of the present disclosure, an image processing method is provided which is performed by an image processing device, the image processing device includes a processing unit, the method comprising: reading a plurality of images arranged in a predetermined order from a first memory and writing them to a second memory readable and writable by the processing unit; reading the plurality of images from the second memory using the processing unit; determining a plurality of first feature data corresponding to each of the plurality of images; and performing a first process on each of the plurality of first feature data to obtain a second feature data having a predetermined amount of data, wherein the first process is The method includes: determining a first number based on the amount of data in the first feature data and the amount of data set in advance to obtain filling data consisting of a first number of filling elements; splicing the first feature data and the filling data to obtain the second feature data; sequentially writing a plurality of second feature data corresponding to the plurality of first feature data to the second memory in the order set in advance; and transporting the second feature data stored in the second memory to the first memory in response to the determination that the amount of data in the second feature data stored in the second memory has reached a predetermined threshold.

[0008] According to one aspect of the present disclosure, an image processing apparatus is provided, comprising: a processing unit; a writing unit configured to read a plurality of images arranged in a predetermined order from a first memory and write them to a second memory readable and writable by the processing unit; and a carrying unit configured to carry the second feature data stored in the second memory to the first memory in response to the determination that the amount of second feature data stored in the second memory has reached a predetermined threshold, wherein the processing unit comprises: a reading subunit configured to read the plurality of images from the second memory; and a first determination subunit configured to determine a plurality of first feature data corresponding to each of the plurality of images. A unit and a processing subunit configured to perform a first process on each of the plurality of first feature data to obtain second feature data having a predetermined amount of data, wherein the first process includes a processing subunit that determines a first number based on the amount of data of the first feature data and the predetermined amount of data to obtain filling data consisting of a first number of filling elements, and splices the first feature data and the filling data to obtain the second feature data, and a first writing subunit configured to sequentially write a plurality of second feature data corresponding to each of the plurality of first feature data to the second memory in the predetermined order.

[0009] According to one aspect of this disclosure, a chip including the image processing apparatus described above is provided.

[0010] According to one aspect of the present disclosure, an electronic device is provided, the electronic device comprising at least one processor and a memory communicated to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, thereby enabling the at least one processor to perform the image processing method described above.

[0011] According to one aspect of this disclosure, a non-temporary computer-readable storage medium is provided which stores computer instructions for causing a computer to perform the image processing method described above.

[0012] According to one aspect of this disclosure, a computer program product is provided which, when executed by a processor, can implement the above-described image processing method.

[0013] According to one or more embodiments of this disclosure, image processing efficiency can be improved. It should be understood that the content described in this section is not intended to identify the essential or important features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will be readily apparent from the following specification.

[0014] The drawings illustrate embodiments and constitute part of the specification, and are used to illustrate exemplary embodiments of the embodiments together with the textual description of the specification. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to elements that are similar but not necessarily identical. [Brief explanation of the drawing]

[0015] [Figure 1] This is a schematic diagram showing an exemplary system in which various methods described herein can be carried out according to exemplary embodiments of the present disclosure. [Figure 2] This flowchart shows an image processing method according to an exemplary embodiment of the present disclosure. [Figure 3A] This is a schematic diagram illustrating an image processing process according to an exemplary embodiment of the present disclosure. [Figure 3B] This is a schematic diagram illustrating an image processing process according to an exemplary embodiment of the present disclosure. [Figure 4] This is a block diagram showing an image processing apparatus according to an exemplary embodiment of the present disclosure. [Figure 5] This is a configuration block diagram showing exemplary electronic equipment that can be used to implement embodiments of the present disclosure. [Modes for carrying out the invention]

[0016] The following description illustrates exemplary embodiments of the disclosure, accompanied by drawings, and includes various details of the embodiments for the sake of ease of understanding; however, these should be considered merely illustrative. Therefore, as those skilled in the art should recognize, various changes and modifications can be made to the embodiments described herein without departing from the scope of the disclosure. Similarly, for clarity and brevity, descriptions of known functions and structures are omitted in the following description.

[0017] In this disclosure, unless otherwise specified, the use of terms such as “first,” “second,” etc., to describe various elements is not intended to limit the spatial, timing, or importance relationships of these elements. Such terms are used solely to distinguish one element from another. In some examples, the first element and the second element may refer to the same example of that element, or, depending on the contextual description, to different examples.

[0018] The terms used in describing the various examples in this disclosure are for illustrative purposes only and are not intended to limit them. Unless otherwise explicitly indicated in the context, such elements may be one or more, unless the number of elements is specifically limited. The terms "and / or" as used in this disclosure cover any one of the listed items and all possible combinations thereof.

[0019] In related technologies, when it is necessary to extract feature data of multiple images using a processing unit, first, the multiple images need to be transferred from a first memory for storing the original images to a second memory that can directly interact with the processing unit. Each time the calculation of the feature data of one image is completed, it needs to be directly written back to the first memory, which leads to fragmentation of request writing and causes interruptions and delays in data reading and writing.

[0020] Based on this, the present disclosure provides an image processing method. When extracting feature data of multiple images using a processing unit, this method converts the first feature data of each image into second feature data having a preset data amount, realizes length alignment of the feature data, and further temporarily stores the multiple feature data in a second memory. When the temporarily stored data reaches a certain length, it is comprehensively written back to the first memory, effectively reducing the data writing operation, improving the image processing efficiency, and accurately dividing the temporarily stored feature data based on the preset data amount, thereby avoiding the temporary storage operation from affecting the accuracy of image processing.

[0021] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0022] According to an embodiment of the present disclosure, FIG. 1 shows a schematic diagram of an exemplary system 100 in which various methods and apparatuses described in this specification can be implemented. Referring to FIG. 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 that couple the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 can be configured to execute one or more applications.

[0023] In an embodiment of the present disclosure, the server 120 can execute one or more services or software applications that enable the image processing method.

[0024] In some embodiments, server 120 can also provide other services or software applications that can include non-virtual environments and virtual environments. In some embodiments, these services can be provided as web-based services or cloud services, for example, and provided to users of client devices 101, 102, 103, 104, 105, and / or 106 in a software as a service (SaaS) model.

[0025] In the configuration shown in FIG. 1, server 120 may include one or more assemblies that implement the functions executed by server 120. These assemblies may include software assemblies, hardware assemblies, or combinations thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 can interact with server 120 by sequentially using one or more client applications to utilize the services provided by these assemblies. It should be understood that various different system configurations are possible and may be different from system 100. Therefore, FIG. 1 is an example of a system for implementing various methods described herein and is not intended to be limiting.

[0026] A user can send an image to be processed using client devices 101, 102, 103, 104, 105, and / or 106. The client device can provide an interface through which a user of the client device can interact with the client device. The client device can also output information to the user through the interface. Although only six client devices are illustrated in FIG. 1, as will be understood by those skilled in the art, the present disclosure can support any number of client devices.

[0027] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices such as portable handheld devices, general-purpose computers (e.g., personal computers and laptops), workstation computers, wearable devices, smartscreen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices may run various types and versions of software applications and operating systems such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (e.g., GOOGLE Chrome OS), or include various mobile operating systems such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include mobile phones, intelligent phones, tablets, and personal digital assistants (PDAs). Wearable devices may include head-mounted displays (e.g., smart glasses) and other devices. Game systems may include various handheld game devices, internet-enabled game devices, and so on. Client devices can run various applications, such as Internet-related applications, communication applications (e.g., email applications), short message service (SMS) applications, and can use various communication protocols.

[0028] Network 110 may be any type of network known to those skilled in the art, and it may use any one of several available protocols (including but not limited to TCP / IP, SNA, IPX, etc.) to support data communication. For example, one or more networks 110 may be a local area network (LAN), an Ethernet-based network, Token Ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, Wi-Fi), and / or any combination of these and / or other networks.

[0029] Server 120 may include one or more general-purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, midrange servers), blade servers, large computers, server clusters, or any other suitable configuration and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures related to virtualization (e.g., one or more flexible pools of virtualized logical memory devices to maintain the server's virtual memory devices). In various embodiments, Server 120 may run one or more services or software applications that provide the functions described below.

[0030] The computing units in server 120 can run one or more operating systems, including any of the above-mentioned operating systems and any commercial server operating systems. Server 120 can also run any one of a variety of additional server applications and / or middle-tier applications, such as an HTTP server, FTP server, CGI server, Java server, or database server.

[0031] In some embodiments, the server 120 may include one or more applications for analyzing and integrating data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. The server 120 may also include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0032] In some embodiments, server 120 may be a server in a distributed system or a server incorporating blockchain. Server 120 may be a cloud server, or an intelligent cloud computing server or intelligent cloud host equipped with artificial intelligence technology. A cloud server is a host product in a cloud computing service system and solves the problems of high management difficulty and low business scalability that exist in conventional physical hosts and virtual private server (VPS) services.

[0033] System 100 may also include one or more databases 130. In some embodiments, these databases can be used to store data and other information. For example, one or more of the databases 130 can be used to store information such as audio files and video files. The databases 130 can be located in various locations. For example, a database used by server 120 may be located locally with server 120, or it may be located away from server 120 and communicate with server 120 via a network or a dedicated connection. The databases 130 may be of different types. In some embodiments, the database used by server 120 may be a relational database. One or more of these databases can store, update, and retrieve data from the databases in response to instructions.

[0034] In some embodiments, one or more of the databases 130 may be used by an application to store application data. The databases used by the application may be of different types, such as a key-value repository, an object repository, or a general-purpose repository supported by the file system.

[0035] The system 100 in Figure 1 may be configured and operated in various ways so as to allow the application of the various methods and apparatus described herein.

[0036] Figure 2 is a flowchart showing an image processing method 200 performed by an image processing apparatus according to an exemplary embodiment of the present disclosure, the image processing apparatus including a processing unit. As shown in Figure 2, the method 200 is: Step S210 reads multiple images arranged in a predetermined order from the first memory and writes them to a second memory that can be read and written by the processing unit, Step S220 involves reading the plurality of images from the second memory using the processing unit, Step S230 involves determining a plurality of first feature data corresponding to each of the plurality of images, Step S240 is to perform a first process on each of the plurality of first feature data to obtain second feature data having a predetermined amount of data, Here, the first process in step S240 is Step S241 involves determining a first number based on the amount of data in the first feature data and the preset amount of data, thereby obtaining filling data consisting of a first number of filling elements. Step S240 includes step S242, which involves splicing the first feature data and the filling data to obtain the second feature data, Step S250, in the predetermined order, sequentially writes a plurality of second feature data corresponding to each of the plurality of first feature data into the second memory, The process includes step S260, which, in response to the determination that the amount of data of the second feature data stored in the second memory has reached a preset threshold, transports the second feature data stored in the second memory to the first memory.

[0037] By applying the above method 200, when extracting feature data from multiple images using a processing unit, a transformation operation of the feature data is performed to achieve length alignment of the second feature data. Furthermore, the second feature data, having a unified, pre-set amount of data, is temporarily stored in a second memory. When the temporarily stored data reaches a certain length, it is comprehensively written back to the first memory, effectively reducing data writing operations, lowering image processing delays, and improving efficiency.

[0038] In some examples, the first memory may be a large-capacity main memory, while the second memory may be a cache that can directly interact with the processing unit. By using a combination of a first memory with slow read / write speeds and large capacity, and a second memory with fast read / write speeds and small capacity, hardware resources can be used more rationally, and image processing efficiency can be improved.

[0039] In some examples, the processing unit in an image processing device may be a graphics processing unit (GPU), a central processing unit (CPU), or a logical device such as various types of computing chips or computing arrays, but this disclosure is not limited thereto.

[0040] In some cases, the pre-set thresholds may be manually set in advance according to the needs, for example, depending on the maximum storage capacity of the second memory or the transmission bandwidth of the data transmission link between the first and second memories.

[0041] In some examples, the preset threshold for the amount of data in the second feature data may be in units of images, for example, referring to the feature data of N images, that is, corresponding to a preset amount of data N times greater.

[0042] In some examples, the second feature data may consist of unit feature data having a predetermined size, in which case the predetermined threshold may be unit feature data, i.e., the feature data of a single image can be divided and carried in different batches. Processing efficiency can be improved by further refining the granularity of data sorting and carrying.

[0043] According to some embodiments, Method 200 further includes determining the target feature data from the plurality of second feature data based on a preset amount of data, and performing the second processing based on the target feature data, in response to determining that a second processing should be performed on the target feature data among the plurality of second feature data. Therefore, if temporary storage and transport can be performed comprehensively for the plurality of second feature data, the second feature data that has been spliced ​​and transported can be divided based on a preset amount of data, thereby obtaining accurate target feature data and ensuring accuracy in image processing while improving efficiency.

[0044] According to some embodiments, each element in the first feature data is a first data format, the filler element is a second data format, the bit width of the second data format is smaller than the bit width of the first data format, where the first processing for each first feature data further includes obtaining a third feature data by converting each element of the first feature data into a conversion element having the second data format, where the obtaining the second feature data by splicing the first feature data and the filler data includes obtaining the second feature data by splicing the third feature data and the filler data. This allows the first feature data to be further compressed and converted into data with a smaller bit width, thereby improving image processing efficiency. As can be understood, by simply aligning, splicing, and temporarily storing the compressed feature data while maintaining the storage capacity of the second memory, a larger set of feature data can be temporarily stored in the second memory, thereby further reducing the number of write operations and improving image processing efficiency.

[0045] In some examples, the third feature data may be bit data; that is, the feature map is converted to a bit map, thereby achieving data compression.

[0046] According to some embodiments, the first memory includes a first subunit and a second subunit in which the plurality of images are stored, and reading the plurality of images arranged in a predetermined order from the first memory and writing them to the second memory means determining a plurality of sub-image data, including a first sub-image data and a second sub-image data, based on the plurality of images, writing the first sub-image data to the second subunit so that the first sub-image data can be read from the second subunit, and in response to the determination that the first sub-image data has already been read, the second sub-image data The process includes writing the data to the second subunit, wherein the transfer of the second feature data stored in the second memory to the first memory in response to the determination that the amount of data of the second feature data stored in the second memory has reached a preset threshold includes the transfer of the second feature data stored in the second memory to the second subunit in response to the determination that the amount of data of the second feature data stored in the second memory has reached a preset threshold and that the second processing should be performed based on the second feature data stored in the second memory.

[0047] This allows the first memory to be configured as a multi-level memory system, which can include, for example, a first subunit and a second subunit. In this case, the image data to be processed must be transported stepwise through the first subunit → second subunit → second memory, and if it is determined that processing needs to be re-executed for the second feature data, it can be temporarily stored in the second subunit, which is closer to the second memory, thereby reducing data transport delays and further improving image processing efficiency.

[0048] According to some embodiments, the act of transporting the second feature data stored in the second memory to the second subunit includes transporting the second feature data stored in the second memory to the target storage location in response to the determination that a target storage location exists in the second subunit, wherein the target storage location does not include sub-image data that has not yet been read. This allows the second feature data to be temporarily stored at the target storage location to avoid data loss and ensure accuracy in image processing, provided that data at the target storage location has already been read.

[0049] According to some embodiments, each of the plurality of images includes a first image channel and a second image channel, and each of the plurality of feature data includes a first sub-feature data corresponding to the first image channel and a second sub-feature data corresponding to the second image channel, wherein determining the plurality of sub-image data based on the plurality of images includes determining the first sub-image data based on the first image channel of each of the plurality of images and determining the second sub-image data based on the second image channel of each of the plurality of images.

[0050] In accordance with the embodiments described in the disclosure, when extracting first feature data on an image channel basis, the image data is divided according to the channel dimensions, and the channel extraction operation is performed between the first subunit and the second subunit. This eliminates the need for the processing unit to jump to read the channel data, thereby improving processing efficiency. Figures 3A-3B are schematic diagrams showing the image processing process according to an exemplary embodiment of the disclosure.

[0051] In this example, the image processing method 200 can be applied to the inference or training process of an image processing model.

[0052] In some examples, the inference or training process of an image processing model can be represented as a directed acyclic computation graph composed of multiple types of operators. These operators may include convolution operators, batch normalization operators, vector addition operators, activation operators, maximum value search operators, pooling operators, and others.

[0053] In some examples, sequences consisting of batch normalization operators, vector addition operators, activation operators, and maximum value search operators connected in series appear frequently in computation graphs. Constructing a fusion operator based on a sequence consisting of these four operators connected in series can reduce data read / write operations between operators and improve the execution efficiency of the computation graph.

[0054] In some examples, during the inference or training process of an image processing model, calculations may be performed by calling a fusion operator based on input task information. For example, if the input task information indicates that calculations should be performed based on a sequence consisting of a batch normalization operator, a vector addition operator, an activation operator, and a maximum value search operator connected in series, the four operators included in the fusion operator are called in order. If the input task information indicates that calculations should be performed based only on the batch normalization operator and the activation operator, only the batch normalization operator and the activation operator from the fusion operator are called in order.

[0055] In some examples, a processing unit for executing the image processing computation process may include a cache corresponding to the aforementioned second memory that directly interacts with the processing unit. Furthermore, the first memory may be a multilevel storage system including a first subunit and a second subunit, i.e., corresponding to different levels of cache.

[0056] As shown in Figure 3A, the input image data may be divided according to the channel dimensions. Sub-image data for multiple image channels is copied from the first subunit to the second subunit each time, according to the size of the second subunit in the first memory. The processing unit reads the data from the second subunit and performs the initial calculation. When it is determined that the intermediate results obtained from the initial calculation need to participate in the remaining calculation, the intermediate results can be written back to the second subunit. Subsequently, the intermediate results can be read directly from the second subunit and the remaining calculation can be performed, eliminating the need to re-execute the data transfer from the first subunit to the second subunit, thereby improving image processing efficiency.

[0057] As shown in Figure 3B, after extracting feature data from multiple images using a processing unit, the length alignment of the feature data can be achieved by splicing the first feature data of each image with the corresponding filler data, thereby obtaining second feature data with a unified, pre-set amount of data.

[0058] In some examples, the first feature data may be a feature map of the activation function output by the activation operator. Furthermore, in one example, the first feature data can be further compressed and the entire graph of the activation function converted into a bitmap, thereby reducing the amount of data and improving image processing efficiency.

[0059] According to one aspect of the present disclosure, an image processing apparatus is further provided. Figure 4 is a block diagram showing an image processing apparatus 400 according to an exemplary embodiment of the present disclosure. As shown in Figure 4, the apparatus 400 is Processing unit 410 and, The system includes a writing unit 420 configured to read a plurality of images arranged in a predetermined order from a first memory and write them to a second memory that can be read and written by the processing unit, and a carrying unit 430 configured to carry the second feature data stored in the second memory to the first memory in response to the determination that the amount of second feature data stored in the second memory has reached a predetermined threshold. Here, the processing unit 410 is A reading subunit 411 configured to read the plurality of images from the second memory, A first determination subunit 412 is configured to determine a plurality of first feature data corresponding to each of the plurality of images, A processing subunit 413 is configured to perform a first process on each of the plurality of first feature data to obtain a second feature data having a predetermined amount of data, wherein the first process includes obtaining filling data consisting of a first number of filling elements by determining a first number based on the amount of data of the first feature data and the predetermined amount of data, and obtaining the second feature data by splicing the first feature data and the filling data. The system includes a first writing subunit 414 configured to sequentially write a plurality of second feature data corresponding to each of the plurality of first feature data to the second memory in the predetermined order.

[0060] According to some embodiments, each element in the first feature data is a first data format, the fill element is a second data format, the bit width of the second data format is smaller than the bit width of the first data format, where the first processing for each first feature data further includes converting each element of the first feature data into a conversion element having the second data format, thereby obtaining a third feature data, where the splicing of the first feature data and the fill data to obtain the second feature data includes splicing the third feature data and the fill data to obtain the second feature data.

[0061] According to some embodiments, the first memory includes a first subunit and a second subunit storing the plurality of images, and the write unit 420 includes a second determination subunit that determines a plurality of sub-image data, including a first sub-image data and a second sub-image data, based on the plurality of images, and a second write subunit configured to write the first sub-image data to the second subunit so that the first sub-image data can be read from the second subunit, and further configured to write the second sub-image data to the second subunit in response to the determination that the first sub-image data has already been read, wherein the carry unit 430 is configured to carry the second feature data stored in the second memory to the second subunit in response to the determination that the amount of data of the second feature data stored in the second memory has reached a preset threshold, and in response to the determination that the second processing should be performed based on the second feature data stored in the second memory.

[0062] According to some embodiments, the transport unit 430 is configured to transport the second feature data stored in the second memory to the target storage location in response to the determination that a target storage location exists in the second subunit, wherein the target storage location does not include unreadable sub-image data.

[0063] According to some embodiments, each of the plurality of images includes a first image channel and a second image channel, and each of the plurality of feature data includes a first sub-feature data corresponding to the first image channel and a second sub-feature data corresponding to the second image channel, wherein the second determinative subunit is configured to determine the first sub-image data based on the first image channel of each of the plurality of images and to determine the second sub-image data based on the second image channel of each of the plurality of images.

[0064] According to some embodiments, the apparatus 400 further includes a determination unit configured to determine the target feature data from the plurality of second feature data based on a preset amount of data, in response to a determination that a second process should be performed on the target feature data among the plurality of second feature data, wherein the processing unit 410 is further configured to perform the second process based on the target feature data.

[0065] It should be understood that the operation of each unit of the image processing apparatus 400 shown in Figure 4 can correspond to each step in the image processing method 200 described in Figure 2. Therefore, the operations, features, and advantages described above for method 200 are similarly applicable to the apparatus 400 and each unit contained therein. For the sake of brevity, some operations, features, and advantages are omitted from this description.

[0066] According to one aspect of this disclosure, the chip further includes the image processing device 400 described above.

[0067] According to one aspect of the present disclosure, an electronic device is provided, the electronic device comprising at least one processor and a memory communicated to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, thereby enabling the at least one processor to perform the image processing method described above.

[0068] According to one aspect of this disclosure, a non-temporary computer-readable storage medium is further provided, which stores computer instructions for causing a computer to perform the image processing method described above.

[0069] According to one aspect of this disclosure, a computer program product including a computer program is also provided. When the computer program is executed by a processor, it implements the image processing method described above.

[0070] As shown in Figure 5, a configuration block diagram of an electronic device 500, which can be used as a server or client of the Disclosure, is described here as an example of a hardware device applicable to various aspects of the Disclosure. The electronic device represents various forms of digital electronic computer equipment, such as laptop computers, desktop computers, stages, personal digital assistants, servers, blade servers, large computers, and other suitable computers. The electronic device may further represent various forms of mobile devices, such as personal digital processing, cellular phones, smartphones, wearable devices, and other similar computing devices. The components, their connections, and their functions shown herein are illustrative and are not intended to limit the realization of the Disclosure as described and / or claimed herein.

[0071] As shown in Figure 5, the device 500 includes a computing unit 501, which can perform various appropriate operations and processes by computer programs stored in read-only memory (ROM) 502 or by computer programs loaded from storage unit 508 into random access memory (RAM) 503. The RAM 503 may store various programs and data necessary to operate the device 500. The computing unit 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0072] Multiple components in the device 500 are connected to an I / O interface 505, which includes an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 may be any type of device capable of inputting information into the device 500, and may receive input numeric or character information and generate key signal inputs relating to user settings and / or function control of the electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackboard, trackball, lever, microphone, and / or remote control. The output unit 507 may be any type of device capable of presenting information, and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. The storage unit 508 may include, but is not limited to, a magnetic disk or an optical disk. The communication unit 509 enables the device 500 to exchange information / data with other devices via computer networks, such as the Internet, and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, 802.11 devices, WiFi devices, WiMAX devices, cellular communication devices, and / or similar devices.

[0073] The computing unit 501 may be a variety of general-purpose and / or dedicated processing components having processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units for executing machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs each of the methods and processes described above, such as the image processing method. For example, in some embodiments, the image processing method may be implemented as a computer software program and tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed in the device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the image processing method described above can be performed. Alternatively, in another embodiment, the computing unit 501 may be configured to perform the image processing method in any other suitable manner (for example, by firmware).

[0074] Various embodiments of the systems and technologies described herein may be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and which may transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0075] These program codes may be provided to a processor or controller of a general-purpose computer, a dedicated computer, or other programmable data processing device, so that when the program codes are executed by the processor or controller, the functions / operations defined in the flowcharts and / or block diagrams are performed. The program codes may be executed entirely by machine, partially by machine, partially by machine and partially by remote machine as an independent software package, or entirely by remote machine or server.

[0076] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or stores a program for use by or in combination with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any appropriate combination of the above. More specific examples of machine-readable storage media include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any appropriate combination of the above.

[0077] To provide user interaction, a computer may implement the systems and techniques described herein, the computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitoring monitor), and a keyboard and pointing device (e.g., a mouse or trackball), the user may provide input to the computer using the keyboard and pointing device. Other types of devices may further provide user interaction, for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and the computer may receive input from the user in any form (including sound input, voice input, or tactile input).

[0078] The systems and technologies described herein may be implemented in computing systems including backstage components (e.g., as data servers), computing systems including middleware components (e.g., application servers), computing systems including front-end components (e.g., user computers with graphical user interfaces or web browsers, through which users can interact with embodiments of these systems and technologies), or computing systems consisting of any combination of these backstage components, middleware components, or front-end components. The components of the system may be interconnected by digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local networks (LANs), wide area networks (WANs), the internet, and blockchain networks.

[0079] A computer system may include a client and a server. The client and server are generally geographically distant from each other and typically interact via a communication network. The client-server relationship is created by running computer programs on the relevant computers that have a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server incorporating blockchain technology.

[0080] It should be understood that the steps may be reordered, added, or deleted using the various forms of flows described above. For example, each step described herein may be performed in parallel, sequentially, or in a different order, as long as it achieves the desired results of the proposed technology disclosed herein, and this specification does not limit this.

[0081] While examples of the embodiments or models described herein have been explained with reference to the drawings, it should be understood that the methods, systems, and apparatus described above are merely illustrative examples, and the scope of the present invention is not limited by these examples. Various elements in the embodiments or models may be omitted or replaced by equivalent elements. Furthermore, each step may be performed in a different order than that described herein. In addition, various elements in the embodiments or models may be combined in various ways. In essence, as technology advances, many of the elements described herein may be replaced by equivalent elements appearing later in this disclosure.

Claims

1. An image processing method performed by an image processing device, wherein the image processing device includes a processing unit, and the method is The process involves reading multiple images arranged in a predetermined order from a first memory and writing them to a second memory that can be read and written by the processing unit. The processing unit is used to read the plurality of images from the second memory, To determine a plurality of first feature data corresponding to each of the plurality of images, The first process is performed on each of the plurality of first feature data to obtain second feature data having a predetermined amount of data, wherein the first process is: Based on the amount of data in the first feature data and the amount of data set in advance, a first number is determined, thereby obtaining filling data composed of filling elements of the first number. This includes splicing the first feature data and the filling data to obtain the second feature data, The process involves sequentially writing a plurality of second feature data, each corresponding to one of the plurality of first feature data, into the second memory in the predetermined order, An image processing method comprising: transporting the second feature data stored in the second memory to the first memory in response to the determination that the amount of data of the second feature data stored in the second memory has reached a preset threshold.

2. Each element in the first feature data is a first data format, the fill element is a second data format, the bit width of the second data format is smaller than the bit width of the first data format, and the first processing for each of the first feature data is: To obtain third feature data, the method further includes converting each element of the first feature data into a transformation element having the second data format, Here, splicing the first feature data and the fill data to obtain the second feature data is, The method according to claim 1, comprising splicing the third feature data and the filling data to obtain the second feature data.

3. The first memory includes a first subunit and a second subunit in which the plurality of images are stored, and reading the plurality of images arranged in a predetermined order from the first memory and writing them to the second memory is: Based on the aforementioned multiple images, a plurality of sub-image data, including a first sub-image data and a second sub-image data, is determined. The first sub-image data is written to the second sub-unit, and the first sub-image data can be read from the second sub-unit. This includes writing the second sub-image data to the second sub-unit in response to the confirmation that the first sub-image data has already been read. Here, in response to the determination that the amount of data of the second feature data stored in the second memory has reached a preset threshold, the second feature data stored in the second memory is moved to the first memory. The method according to claim 1, comprising transporting the second feature data stored in the second memory to the second subunit in response to the determination that the amount of data of the second feature data stored in the second memory has reached a preset threshold, and in response to the determination that a second process should be performed based on the second feature data stored in the second memory.

4. Transporting the second feature data stored in the second memory to the second subunit is, The method according to claim 3, comprising transporting second feature data stored in the second memory to the target storage location in response to the determination that a target storage location exists in the second subunit, wherein the target storage location does not include subimage data that has not yet been read.

5. Each of the plurality of images includes a first image channel and a second image channel, and each of the plurality of feature data includes a first sub-feature data corresponding to the first image channel and a second sub-feature data corresponding to the second image channel. Here, determining multiple sub-image data based on the aforementioned multiple images is The first sub-image data is determined based on the first image channel of each of the plurality of images, The method according to claim 3, further comprising determining the second sub-image data based on the second image channel of each of the plurality of images.

6. In response to the determination that a second process should be performed on the target feature data among the plurality of second feature data, the target feature data is determined from the plurality of second feature data based on the predetermined amount of data. The method according to claim 1, further comprising performing the second processing based on the target feature data.

7. An image processing device, Processing unit and The system includes a writing unit configured to read a plurality of images arranged in a predetermined order from a first memory and write them to a second memory that can be read and written by the processing unit, and a carrying unit configured to carry the second feature data stored in the second memory to the first memory in response to the determination that the amount of data of the second feature data stored in the second memory has reached a predetermined threshold. Here, the processing unit is A reading subunit configured to read the plurality of images from the second memory, A first determination subunit configured to determine a plurality of first feature data corresponding to each of the plurality of images, A processing subunit configured to perform a first process on each of the plurality of first feature data to obtain a second feature data having a predetermined amount of data, The first process is, Based on the amount of data in the first feature data and the amount of data set in advance, a first number is determined, thereby obtaining filling data composed of filling elements of the first number. A processing subunit that includes splicing the first feature data and the filling data to obtain the second feature data, An image processing apparatus including a first writing subunit configured to sequentially write a plurality of second feature data corresponding to each of the plurality of first feature data to the second memory in the predetermined order.

8. Each element in the first feature data is a first data format, the fill element is a second data format, the bit width of the second data format is smaller than the bit width of the first data format, and the first processing for each of the first feature data is: To obtain third feature data, the method further includes converting each element of the first feature data into a transformation element having the second data format, Here, splicing the first feature data and the fill data to obtain the second feature data is, The apparatus according to claim 7, comprising splicing the third feature data and the filling data to obtain the second feature data.

9. The first memory includes a first subunit and a second subunit in which the plurality of images are stored, and the writing unit is A second determination subunit that determines a plurality of sub-image data, including a first sub-image data and a second sub-image data, based on the plurality of images, The first sub-image data is written to the second sub-unit, and the first sub-image data is configured to be read from the second sub-unit. Furthermore, it includes a second writing subunit configured to write the second subunit to the second subunit in response to the confirmation that the first subunit image data has already been read, The apparatus according to claim 7, configured to transport the second feature data stored in the second memory to the second subunit in response to the determination that the amount of data of the second feature data stored in the second memory has reached a preset threshold, and in response to the determination that a second process should be performed based on the second feature data stored in the second memory.

10. The aforementioned transport unit is The apparatus according to claim 9, wherein the second subunit is configured to transport second feature data stored in the second memory to the target storage location in response to the determination that a target storage location exists, wherein the target storage location does not include sub-image data that has not yet been read.

11. Each of the plurality of images includes a first image channel and a second image channel, and each of the plurality of feature data includes a first sub-feature data corresponding to the first image channel and a second sub-feature data corresponding to the second image channel. Here, the second definitive subunit is, Based on the first image channel of each of the plurality of images, the first sub-image data is determined. The apparatus according to claim 9, configured to determine the second sub-image data based on the second image channel of each of the plurality of images.

12. The system further includes a determination unit configured to determine the target feature data from the plurality of second feature data based on a predetermined amount of data, in response to the determination that a second process should be performed on the target feature data among the plurality of second feature data, The apparatus according to claim 7, wherein the processing unit is further configured to perform the second processing based on the target feature data.

13. A chip comprising an image processing apparatus according to any one of claims 7 to 12.

14. It is an electronic device, At least one processor, The memory is connected to at least one of the processors, where, An electronic device wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to cause the at least one processor to perform the method according to any one of claims 1 to 6.

15. A non-temporary computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method described in any one of claims 1 to 6.

16. A computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 6.

Images