Image processing method and apparatus, device, and medium

Abstract

The present disclosure relates to the technical field of computers, in particular to the fields of chip technology, artificial intelligence technology, and image processing technology, and provides an image processing method and apparatus, a device, and a medium. According to an implementation solution, the method comprises: reading, from a first memory, a plurality of images arranged according to a preset sequence, and writing the images into a second memory capable of being read and written by a processing unit; using the processing unit to read the plurality of images from the second memory; determining a plurality of pieces of first feature data respectively corresponding to the plurality of images; for each piece of first feature data, on the basis of the data volume of said first feature data and a preset data volume, determining filling data; combining the first feature data and the filling data to obtain second feature data having the preset data volume; according to the preset sequence, sequentially writing a plurality of pieces of second feature data into the second memory; and in response to determining that the data volume of the second feature data stored in the second memory has reached a preset threshold, transferring to the first memory the second feature data stored in the second memory.

Description

Image processing method, device, equipment and medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202311170382.8 filed on September 11, 2023, the entire contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to the field of computer technology, in particular to the fields of chip technology, artificial intelligence technology, and image processing technology, and specifically to an image processing method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0004] Artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0005] With the development of artificial intelligence (AI), a growing number of applications are achieving results far exceeding those of traditional algorithms. Deep learning is a data-intensive and computationally intensive algorithm. To improve the efficiency of AI-based image processing, it is necessary to rationally utilize the hardware resources of image processing devices and reduce processing latency.

[0006] The approaches described in this section are not necessarily approaches that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any approach described in this section is prior art simply by virtue of its inclusion in this section. Similarly, unless otherwise indicated, the issues raised in this section should not be considered as having been recognized in any prior art.

[0007] Summary of the Invention

[0008] The present disclosure provides an image processing method, apparatus, chip, electronic device, computer-readable storage medium, and computer program product.

[0009] According to one aspect of the present disclosure, there is provided an image processing method performed by an image processing device, the image processing device including a processing unit, the method including: reading a plurality of images arranged in a preset order from a first memory and writing them into a second memory that can be read and written by the processing unit; using the processing unit to read the plurality of images from the second memory; determining a plurality of first feature data corresponding to the plurality of images respectively; performing a first processing on each of the plurality of first feature data to obtain second feature data with a preset data amount, the first processing including: determining a first quantity based on the data amount of the first feature data and the preset data amount to obtain filling data consisting of a first quantity of filling elements; and splicing the first feature data and the filling data to obtain the second feature data; writing a plurality of second feature data corresponding to the plurality of first feature data respectively into the second memory in sequence according to the preset order; and in response to determining that the data amount of the second feature data stored in the second memory reaches a preset threshold, moving the second feature data stored in the second memory to the first memory.

[0010] According to one aspect of the present disclosure, an image processing device is provided, comprising: a processing unit; a writing unit configured to read a plurality of images arranged in a preset order from a first memory and write the images into a second memory that can be read and written by the processing unit, wherein the processing unit comprises: a reading subunit configured to read the plurality of images from the second memory; a first determining subunit configured to determine a plurality of first feature data corresponding to the plurality of images; a processing subunit configured to perform a first process on each of the plurality of first feature data to obtain second feature data having a preset data volume, wherein the first process comprises: determining a first quantity based on the data volume of the first feature data and the preset data volume to obtain padding data consisting of a first quantity of padding elements; and concatenating the first feature data and the padding data to obtain the second feature data; the first writing subunit configured to write a plurality of second feature data corresponding to the plurality of first feature data in sequence to the second memory in the preset order; and a transporting unit configured to transport the second feature data stored in the second memory to the first memory in response to determining that the data volume of the second feature data stored in the second memory reaches a preset threshold.

[0011] According to one aspect of the present disclosure, a chip is provided, including the image processing device as described above.

[0012] According to one aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned image processing method.

[0013] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned image processing method.

[0014] According to one aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program can implement the above-mentioned image processing method when executed by a processor.

[0015] According to one or more embodiments of the present disclosure, image processing efficiency can be improved.

[0016] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings illustrate exemplary embodiments and constitute a part of the specification. Together with the description of the specification, they serve to explain exemplary implementation of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals designate similar, but not necessarily identical, elements.

[0018] FIG1 shows a schematic diagram of an exemplary system in which various methods described herein may be implemented according to an exemplary embodiment of the present disclosure;

[0019] FIG2 shows a flowchart of an image processing method according to an exemplary embodiment of the present disclosure;

[0020] 3A-3B are schematic diagrams showing an image processing process according to an exemplary embodiment of the present disclosure;

[0021] FIG4 shows a structural block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure;

[0022] FIG5 shows a structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] The following description of exemplary embodiments of the present disclosure is provided in conjunction with the accompanying drawings, which include various details to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.

[0024] In this disclosure, unless otherwise specified, the use of terms such as "first" and "second" to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements. Such terms are only used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, while in some cases, based on the context of the description, they may also refer to different instances.

[0025] The terms used in the descriptions of the various examples described in this disclosure are for the purpose of describing specific examples only and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element may be one or more. In addition, the term "and / or" used in this disclosure encompasses any one and all possible combinations of the listed items.

[0026] In the related art, when a processing unit is needed to extract feature data of multiple images, the multiple images need to be first moved from a first memory used to store the original images to a second memory that can directly interact with the processing unit. After the feature data of each image is calculated, it needs to be written back to the first memory, resulting in fragmented write requests and interruption delays in data reading and writing.

[0027] Based on this, the present disclosure provides an image processing method. When using a processing unit to extract feature data of multiple images, the first feature data of each image is converted into second feature data with a preset data amount to achieve length alignment of the feature data, and then the multiple feature data are temporarily stored in the second memory. When the temporarily stored data reaches a certain length, it is uniformly written back to the first memory, effectively reducing data writing operations and improving image processing efficiency. It can also correctly split the temporarily stored feature data based on the preset data amount to avoid the temporary storage operation affecting the correctness of image processing.

[0028] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0029] FIG1 shows a schematic diagram of an exemplary system 100 in which the various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Referring to FIG1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0030] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of the image processing method.

[0031] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0032] In the configuration shown in FIG1 , server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may, in turn, utilize one or more client applications to interact with server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from system 100. Therefore, FIG1 is an example of a system for implementing the various methods described herein and is not intended to be limiting.

[0033] A user can use client devices 101, 102, 103, 104, 105, and / or 106 to send an image to be processed. The client device can provide an interface that enables the user of the client device to interact with the client device. The client device can also output information to the user via the interface. Although Figure 1 only depicts six client devices, those skilled in the art will appreciate that the present disclosure can support any number of client devices.

[0034] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. The client device is capable of executing various different applications, such as various Internet-related applications, communication applications (eg, email applications), Short Message Service (SMS) applications, and may use various communication protocols.

[0035] The network 110 may be any type of network known to those skilled in the art that can support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0036] Server 120 may include one or more general-purpose computers, specialized server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0037] The computing units in the server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

[0038] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display the data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0039] In some embodiments, server 120 may be a distributed system server or a server integrated with blockchain. Server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host equipped with artificial intelligence technology. A cloud server is a host product within the cloud computing service system that addresses the management difficulties and poor scalability of traditional physical hosts and virtual private servers (VPS) services.

[0040] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The databases 130 may reside in a variety of locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different types. In some embodiments, the databases used by the server 120 may be, for example, relational databases. One or more of these databases may store, update, and retrieve data to and from the databases in response to commands.

[0041] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0042] The system 100 of FIG. 1 may be configured and operated in various ways to enable application of the various methods and apparatuses described in accordance with this disclosure.

[0043] FIG2 shows a flow chart of an image processing method 200 performed by an image processing apparatus according to an exemplary embodiment of the present disclosure, wherein the image processing apparatus includes a processing unit. As shown in FIG2 , the method 200 includes:

[0044] Step S210: reading a plurality of images arranged in a preset order from a first memory and writing the images into a second memory that can be read and written by the processing unit;

[0045] Step S220: reading the plurality of images from the second memory using the processing unit;

[0046] Step S230: determining a plurality of first feature data corresponding to the plurality of images respectively;

[0047] Step S240: performing a first process on each of the plurality of first feature data to obtain second feature data having a preset data volume.

[0048] The first process in step S240 includes:

[0049] Step S241: determining a first quantity based on the data quantity of the first characteristic data and the preset data quantity to obtain filling data consisting of a first number of filling elements; and

[0050] Step S242: concatenate the first feature data and the filling data to obtain the second feature data;

[0051] Step S250: writing the plurality of second feature data corresponding to the plurality of first feature data into the second memory in sequence according to the preset order; and

[0052] Step S260: In response to determining that the amount of the second feature data stored in the second memory reaches a preset threshold, the second feature data stored in the second memory is moved to the first memory.

[0053] By applying the above-mentioned method 200, when the processing unit is used to extract feature data of multiple images, the length alignment of the second feature data is achieved by performing a conversion operation on the feature data, so that the second feature data with a uniform preset data amount can be temporarily stored in the second memory. When the temporarily stored data reaches a certain length, it is uniformly written back to the first memory, effectively reducing data writing operations, reducing the delay of image processing, and improving efficiency.

[0054] In some examples, the first memory can be a larger main memory, and the second memory can be a cache memory that can directly interact with the processing unit. By combining the first memory with a lower read / write speed and a larger capacity with the second memory with a higher read / write speed and a smaller capacity, hardware resources can be more efficiently utilized, improving image processing efficiency.

[0055] In some examples, the processing unit in the image processing device can be a graphics processing unit (GPU), a central processing unit (CPU), or various types of computing chips, computing arrays and other logical devices, which are not limited in this disclosure.

[0056] In some examples, the preset threshold may be pre-configured manually based on demand, for example, it may be set based on the maximum storage capacity of the second memory or based on the transmission bandwidth of the data transmission link between the first memory and the second memory.

[0057] In some examples, the preset threshold value of the data volume of the second feature data may be in units of images, for example, it may refer to feature data of N images, corresponding to N times the preset data volume.

[0058] In some examples, the second feature data may be composed of unit feature data of a preset size. In this case, the preset threshold may be based on unit feature data, meaning that the feature data of a single image can be split into different batches for processing. By further refining the granularity of data alignment and processing, processing efficiency can be improved.

[0059] According to some embodiments, method 200 further includes: in response to determining that a second process is to be performed on target feature data among the plurality of second feature data, determining the target feature data from the plurality of second feature data based on the preset data volume; and performing the second process based on the target feature data. Thus, while the plurality of second feature data are uniformly temporarily stored and transferred, the spliced ​​and transferred second feature data can be split based on the preset data volume to obtain the correct target feature data, thereby improving efficiency while ensuring the accuracy of image processing.

[0060] According to some embodiments, each element in the first feature data is in a first data format, the padding element is in a second data format, the bit width of the second data format is smaller than the bit width of the first data format, and wherein the first processing for each first feature data further includes: for each element in the first feature data, converting the element into a conversion element having the second data format to obtain third feature data, wherein the splicing of the first feature data and the padding data to obtain the second feature data includes: splicing the third feature data and the padding data to obtain the second feature data. Thus, the first feature data can be further compressed and converted into data with a smaller bit width to improve image processing efficiency. It can be understood that, under the condition that the storage capacity of the second memory remains unchanged, by aligning and splicing the compressed feature data for temporary storage, more groups of feature data can be temporarily stored in the second memory to further reduce the number of write operations and improve image processing efficiency.

[0061] In some examples, the third feature data may be bit data, that is, the feature map is converted into a bit map to achieve data compression.

[0062] According to some embodiments, the first memory includes a first sub-unit and a second sub-unit, the first sub-unit storing the multiple images, and the reading of the multiple images arranged in a preset order from the first memory to write into the second memory includes: determining multiple sub-image data based on the multiple images, the multiple sub-image data including first sub-image data and second sub-image data; writing the first sub-image data into the second sub-unit so that the first sub-image data can be read from the second sub-unit; and writing the second sub-image data into the second sub-unit in response to determining that the first sub-image data has been read, and wherein, in response to determining that the data amount of the second feature data stored in the second memory reaches a preset threshold, moving the second feature data stored in the second memory to the first memory includes: in response to determining that the data amount of the second feature data stored in the second memory reaches a preset threshold, and in response to determining that the second processing is to be performed based on the second feature data stored in the second memory, moving the second feature data stored in the second memory to the second sub-unit.

[0063] This allows the first memory to be configured as a multi-level storage system, for example, comprising a first subunit and a second subunit. In this case, the image data to be processed needs to be moved step by step from the first subunit to the second subunit and then to the second memory. When it is determined that further processing of the second feature data is required, it can be temporarily stored in the second subunit, which is closer to the second memory, to reduce data transfer latency and further improve image processing efficiency.

[0064] According to some embodiments, transferring the second feature data stored in the second memory to the second sub-unit includes: in response to determining that a target storage location exists in the second sub-unit, transferring the second feature data stored in the second memory to the target storage location, wherein the target storage location does not include unread sub-image data. In this way, the second feature data can be temporarily stored in the target storage location while ensuring that the data in the target storage location has been read, thereby avoiding data loss and ensuring the accuracy of image processing.

[0065] According to some embodiments, each of the multiple images includes a first image channel and a second image channel, each of the multiple feature data includes first sub-feature data corresponding to the first image channel and second sub-feature data corresponding to the second image channel, and wherein determining the multiple sub-image data based on the multiple images includes: determining the first sub-image data based on the first image channel of each image in the multiple images; and determining the second sub-image data based on the second image channel of each image in the multiple images.

[0066] In the corresponding embodiment of the disclosure book, when the extraction of the first feature data is based on the image channel, the image data is divided according to the channel dimension, which is equivalent to performing the channel extraction operation between the first sub-unit and the second sub-unit, without the need for the processing unit to jump and read the channel data, thereby improving processing efficiency.

[0067] 3A-3B are schematic diagrams showing an image processing process according to an exemplary embodiment of the present disclosure.

[0068] In this example, the image processing method 200 may be applied during inference or training of an image processing model.

[0069] In some examples, the inference or training process of an image processing model can be represented as a directed acyclic computation graph consisting of multiple types of operators. These operators may include convolution, batch normalization, vector addition, activation, maximum value, and pooling.

[0070] In some examples, the computation graph frequently contains sequences consisting of the Batch Normalization operator, the Vector Addition operator, the Activation operator, and the Maximum operator. Constructing fused operators based on these sequences can reduce data read and write operations between operators and improve computation graph execution efficiency.

[0071] In some examples, the fusion operator may be called to perform calculations based on input task information during the inference or training process of the image processing model. For example, when the input task information indicates that calculations need to be performed based on a sequence consisting of a batch normalization operator, a vector addition operator, an activation operator, and a maximum value operator, the above four operators included in the fusion operator are called sequentially. When the input task information indicates that calculations only need to be performed based on a batch normalization operator and an activation operator, only the batch normalization operator and the activation operator in the fusion operator are called sequentially.

[0072] In some examples, the processing unit used to perform the image processing computation may include a cache, corresponding to the second memory described above that directly interacts with the processing unit. Furthermore, the first memory may be a multi-level storage system including a first sub-unit and a second sub-unit, corresponding to different levels of cache.

[0073] As shown in Figure 3A, the input image data can be divided according to the channel dimension. According to the size of the second sub-unit in the first memory, the sub-image data of multiple image channels are copied from the first sub-unit to the second sub-unit each time, so that the processing unit can read the data from the second sub-unit and perform preliminary calculations. When it is determined that the intermediate result obtained by the preliminary calculation still needs to participate in further remaining calculations, the intermediate result can be written back to the second sub-unit, and then the intermediate result can be directly read from the second sub-unit and the remaining calculations can be performed, without the need to perform data transfer from the first sub-unit to the second sub-unit, thereby improving image processing efficiency.

[0074] As shown in FIG3B , after extracting feature data of multiple images using a processing unit, the length alignment of the feature data can be achieved by splicing the first feature data of each image with its corresponding padding data, that is, obtaining second feature data with a uniform preset data amount.

[0075] In some examples, the first feature data may be an activation function feature map output by an activation operator. Furthermore, in one example, the first feature data may be further compressed, converting the entire activation function map into a bitmap to reduce data volume and improve image processing efficiency.

[0076] According to one aspect of the present disclosure, an image processing apparatus is also provided. FIG4 shows a structural block diagram of an image processing apparatus 400 according to an exemplary embodiment of the present disclosure. As shown in FIG4 , the apparatus 400 includes:

[0077] Processing unit 410;

[0078] The writing unit 420 is configured to read a plurality of images arranged in a preset order from the first memory and write the images into a second memory that can be read and written by the processing unit.

[0079] The processing unit 410 includes:

[0080] a reading subunit 411, configured to read the plurality of images from the second memory;

[0081] A first determining subunit 412 is configured to determine a plurality of first feature data corresponding to the plurality of images respectively;

[0082] The processing subunit 413 is configured to perform a first processing on each first feature data of the plurality of first feature data to obtain second feature data having a preset data amount, wherein the first processing includes: determining a first quantity based on the data amount of the first feature data and the preset data amount to obtain padding data consisting of a first number of padding elements; and concatenating the first feature data and the padding data to obtain the second feature data; and

[0083] The first writing sub-unit 414 is configured to write the plurality of second feature data corresponding to the plurality of first feature data into the second memory in sequence according to the preset order;

[0084] The transfer unit 430 is configured to transfer the second feature data stored in the second memory to the first memory in response to determining that the amount of the second feature data stored in the second memory reaches a preset threshold.

[0085] According to some embodiments, each element in the first feature data is in a first data format, the padding element is in a second data format, the bit width of the second data format is smaller than the bit width of the first data format, and wherein the first processing for each first feature data further includes: for each element in the first feature data, converting the element into a conversion element having the second data format to obtain third feature data, wherein the splicing of the first feature data and the padding data to obtain the second feature data includes: splicing the third feature data and the padding data to obtain the second feature data.

[0086] According to some embodiments, the first memory includes a first sub-unit and a second sub-unit, the first sub-unit stores the multiple images, and the writing unit 420 includes: a second determining sub-unit that determines multiple sub-image data based on the multiple images, and the multiple sub-image data include first sub-image data and second sub-image data; a second writing sub-unit that is configured to write the first sub-image data into the second sub-unit so that the first sub-image data can be read from the second sub-unit, and the second writing sub-unit is further configured to write the second sub-image data into the second sub-unit in response to determining that the first sub-image data has been read, and wherein the transporting unit 430 is configured to: in response to determining that the amount of second feature data stored in the second memory reaches a preset threshold, and in response to determining that the second processing is to be performed based on the second feature data stored in the second memory, transport the second feature data stored in the second memory to the second sub-unit.

[0087] According to some embodiments, the transport unit 430 is configured to: in response to determining that a target storage location exists in the second sub-unit, transport the second feature data stored in the second memory to the target storage location, wherein the target storage location does not include unread sub-image data.

[0088] According to some embodiments, each of the multiple images includes a first image channel and a second image channel, each of the multiple feature data includes first sub-feature data corresponding to the first image channel and second sub-feature data corresponding to the second image channel, and wherein the second determination subunit is configured to: determine the first sub-image data based on the first image channel of each image in the multiple images; and determine the second sub-image data based on the second image channel of each image in the multiple images.

[0089] According to some embodiments, the device 400 also includes: a determination unit, configured to, in response to determining that second processing is to be performed on target feature data among the multiple second feature data, determine the target feature data from the multiple second feature data based on the preset data amount, wherein the processing unit 410 is also configured to perform the second processing based on the target feature data.

[0090] It should be understood that the operations of the various units of the image processing apparatus 400 shown in FIG4 may correspond to the various steps in the image processing method 200 described in FIG2 . Therefore, the operations, features, and advantages described above for the method 200 are also applicable to the apparatus 400 and the various units included therein. For the sake of brevity, certain operations, features, and advantages are not described in detail herein.

[0091] According to one aspect of the present disclosure, a chip is further provided, including the image processing device 400 as described above.

[0092] According to one aspect of the present disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned image processing method.

[0093] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is further provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned image processing method.

[0094] According to one aspect of the present disclosure, a computer program product is further provided, comprising a computer program, wherein the computer program implements the above-mentioned image processing method when executed by a processor.

[0095] With reference to Figure 5, a block diagram of an electronic device 500 that can serve as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0096] As shown in Figure 5, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0097] Multiple components within device 500 are connected to I / O interface 505, including an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. Input unit 506 can be any device capable of inputting information into device 500. Input unit 506 can receive input numeric or character information and generate key signal input related to user settings and / or function control of the electronic device. It may include, but is not limited to, a mouse, keyboard, touch screen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 507 can be any device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 508 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 509 allows device 500 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks. It may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0098] The computing unit 501 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the image processing method. For example, in some embodiments, the image processing method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the image processing method described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the image processing method by any other appropriate means (e.g., by means of firmware).

[0099] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0100] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0103] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0104] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0105] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0106] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples. Various elements in the embodiments or examples may be omitted or replaced by their equivalent elements. In addition, each step may be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced by equivalent elements that appear after the present disclosure.

Claims

1. An image processing method performed by an image processing device, the image processing device comprising a processing unit, the method comprising: Reading a plurality of images arranged in a preset order from a first memory and writing them into a second memory that can be read and written by the processing unit; reading the plurality of images from the second memory using the processing unit; Determine a plurality of first feature data respectively corresponding to the plurality of images; Performing a first process on each of the plurality of first feature data to obtain second feature data having a preset data amount, wherein the first process comprises: Determine a first quantity based on the data quantity of the first characteristic data and the preset data quantity to obtain filling data consisting of the first quantity of filling elements; and Concatenating the first characteristic data and the filling data to obtain the second characteristic data; Writing the plurality of second feature data respectively corresponding to the plurality of first feature data into the second memory in sequence according to the preset order; and In response to determining that the amount of the second feature data stored in the second memory reaches a preset threshold, the second feature data stored in the second memory is moved to the first memory.

2. The method of claim 1, wherein: Each element in the first feature data is in a first data format, the filling element is in a second data format, the bit width of the second data format is smaller than the bit width of the first data format, and wherein the first processing for each first feature data further includes: For each element in the first characteristic data, convert the element into a conversion element having the second data format to obtain third characteristic data, The step of concatenating the first feature data and the filling data to obtain the second feature data includes: The third characteristic data and the filling data are concatenated to obtain the second characteristic data.

3. The method according to claim 1 or 2, wherein: The first memory includes a first subunit and a second subunit, the first subunit stores the plurality of images, and the reading the plurality of images arranged in a preset order from the first memory to write into the second memory includes: Based on the plurality of images, determining a plurality of sub-image data, the plurality of sub-image data comprising first sub-image data and second sub-image data; writing the first sub-image data into the second sub-unit so that the first sub-image data can be read from the second sub-unit; and In response to determining that the first sub-image data has been read, writing the second sub-image data into the second sub-unit, And wherein, in response to determining that the amount of the second characteristic data stored in the second memory reaches a preset threshold, moving the second characteristic data stored in the second memory to the first memory includes: In response to determining that the amount of second feature data stored in the second memory reaches a preset threshold, and in response to determining that the second processing is to be performed based on the second feature data stored in the second memory, the second feature data stored in the second memory is transferred to the second subunit.

4. The method of claim 3, wherein: The transferring the second feature data stored in the second memory to the second subunit comprises: In response to determining that a target storage location exists in the second subunit, the second feature data stored in the second memory is moved to the target storage location, wherein the target storage location does not include unread sub-image data.

5. The method according to claim 3 or 4, wherein: Each of the plurality of images includes a first image channel and a second image channel, each of the plurality of feature data includes first sub-feature data corresponding to the first image channel and second sub-feature data corresponding to the second image channel, And wherein, based on the multiple images, determining multiple sub-image data comprises: determining the first sub-image data based on a first image channel of each image in the plurality of images; and The second sub-image data is determined based on a second image channel of each image in the plurality of images.

6. The method according to any one of claims 1 to 5, further comprising: In response to determining that a second process is to be performed on target feature data among the plurality of second feature data, determining the target feature data from the plurality of second feature data based on the preset data amount; as well as The second process is performed based on the target feature data.

7. An image processing device, comprising: Processing unit; a writing unit configured to read a plurality of images arranged in a preset order from the first memory and write the images into a second memory that can be read and written by the processing unit, Wherein, the processing unit includes: a reading subunit, configured to read the plurality of images from the second memory; A first determining subunit is configured to determine a plurality of first feature data corresponding to the plurality of images respectively; a processing subunit configured to perform a first process on each of the plurality of first feature data to obtain second feature data having a preset data amount, The first process comprises: Determine a first quantity based on the data quantity of the first characteristic data and the preset data quantity to obtain filling data consisting of the first quantity of filling elements; and Concatenating the first characteristic data and the filling data to obtain the second characteristic data; A first writing subunit is configured to write a plurality of second feature data corresponding to the plurality of first feature data into the second memory in sequence according to the preset order; and The transfer unit is configured to transfer the second feature data stored in the second memory to the first memory in response to determining that the data amount of the second feature data stored in the second memory reaches a preset threshold.

8. The device according to claim 7, wherein: Each element in the first feature data is in a first data format, the filling element is in a second data format, the bit width of the second data format is smaller than the bit width of the first data format, and wherein the first processing for each first feature data further includes: For each element in the first characteristic data, convert the element into a conversion element having the second data format to obtain third characteristic data, The step of concatenating the first feature data and the filling data to obtain the second feature data includes: The third characteristic data and the filling data are concatenated to obtain the second characteristic data.

9. The device according to claim 7 or 8, wherein: The first memory includes a first subunit and a second subunit, the first subunit stores the plurality of images, and the writing unit includes: The second determining subunit determines a plurality of sub-image data based on the plurality of images, wherein the plurality of sub-image data includes first sub-image data and second sub-image data; a second writing subunit, configured to write the first sub-image data into the second subunit so that the first sub-image data can be read from the second subunit, The second writing sub-unit is further configured to write the second sub-image data into the second sub-unit in response to determining that the first sub-image data has been read. And wherein, the transport unit is configured as: In response to determining that the amount of second feature data stored in the second memory reaches a preset threshold, and in response to determining that the second processing is to be performed based on the second feature data stored in the second memory, the second feature data stored in the second memory is transferred to the second subunit.

10. The device of claim 9, wherein: The transport unit is configured as follows: In response to determining that a target storage location exists in the second subunit, the second feature data stored in the second memory is moved to the target storage location, wherein the target storage location does not include unread sub-image data.

11. The device according to claim 9 or 10, wherein: Each of the plurality of images includes a first image channel and a second image channel, each of the plurality of feature data includes first sub-feature data corresponding to the first image channel and second sub-feature data corresponding to the second image channel, And wherein the second determining subunit is configured as: determining the first sub-image data based on a first image channel of each image in the plurality of images; and The second sub-image data is determined based on a second image channel of each image in the plurality of images.

12. The apparatus according to any one of claims 7 to 11, further comprising: a determining unit configured to, in response to determining that a second process is to be performed on target feature data among the plurality of second feature data, determine the target feature data from the plurality of second feature data based on the preset data amount, Wherein, the processing unit is further configured to perform the second processing based on the target feature data.

13. A chip comprising the image processing device according to any one of claims 7 to 12.

14. An electronic device, comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-6.

16. A computer program product comprising a computer program, wherein: The computer program implements the method according to any one of claims 1 to 6 when executed by a processor.

Images