Graph data processing method and device, electronic equipment and storage medium

Abstract

An embodiment of the present application provides a method, apparatus, electronic device, and storage medium for processing graph data, which relate to the fields of artificial intelligence and cloud technology. The method includes: obtaining at least one graph to be processed and relevant information of the at least one graph to be processed; obtaining a first quantity of multiple candidate values and device resource information of an electronic device; determining the device resource utilization rate corresponding to each candidate value according to the device resource information and the relevant information, and determining a first quantity from each candidate value according to the device resource utilization rate corresponding to each candidate value; determining a second quantity according to the first quantity and graph size information, and starting the second quantity of threads; for each thread, obtaining pixel data of an original pixel point corresponding to the thread in at least one graph to be processed through the thread, and performing pixel interpolation calculation according to the pixel data of the original pixel point to obtain pixel data of each target pixel point in a target graph corresponding to the original pixel point. In the present application, the data processing efficiency can be effectively improved.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and cloud technology, and more specifically, to a method, apparatus, electronic device, and storage medium for processing graph data. Background Technology

[0002] Graph data upscaling is a common operation in deep learning computer vision. In many applications, graph data needs to be enlarged to a fixed size to align with the model's input size, ensuring successful inference or meeting training input requirements. For example, in OCR (Optical Character Recognition), training sample images need to be enlarged to a certain size before being input into the model, while the recognition stage requires proportionally enlarging the image to be recognized to a fixed size. In deep learning models, upsampling operations are used to enlarge feature maps, generating feature maps at multiple scales to enhance the model's detection accuracy for objects of different sizes. However, current graph data upscaling methods suffer from excessive processing time and low efficiency. Summary of the Invention

[0003] This application provides a method, apparatus, electronic device, and storage medium for processing graph data, which can improve data processing efficiency and increase the resource utilization of the equipment.

[0004] On one hand, embodiments of this application provide a method for processing graph data, which is executed by an electronic device and includes:

[0005] Obtain at least one image to be processed and at least one related information about the image to be processed. The related information includes the image data format and image information. The image information includes the number of images, the number of channels of the image to be processed, and the image size information. The image size information includes the width and height of the image to be processed, as well as the width and height of the target image corresponding to the image to be processed.

[0006] Obtain a first number of candidate values ​​and device resource information of the electronic device, where the first number is the number of pixels processed by one thread of the electronic device;

[0007] Based on equipment resource information and related information, determine the equipment resource utilization rate corresponding to each candidate value, and determine the first quantity from each candidate value based on the equipment resource utilization rate corresponding to each candidate value.

[0008] Based on the first quantity and relevant information, determine the second number of threads required to process at least one graph to be processed, and start the second number of threads;

[0009] For each thread, the pixel data of the original pixel corresponding to the thread in at least one image to be processed is obtained through the thread, and interpolation calculation is performed based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel. The number of each target pixel is the first quantity.

[0010] On the other hand, embodiments of this application provide a graph data processing apparatus, the apparatus comprising:

[0011] The graph data acquisition module is used to acquire at least one graph to be processed and at least one related information of the graph to be processed. The related information includes the graph data format and graph information. The graph information includes the number of graphs, the number of channels of the graph to be processed, and the graph size information. The graph size information includes the width and height of the graph to be processed, as well as the width and height of the target graph corresponding to the graph to be processed.

[0012] The first quantity determination module is used to obtain multiple candidate values ​​for the first quantity and device resource information of the electronic device. The first quantity is the number of pixels processed by one thread of the electronic device. Based on the device resource information and related information, the module determines the device resource utilization rate corresponding to each candidate value, and determines the first quantity from each candidate value based on the device resource utilization rate corresponding to each candidate value.

[0013] The second quantity determination module is used to determine, based on the first quantity and relevant information, the second number of threads required to process at least one graph to be processed, and to start the second number of threads;

[0014] The pixel interpolation calculation module is used to obtain the pixel data of the original pixel corresponding to the thread in at least one image to be processed for each thread, and perform interpolation calculation based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel, wherein the number of each target pixel is the first quantity.

[0015] Optionally, the graph data format is NCHW(batch_number, channel, height, width, batch number, channel, height, width) or NHWC(batch, height, width, channel, batch number, height, width, channel) format; the first quantity determination module, when obtaining multiple candidate values ​​for the first quantity, is specifically used for:

[0016] If the graph data format is NCHW, determine the first ratio between the height of the target graph and the height of the graph to be processed, and use each integer value that is not greater than the first ratio as multiple candidate values.

[0017] If the image data format is NHWC, determine the second ratio of the height of the target image to the height of the image to be processed, and the third ratio of the width of the target image to the width of the image to be processed; and take integer values ​​that are not greater than the product of the second ratio and the third ratio, and that are multiples of the third ratio, as multiple candidate values.

[0018] Optionally, the graph data format can be NCHW or NHWC.

[0019] The first quantity determination module, when determining the equipment resource utilization rate corresponding to each candidate value based on equipment resource information and related information, is specifically used for:

[0020] Based on the device resource information, determine the number of threads that the electronic device can start;

[0021] If the graph data format is NCHW, for each candidate value, the number of candidate threads corresponding to the candidate value is determined based on the height of the graph to be processed, the number of graphs, the width of the target graph, and the number of channels. The device resource rate corresponding to the candidate value is determined based on the number of candidate threads and the number of threads that can be started. The number of candidate threads refers to the number of threads required to process at least one graph to be processed if the first quantity is a candidate value.

[0022] If the graph data format is NHWC, for each candidate value, the number of candidate threads corresponding to the candidate value is determined based on the height and width of the graph to be processed, the number of graphs, and the number of channels. Based on the number of candidate threads and the number of threads that can be started, the device resource utilization rate corresponding to the candidate value is determined.

[0023] Optionally, the first quantity determination module, when determining the device resource utilization corresponding to each candidate value based on the number of candidate threads and the number of threads that can be started, is specifically used for:

[0024] Determine the absolute value of the difference between the number of candidate threads corresponding to the candidate value and the number of threads that can be started, and use the absolute value as the device resource utilization rate corresponding to the candidate value;

[0025] When determining the first quantity from each candidate value based on the equipment resource utilization rate corresponding to each candidate value, the first quantity determination module is specifically used for:

[0026] The candidate value corresponding to the smallest absolute value among all candidate values ​​is determined as the first quantity.

[0027] Optionally, the device resource information is the number of SMs (streaming multiprocessors) in the electronic device;

[0028] If the graph data format is NCHW, for any candidate value k, the device resource utilization rate corresponding to the candidate value is determined by the following expression:

[0029] cost=|out_width*in_height*batch_num*channel_num*k-512*sm_num*r|

[0030] Where cost is the device resource utilization rate, out_width is the width of the target graph, in_height is the height of the graph to be processed, channel_num is the number of channels, sm_num is the number of SMs, r is the preset value, batch_num is the number of graphs, out_width*in_height*batch_num*channel_num*k is the number of candidate threads corresponding to the candidate value k, and 512*sm_num*r is the number of threads that can be started.

[0031] If the graph data format is NHWC, for any candidate value k, the equipment resource utilization rate corresponding to the candidate value is determined by the following expression:

[0032] cost=|in_width*in_height*batch_num*channel_num*k-512*sm_num*r|

[0033] Where in_width is the width of the graph to be processed, in_width*in_height*batch_num*channel_num*k is the number of candidate threads corresponding to the candidate value k, and 512*sm_num*r is the number of threads that can be started.

[0034] Optionally, for each thread, when the pixel interpolation calculation module performs interpolation calculations based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel, it is specifically used for:

[0035] If the image data format is NCHW, bilinear interpolation is performed on the pixels in the height dimension based on the pixel data of the original pixels to obtain the pixel data of each target pixel in the height dimension of the target image corresponding to the original pixels.

[0036] If the image data format is NHWC, for each thread, based on the pixel data of the original pixel, bilinear interpolation calculation is performed on the height and width dimensions of the pixels to obtain the pixel data of each target pixel in the target image corresponding to the original pixel in the height and width dimensions.

[0037] Optionally, the apparatus further includes an information determination module for determining, before determining the first quantity from the candidate values, that at least one of the following graphs to be processed satisfies at least one of the following:

[0038] At least one image to be processed must satisfy at least one of the following:

[0039] The ratio of the height of the target image to the height of the image to be processed is greater than or equal to a first set value;

[0040] The ratio of the width of the target image to the width of the image to be processed is greater than or equal to the second set value;

[0041] The total number of pixels corresponding to at least one image to be processed is greater than or equal to the number of threads that the electronic device can start.

[0042] Optionally, for each thread, when the pixel interpolation calculation module obtains pixel data of the original pixel corresponding to the thread in at least one image to be processed, it is specifically used for:

[0043] For any two threads in the second number of threads, if there are identical original pixel data in the pixel data of the original pixel corresponding to the two threads, then obtain the pixel data of the original pixel corresponding to one of the two threads, transfer the pixel data of the identical original pixel in the obtained pixel data to the other thread, and obtain the pixel data of the other original pixel corresponding to the other thread.

[0044] Among them, the pixel data of other original pixels are the pixel data of the original pixels in the pixel data of the original pixels corresponding to another thread, excluding the pixel data of the same original pixels.

[0045] Optionally, the image to be processed can be an image or a feature map.

[0046] In another aspect, embodiments of this application provide an electronic device, including a processor and a memory: the memory is configured to store a computer program, which, when executed by the processor, causes the processor to perform a method for processing graph data.

[0047] In another aspect, embodiments of this application provide a computer-readable storage medium for storing a computer program that, when run on a computer, enables the computer to perform a method for processing graph data.

[0048] On the other hand, this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the graph data processing method provided in any of the optional embodiments of this application described above.

[0049] The beneficial effects of the technical solutions provided in this application are:

[0050] In this embodiment, the first number of pixels processed by a single thread can be determined based on the image data format and related information of the image to be processed. In this case, multiple pixels can be calculated in a single thread, which effectively improves data processing efficiency compared to the prior art where one thread calculates one pixel. Furthermore, since the first number is determined based on the resource utilization rate calculated from device resource information and related information, the resource utilization of the device is taken into account when determining the first number. Consequently, when determining the second number of threads for processing at least one image to be processed based on the first number and related information, the hardware computing resources of the electronic device can be fully utilized, reducing hardware costs. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0052] Figure 1 A schematic diagram of a drawing to be processed provided in an embodiment of this application;

[0053] Figure 2 A schematic flowchart illustrating a method for processing graph data provided in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram illustrating the principle of thread-based pixel data reading in an embodiment of this application.

[0055] Figure 4 A schematic flowchart illustrating another method for processing graph data provided in an embodiment of this application;

[0056] Figure 5a A schematic diagram of a drawing to be processed provided in an embodiment of this application;

[0057] Figure 5b A schematic diagram illustrating the principle of interpolation calculation based on pixel data of the original pixel points, provided in an embodiment of this application;

[0058] Figure 5c A schematic diagram illustrating another principle of interpolation calculation based on pixel data of the original pixel points, provided for an embodiment of this application;

[0059] Figure 6 A time consumption diagram corresponding to several different graph data processing methods provided in the embodiments of this application;

[0060] Figure 7 This is a schematic diagram of the structure of a graph data processing device provided in an embodiment of this application;

[0061] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0062] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0063] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0064] This application addresses the problems of low processing efficiency and low resource utilization in existing graph data magnification processing methods, and proposes a graph data processing method. In this application, the graph to be processed can be an image or a feature map. Based on the solution provided by this application, the interpolation calculation efficiency of graph data can be effectively improved, allowing a smaller graph to be processed to be processed into a larger target graph through interpolation calculation. The method provided by this application is applicable to any application scenario requiring graph data processing.

[0065] Bilinear interpolation is a commonly used method for graph data upscaling. Currently, conventional GPU (graphics processing unit) implementations of bilinear interpolation upscaling algorithms, such as TensorFlow (an end-to-end open-source machine learning platform), PyTorch (a machine learning library for applications like natural language processing), and TensorRT (a high-performance deep learning engine), can largely meet the processing requirements of graph data methods. However, the inventors of this application have discovered at least the following technical problems in existing graph data upscaling methods:

[0066] (1) Some input pixels (i.e., pixel data of the original image to be enlarged) are repeatedly accessed by different threads: In the prior art, each thread calculates one interpolation pixel. During the interpolation calculation, since some different interpolation pixels may require partially or even all of the same input pixels, these partially or even entirely identical input pixels will be repeatedly accessed by different threads. As the magnification factor of the image data increases, the amount of data access will increase significantly. Although GPUs have powerful parallel computing capabilities, the increase in computational load will lead to an excessive number of threads allocated, exceeding the limits of hardware resources. Some threads must wait for the active threads to finish their tasks before they can use hardware resources for computation, which results in these threads being executed serially with the active threads.

[0067] As shown in Figure 1 Figure 1 This is a portion of a magnified feature map, where the x-axis represents the width of the feature map, the y-axis represents the height, each square in the map represents an interpolated pixel (i.e., an interpolated pixel obtained through interpolation), and F(xi, yj) (i, j = 1, 2) represents the input pixels (shown by the white squares in the map, i.e., the pixels in the feature map before magnification). Using existing technology, each thread calculates one interpolated pixel, resulting in 25 pixels distributed across 25 threads. The pixel data for these 25 pixels is calculated based on the pixel data of the 4 pixels shown in the map, which are repeatedly accessed by the 25 threads. Therefore, the total data access frequency for F(xi, yj) is 25 times. If both the image width and length increase by a factor of n, the theoretical data access frequency will be n*n times. As n increases, the GPU may experience excessive thread allocation, exceeding its hardware resource limits.

[0068] (2) Existing solutions are limited in their applicable scenarios and cannot achieve their full performance in different hardware or image magnification scenarios. For example, TensorRT uses the TLP (Thread-Level Parallelism) method, which may have insufficient computing resources when the magnification factor is large. PyTorch uses the ILP (Instruction-Level Parallelism) parallel method, which may have problems with GPU computing resource utilization and low processing efficiency, and cannot meet the application scenarios with high timeliness requirements.

[0069] To address at least one of the problems existing in the prior art, the image data processing method provided in this application, considering the resource utilization of electronic devices, can calculate multiple pixels in a single thread, effectively improving data processing efficiency. This method can be executed by any electronic device, which can be a server or a terminal device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, tablet computer, laptop computer, desktop computer, etc., but is not limited thereto, and this application does not impose any limitations thereon.

[0070] Optionally, the electronic device can be a cloud device, and the data processing / computation involved in this embodiment can be implemented based on cloud technology. For example, the step of performing pixel interpolation calculation based on the pixel data of the original pixel corresponding to the thread to obtain the pixel data of each target pixel in the target image corresponding to the original pixel can be implemented using cloud computing.

[0071] Cloud computing is a computing model that distributes computing tasks across a resource pool composed of numerous computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" are infinitely scalable, readily available, and can be used on demand, expanded at any time, and paid for based on usage. As the foundational providers of cloud computing capabilities, they establish cloud resource pools (or cloud platforms, generally referred to as IaaS (Infrastructure as a Service) platforms), deploying various types of virtual resources within these pools for external customers to choose from. The cloud resource pool primarily includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0072] Optionally, the method provided in this application embodiment also relates to the field of computer vision in artificial intelligence. The solution provided in this application embodiment is applicable to the upsampling (i.e., upsampling) of feature maps in the field of artificial intelligence.

[0073] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0074] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0075] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0076] Figure 2 This illustration shows a flowchart of a graph data processing method provided in an embodiment of this application. The method includes:

[0077] Step S201: Obtain at least one image to be processed and at least one related information of the image to be processed. The related information includes the image data format and image information. The image information includes the number of images, the number of channels of the image to be processed, and the image size information. The image size information includes the width and height of the image to be processed, and the width and height of the target image corresponding to the image to be processed.

[0078] The image to be processed refers to image data whose size information needs to be changed. This image can be a feature map or a picture (image), etc. This embodiment does not limit the type of image to be processed. Related information refers to information about the image to be processed, which may include the image data format (e.g., whether it is NCHW or NHWC format) and image information. The image information may specifically include the number of images to be processed, the number of channels, and image size information. The image size information may include the width and height of the image to be processed, and the width and height of the target image corresponding to the image to be processed. The width and height of the target image refer to the width and height of the processed image after magnification, which is the final size of the target image. When the image to be processed is an image, the number of channels may refer to the number of color channels in the image. For example, when the image to be processed is an RGB (Red, Green, Blue) image, the number of channels is 3. Optionally, when the image to be processed is a feature map, the number of channels may refer to the number of channels in the feature map output by the feature extraction module.

[0079] Step S202: Obtain a first number of candidate values ​​and device resource information of the electronic device, wherein the first number is the number of pixels processed by one thread of the electronic device.

[0080] Step S203: Determine the resource utilization rate corresponding to each candidate value based on the equipment resource information and related information, and determine the first quantity from each candidate value based on the resource utilization rate corresponding to each candidate value.

[0081] In this context, a thread is the smallest unit of computation that an operating system can schedule. It is contained within a process and is the actual unit of operation within that process. In this embodiment, device resource information refers to information about the resources in an electronic device that can be used to process the image to be processed. Optionally, the acquired device resource information may be the number of SMs (Streaming Service Controllers) in the GPU. Multiple candidate values ​​are used to determine a first quantity, which refers to the number of pixels processed by a thread used to process at least one image to be processed. This embodiment does not limit the method for acquiring multiple candidate values; for example, each candidate value may be an empirical value or an experimental value. For each candidate value, the resource utilization rate corresponding to each candidate value can be determined based on the acquired device resource information and related information. The resource utilization rate corresponding to each candidate value refers to the degree of resource utilization in the electronic device when that candidate value is selected as the first quantity. Accordingly, after obtaining the resource utilization rates corresponding to each candidate value, the first quantity can be determined from each candidate value based on the resource utilization rates corresponding to each candidate value.

[0082] Step S204: Based on the first quantity and relevant information, determine the second number of threads required to process at least one graph to be processed, and start the second number of threads;

[0083] In this embodiment of the application, the total number of pixels when processing the at least one image to be processed can be known based on the relevant information obtained. And when the number of pixels processed by a thread is known, the total number of threads required to process the at least one image to be processed (i.e., the second number) can be obtained based on the known total number. Then, the second number of threads can be started to process the at least one image to be processed.

[0084] Step S205: For each thread, obtain the pixel data of the original pixel corresponding to the thread in at least one image to be processed, and perform interpolation calculation based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel. The number of each target pixel is the first quantity.

[0085] After the threads are started, each thread can have a corresponding original pixel. At this time, for each thread, the thread can obtain the pixel data of its own original pixel, and then perform pixel interpolation calculation based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel. When all threads are completed, the target image can be obtained based on the interpolation result of the pixel data obtained by each thread.

[0086] Here, the original pixel refers to a pixel in at least one image to be processed, which is the input pixel during interpolation calculations. It can be understood that for an image, pixel data refers to the pixel value; for a feature map, a pixel is a feature point in the feature map, and its pixel data is the feature value of that feature point.

[0087] For any original pixel, the pixel data of the original pixel refers to the pixel value corresponding to that pixel in the image to be processed. For example, when the image to be processed is an RGB image, the number of channels in the image is 3. The pixel data of an original pixel in the 3 channels refers to the pixel value of that pixel in that channel. When the image to be processed is a feature map, the pixel data of a pixel in the feature map of a channel refers to the pixel value (i.e., feature value) of that pixel in the feature map of that channel.

[0088] In this embodiment, the first number of pixels processed by a single thread can be determined based on the image data format and related information of the image to be processed. In this case, multiple pixels can be calculated in a single thread, which effectively improves data processing efficiency compared to the prior art where one thread calculates one pixel. Furthermore, since the first number is determined based on the resource utilization rate calculated from device resource information and related information, the resource utilization of the device is taken into account when determining the first number. Consequently, when determining the second number of threads for processing at least one image to be processed based on the first number and related information, the hardware computing resources of the electronic device can be fully utilized, reducing hardware costs.

[0089] In optional embodiments of this application, the graph data format is NCHW format or NHWC format;

[0090] Obtain a first number of candidate values, including:

[0091] If the graph data format is NCHW, determine the first ratio between the height of the target graph and the height of the graph to be processed, and use each integer value that is not greater than the first ratio as multiple candidate values;

[0092] If the image data format is NHWC, determine the second ratio of the height of the target image to the height of the image to be processed, and the third ratio of the width of the target image to the width of the image to be processed; select integer values ​​that are not greater than the product of the second and third ratios and are multiples of the third ratio as multiple candidate values.

[0093] In this embodiment, the image data format of the image to be processed can be NCHW format or NHWC format. Since the storage method of pixel data of the pixels in the image to be processed is different for different image data formats, for example, in at least one image to be processed in NCHW format, the pixel data of the pixels in each channel are adjacent to each other, while in at least one image to be processed in NHWC format, the pixel data of the pixels in each channel are spaced apart. Considering this, when determining the first quantity, the determination method of each candidate value of the first quantity is different for different image data formats, so as to improve the data processing efficiency while improving the device resource utilization.

[0094] If the graph data format is NCHW, a first ratio of the height of the target graph to the height of the graph to be processed can be determined, and integer values ​​not greater than the first ratio can be used as multiple candidate values.

[0095] In one example, assume that the device resource information is the number of SMs in the GPU (represented by sm_num), the height of the image to be processed is in_height, the height of the target image is out_height, the width of the target image is out_width, and the number of channels is channel_num. In this case, the first ratio ah = out_height / in_height. At this time, each candidate value k can be an integer value not greater than ah, such as k = 1, ..., ah, and must be an integer.

[0096] When the image data format is NHWC, a second ratio of the height of the target image to the height of the image to be processed, and a third ratio of the width of the target image to the width of the image to be processed can be determined. Integer values ​​that are not greater than the product of the second and third ratios and are multiples of the third ratio are used as multiple candidate values.

[0097] In one example, assume the device resource information is the number of SMs in the GPU (represented by sm_num), the height of the image to be processed is in_height, the width of the image to be processed is in_width, the height of the target image is out_height, the width of the target image is out_width, and the number of channels is channel_num. In this case, the second ratio ah = out_height / in_height, the third ratio aw = out_width / in_width, and the product of the second and third ratios is ah*aw. At this time, each candidate value k can be an integer that is a multiple of aw, such as k = aw, ..., i*aw, ..., ah*aw, where the value of i is an integer from 1 to ah.

[0098] In optional embodiments of this application, the graph data format is NCHW format or NHWC format;

[0099] Based on equipment resource information and related information, determine the equipment resource utilization rate corresponding to each candidate value, including:

[0100] Based on the device resource information, determine the number of threads that the electronic device can start;

[0101] If the graph data format is NCHW, for each candidate value, the number of candidate threads corresponding to the candidate value can be determined based on the height of the graph to be processed, the number of graphs, the width of the target graph, and the number of channels. The device resource rate corresponding to the candidate value can be determined based on the number of candidate threads and the number of threads that can be started. The number of candidate threads refers to the number of threads required to process at least one graph to be processed if the first quantity is a candidate value.

[0102] If the graph data format is NHWC, for each candidate value, the number of candidate threads corresponding to the candidate value is determined based on the height and width of the graph to be processed, the number of graphs, and the number of channels. Based on the number of candidate threads and the number of threads that can be started, the device resource utilization rate corresponding to the candidate value is determined.

[0103] Optionally, the number of threads that an electronic device can initiate can be understood as the maximum number of threads that the electronic device can use to process at least one type of graph to be processed, that is, the number of threads that the electronic device can initiate. As described above, since the storage and retrieval methods of pixel data of the graphs to be processed are different for different graph data formats, the methods for determining the number of candidate threads corresponding to each candidate value are different in determining the device resource utilization rate corresponding to each candidate value.

[0104] Optionally, when the graph data format of the graph to be processed is NCHW format, for each candidate value, the number of threads required to process at least one graph to be processed (i.e., the number of graphs) can be determined based on the height of the graph to be processed, the number of graphs to be processed (i.e., the number of graphs), the width of the target graph, and the number of channels. Then, based on the number of candidate threads and the determined number of threads that can be started, the device resource rate corresponding to the candidate value can be determined.

[0105] Optionally, if the graph data format of the graph to be processed is NHWC format, for each candidate value, the number of candidate threads when using the candidate value as the first quantity can be determined based on the height and width of the graph to be processed, the number of graphs to be processed (i.e., the number of graphs) and the number of channels. Then, based on the number of candidate threads and the determined number of threads that can be started, the device resource rate corresponding to the candidate value can be determined.

[0106] In an optional embodiment of this application, for each candidate value, the device resource utilization rate corresponding to the candidate value is determined based on the number of candidate threads and the number of threads that can be started, including:

[0107] Determine the absolute value of the difference between the number of candidate threads corresponding to the candidate value and the number of threads that can be started, and use the absolute value as the device resource utilization rate corresponding to the candidate value;

[0108] Based on the equipment resource utilization rate corresponding to each candidate value, a first quantity is determined from each candidate value, including:

[0109] The candidate value corresponding to the smallest absolute value among all candidate values ​​is determined as the first quantity.

[0110] Optionally, knowing the number of candidate threads and the number of threads that can be started when using each candidate value as the first quantity, for each candidate value, the absolute value of the difference between the number of candidate threads and the number of threads that can be started can be determined, and this absolute value can be used as the device resource utilization rate corresponding to that candidate value. The smaller the absolute value, the closer the number of threads that need to be started when using that candidate value as the first quantity is to the number of threads that the electronic device can start, and the higher the utilization rate of the electronic device's resources. Optionally, to maximize the utilization of the electronic device's resources, the candidate value with the smallest absolute value among all candidate values ​​can be determined as the first quantity.

[0111] In an optional embodiment of this application, the device resource information is the number of SMs in the electronic device;

[0112] If the graph data format is NCHW, for any candidate value k, the equipment resource utilization rate corresponding to each candidate value is determined by the following expression:

[0113] cost=|out_width*in_height*batch_num*channel_num*k-512*sm_num*r|

[0114] Where cost is the device resource utilization rate, out_width is the width of the target graph, in_height is the height of the graph to be processed, channel_num is the number of channels, SM_NUM is the number of SMs, r is the preset value, batch_num is the number of graphs, out_width*in_height*batch_num*channel_num*k is the number of candidate threads corresponding to the candidate value k, and 512*sm_num*r is the number of threads that can be started.

[0115] In one example, if the graph data format is NCHW, assuming the device resource information is the number of SMs in the GPU (i.e., sm_num), r is preset to 2, the height of the graph to be processed is in_height, the height of the target graph is out_height, the width of the target graph is out_width, the number of channels is channel_num, and the number of graphs to be processed is batch_num, then the first ratio ah = out_height / in_height can be determined. Each candidate value k is an integer value not greater than ah, such as k = 1, ..., ah, and must be an integer. Further, the height of the graph to be processed, the width of the target graph, the number of channels, the device resource information, the number of graphs to be processed, and each k value can be substituted into the above formula. This yields the number of candidate threads and the number of threads that can be started for each candidate value. Furthermore, for each candidate value, the absolute value of the difference between the number of candidate threads and the number of threads that can be started can be determined. Then, the candidate value with the smallest absolute value among all the candidate values ​​is determined as the first quantity.

[0116] If the graph data format is NHWC, for any candidate value k, the equipment resource utilization rate corresponding to each candidate value is determined by the following expression:

[0117] cost=|in_width*in_height*batch_num*channel_num*k-512*sm_num*r|

[0118] Where cost is the device resource utilization rate, in_width is the width of the graph to be processed, in_height is the height of the graph to be processed, channel_num is the number of channels, sm_num is the number of SMs, r is the preset value, k is the candidate value, batch_num is the number of graphs to be processed, in_width*in_height*channel_num*k is the number of candidate threads corresponding to the candidate value, and 512*sm_num*r is the number of threads that can be started.

[0119] In one example, if the graph data format is NCHW, and the device resource information is the number of SMs in the GPU (i.e., sm_num), the height of the graph to be processed is in_height, the width of the graph to be processed is in_width, the height of the target graph is out_height, the width of the target graph is out_width, and the number of channels is channel_num, then we can determine the first ratio ah = out_height / in_height, the second ratio aw = out_width / in_width, and the product of the first ratio and the second ratio is ah*aw. At this time, each candidate value k can be an integer value not greater than ah*aw, such as k = aw, ..., i*aw, ..., ah*aw, where the value of i is an integer from 1 to ah. Furthermore, the height, width, number of channels, device resource information, and each k value of the image to be processed can be substituted into the above formula. This yields the number of candidate threads and the number of threads that can be started for each candidate value. Further, for each candidate value, the absolute value of the difference between the number of candidate threads and the number of threads that can be started can be determined. Then, the candidate value with the smallest absolute value among all the candidate values ​​is determined as the first quantity.

[0120] Optionally, in practical applications, if the number of images to be processed is one, then when determining the device resource utilization rate corresponding to each candidate value, batch_num is set to 1.

[0121] In an optional embodiment of this application, for each thread, interpolation calculation is performed based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel, including:

[0122] If the image data format is NCHW, bilinear interpolation is performed on the pixels in the height dimension based on the pixel data of the original pixels to obtain the pixel data of each target pixel in the height dimension of the target image corresponding to the original pixels.

[0123] If the image data format is NH WC, for each thread, based on the pixel data of the original pixel, bilinear interpolation calculation is performed on the height and width dimensions to obtain the pixel data of each target pixel in the target image corresponding to the original pixel in the height and width dimensions.

[0124] Optionally, when the image data format of the image to be processed is NCHW format, for each thread, after obtaining the pixel data of the original pixel corresponding to the thread in at least one image to be processed, bilinear interpolation calculation of the pixels can be performed in the height dimension based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel in the height dimension.

[0125] In one example, suppose a thread processes 4 pixels. For each thread, 4 pixels can be obtained by bilinear interpolation in the height dimension based on the pixel data of the original pixel corresponding to that thread. Thus, the height dimension changes from one pixel to four pixels, that is, from 1*1 (1 pixel * 1 pixel) to 1*4 (1 pixel * 4 pixels). The resulting target image has 4 target pixels in the height dimension corresponding to the original pixel.

[0126] Optionally, when the image data format of the image to be processed is NHWC format, for each thread, after obtaining the pixel data of the original pixel corresponding to the thread in at least one image to be processed, bilinear interpolation calculation of pixels can be performed simultaneously in the height and width dimensions of the image to be processed based on the pixel data of the original pixel to obtain the pixel data of each pixel in the height and width dimensions of the target image corresponding to the original pixel.

[0127] When the image data format of the image to be processed is NHWC format, bilinear interpolation calculation is performed on the height and width dimensions of the pixels based on the pixel data of the original pixels to obtain the pixel data of each target pixel in the height and width dimensions of the target image corresponding to the original pixels. The first number of pixel data of each target pixel refers to the first number of target pixels in the width dimension and the first number of target pixels in the height dimension.

[0128] In one example, assuming a thread processes 4 pixels, for each thread, bilinear interpolation can be performed on the original pixel data in both the height and width dimensions to obtain 4 pixels. This changes the number of pixels from one in the height dimension to four in the width dimension, i.e., from 1*1 (1 pixel * 1 pixel) to 4*4 (4 pixels * 4 pixels). The resulting target image has 4 target pixels corresponding to the original pixel in both the height and width dimensions.

[0129] In an optional embodiment of this application, before determining the first quantity from the candidate values, the method further includes:

[0130] At least one image to be processed must satisfy at least one of the following:

[0131] The ratio of the height of the target image to the height of the image to be processed is greater than or equal to a first set value;

[0132] The ratio of the width of the target image to the width of the image to be processed is greater than or equal to the second set value;

[0133] The total number of pixels corresponding to at least one image to be processed is greater than or equal to the number of threads that the electronic device can start.

[0134] Optionally, before determining the first number of pixels processed by a thread for processing at least one image to be processed, it can also be determined whether the at least one image to be processed meets a set condition. If the set condition is met, it indicates that the data computation for processing the at least one image to be processed is large. In this case, the at least one image to be processed can be processed based on the graph data processing method provided in this embodiment, thereby reducing the data computation and improving data processing efficiency. If the set condition is not met, other processing methods can be used to process the image to be processed, such as using Tensorflow or PyTorch.

[0135] It is understood that in practical applications, the above-described solution is an optional implementation method. It can be used to process the missing image when it is determined that at least one image to be processed meets the set conditions, or to process the missing image when it is determined that at least one image to be processed does not meet the set conditions. Alternatively, it can be used without making the above-described determination of whether at least one image to be processed meets the set conditions, that is, regardless of whether the conditions are met, the method provided in the embodiments of this application can be used to process the image to be processed.

[0136] Optionally, the setting conditions may include at least one of the following: the ratio of the height of the target image to the height of the image to be processed is greater than or equal to a first setting value; the ratio of the width of the target image to the width of the target image is greater than or equal to a second setting value; and the total number of pixels corresponding to at least one image to be processed is greater than or equal to the number of threads that the electronic device can start.

[0137] Specifically, when the ratio of the height of the target image to the height of the image to be processed is greater than or equal to a first preset value, it indicates that the target image is significantly enlarged in height compared to the image to be processed. Conversely, when the ratio of the width of the target image to the width of the target image is greater than or equal to a second preset value, it indicates that the target image is significantly enlarged in width compared to the image to be processed. Both of these scenarios involve a substantial amount of data computation. For an electronic device, the corresponding amount of data computation (i.e., the number of threads that can be started) is fixed and can be determined based on the device's resource information, such as the number of SMs included within the electronic device.

[0138] Optionally, in practical applications, when determining whether at least one image to be processed meets the set conditions, it can also be determined that at least one image to be processed must meet one of the above three conditions, or meet one of the other conditions. For example, when the total number of pixels corresponding to at least one image to be processed is greater than or equal to the number of threads that the electronic device can start, and the ratio of the height of the target image to the height of the image to be processed is greater than or equal to the first set value or the ratio of the width of the target image to the width of the image to be processed is greater than or equal to the second set value, it can be considered that at least one image to be processed meets the set conditions.

[0139] For example, ah represents the ratio of the height of the target image to the height of the image to be processed, aw represents the ratio of the width of the target image to the width of the image to be processed, out_size represents the total number of pixels corresponding to at least one image to be processed, 512*sm_num*r represents the number of threads that the electronic device can start, and the first and second settings are 2. At this time, at least one image to be processed can be considered to meet the setting conditions only when the total number of pixels corresponding to at least one image to be processed is greater than or equal to 512*sm_num*r, and ah is greater than or equal to 2 or aw is greater than or equal to 2 (i.e., (aw>=2||ah>=2)&&(out_size>512*sm_num*r)).

[0140] In an optional embodiment of this application, for each thread, obtaining pixel data of the original pixel corresponding to the thread in at least one image to be processed includes:

[0141] For any two threads in the second number of threads, if there are identical original pixel data in the pixel data of the original pixel corresponding to the two threads, then obtain the pixel data of the original pixel corresponding to one of the two threads, transfer the pixel data of the identical original pixel in the obtained pixel data to the other thread, and obtain the pixel data of the other original pixel corresponding to the other thread.

[0142] Among them, the pixel data of other original pixels are the pixel data of the original pixels in the pixel data of the original pixels corresponding to another thread, excluding the pixel data of the same original pixels.

[0143] In practical applications, the pixel data of the original pixel corresponding to each thread needs to be read from the memory of the electronic device. If the pixel data of the original pixel corresponding to each thread is read from the memory of the electronic device, there will be a large amount of data reading, which will affect the data processing efficiency. In order to further improve the data processing efficiency, in this embodiment of the application, for any two threads in the second number of threads, if there is the same pixel data of the original pixel corresponding to the two threads, the pixel data of the same original pixel can be transferred from one of the two threads to the other thread in the adjacent thread. At this time, for the other thread, it is not necessary to read the pixel data of the same original pixel from the memory of the electronic device again. It only needs to read the pixel data of the original pixel other than the pixel data of the same original pixel, thereby reducing the amount of data reading.

[0144] In one example, suppose as Figure 3 There are a total of 5 threads, namely threads ID0 to ID5 (thread identifiers). The pixel data of the original pixel corresponding to ID0 is x0 and x1, the pixel data of the original pixel corresponding to ID1 is x0 and x1, the pixel data of the original pixel corresponding to ID2 is x1 and x2, the pixel data of the original pixel corresponding to ID3 is x2 and x3, the pixel data of the original pixel corresponding to ID4 is x3 and x4, and the pixel data of the original pixel corresponding to ID5 is x4 and x5. At this time, ID0 can read x0 from the memory of the electronic device and use the shuffle instruction to transfer x0 to ID1 (as shown by the arrow from x0 to x1 in the figure). ID1 only needs to read x1 from the memory of the electronic device. Similarly, ID1 uses a shuffle instruction to transfer x1 to ID2 (as shown by the arrows from x1 to x2 in the figure). ID2 then only needs to read x2 from the memory of the electronic device. ID2 uses a shuffle instruction to transfer x2 to ID3 (as shown by the arrows from x2 to x3 in the figure). ID3 then only needs to read x3 from the memory of the electronic device. ID3 uses a shuffle instruction to transfer x3 to ID4 (as shown by the arrows from x3 to x4 in the figure). ID4 then only needs to read x4 from the memory of the electronic device. ID4 uses a shuffle instruction to transfer x4 to ID5 (as shown by the arrows from x4 to x5 in the figure). ID5 then only needs to read x5 from the memory of the electronic device.

[0145] Optionally, the method provided in this application embodiment can be applied to scenarios such as bilinear interpolation upscaling algorithms for large-scale images or feature maps. For example, before inputting data into a text recognition detection model, it is necessary to enlarge the image to a certain size (such as 960x960 (960 pixels x 960 pixels) or 480x480 (480 pixels x 480 pixels)). In this case, the image can be enlarged to a certain size based on the method provided in this application embodiment. Alternatively, in some object detection algorithms, in order to improve the model's ability to detect objects of different sizes, it is necessary to use a bilinear interpolation algorithm to perform upsampling processing to generate multi-scale feature maps. In this case, the upsampling processing can also be performed based on the method provided in this application embodiment to generate multi-scale feature maps.

[0146] Optional, such as Figure 4 As shown, in order to better understand the method provided in the embodiments of this application, the method will be described in detail below with reference to specific embodiments.

[0147] Step S401: Obtain device resource information of the electronic device.

[0148] Specifically, the GPU model of the electronic device can be obtained, and the corresponding number of SMs can be determined based on the obtained GPU model.

[0149] Step S402: Obtain at least one image to be processed and related information of at least one image to be processed.

[0150] Optionally, the relevant information includes the graph data format and graph information. The graph information includes the number of graphs, the number of channels of the graph to be processed, and the graph size information. The graph size information includes the width and height of the graph to be processed, as well as the width and height of the target graph corresponding to the graph to be processed.

[0151] Step S403: Determine whether the acquired at least one image to be processed meets the requirements of the large image calculation scenario. If yes, proceed to step S404; otherwise, proceed to step S407.

[0152] Wherein, at least one image to be processed conforms to the large image calculation scenario, it can mean that at least one image to be processed satisfies at least one of the following: the ratio of the height of the target image to the height of the image to be processed is greater than a first set value; the ratio of the width of the target image to the width of the target image is greater than or equal to a second set value; and the total number of pixels corresponding to at least one image to be processed is greater than or equal to the number of threads that the electronic device can start.

[0153] Step S404: Obtain a first quantity of multiple candidate values ​​and device resource information of the electronic device, and determine the first quantity from each candidate value based on the device resource information and related information.

[0154] Optionally, the number of threads that the electronic device can start can be determined based on the device resource information. When the image data format of at least one image to be processed is NCHW format, a first ratio between the height of the target image and the height of the image to be processed can be determined. Integer values ​​that are not greater than the first ratio are taken as candidate values. For each candidate value, the number of candidate threads corresponding to the candidate value can be determined based on the height of the image to be processed, the number of images, the width of the target image, and the number of channels. Based on the absolute value of the difference between the number of candidate threads corresponding to the candidate value and the number of threads that can be started, the candidate value with the smallest absolute value among the absolute values ​​corresponding to each candidate value is determined as the first quantity.

[0155] When at least one image to be processed has an NHWC data format, a second ratio of the height of the target image to the height of the image to be processed and a third ratio of the width of the target image to the width of the image to be processed can be determined. Integer values ​​that are no greater than the product of the second and third ratios and are multiples of the third ratio are taken as multiple candidate values. For each candidate value, the number of candidate threads corresponding to the candidate value is determined based on the height and width of the image to be processed, the number of images, and the number of channels. Then, the absolute value of the difference between the number of candidate threads and the number of threads that can be started is determined. The candidate value with the smallest absolute value among the absolute values ​​corresponding to each candidate value is determined as the first quantity.

[0156] Step S405: Based on the first quantity and relevant information, determine the second quantity of threads for processing at least one graph to be processed, and start the second quantity of threads.

[0157] Step S406: For each thread, obtain the pixel data of the original pixel corresponding to the thread in at least one image to be processed through the thread, and perform pixel interpolation calculation based on the pixel data of the original pixel to obtain the pixel data of each pixel in the target image corresponding to the original pixel.

[0158] Specifically, assuming the number of graphs to be processed is 1, i.e., batch_num = 1, the graphs to be processed are as follows: Figure 5a The image shown has a width and height of 3 (i.e., a size of 3 pixels * 3 pixels). Figure 5a The numbers 0-8 in the image represent the identifiers of the nine pixels in the image. The pixel data of each original pixel in one channel of the image is the pixel value of these nine pixels in that channel. For images with different image data formats, the method provided in this application can be used to perform bilinear interpolation calculation based on the pixel data of the original pixels to obtain the pixel data of each pixel in the target image corresponding to the original pixels. The following describes the method of obtaining the pixel data of each pixel in the target image corresponding to the original pixels by performing pixel interpolation calculation based on the pixel data of the original pixels when the image data formats are NCHW or NHWC.

[0159] (1) For graph data in NCHW format:

[0160] For a thread (denoted as thread 1), assuming the original pixels corresponding to this thread are pixel 0, pixel 1, pixel 3, and pixel 4, the pixel data of these four pixels can be read into this thread. Then, this thread can perform pixel interpolation calculations in the H dimension (i.e., the height dimension) based on the pixel data of these four pixels. The target pixels in the target image obtained based on this thread are as follows: Figure 5b As shown in the dashed box of thread 1, the number of target pixels obtained is 3 (i.e., k=3). Similarly, for the next thread (i.e., thread 2), since the four adjacent original pixels of thread 2 are still pixel 0, pixel 1, pixel 3, and pixel 4, the pixel data of pixel 0, pixel 1, pixel 3, and pixel 4 can be obtained. Thread 2 can perform pixel interpolation calculations in the H dimension (i.e., the height dimension) based on the pixel data of these four pixels. At this time, the pixel data of each target pixel in the target image obtained by this thread is as follows: Figure 5b The dashed box indicates thread 2. Following the same logic, the target graph (as shown in the image) can be obtained after all threads have completed their calculations. Figure 5b (As shown in the overall diagram).

[0161] Optionally, in practical applications, when processing at least one image to be processed (which can be divided into multiple batches of data) in NCHW format, for a single thread, assuming the thread's ID index is global_index, the relevant information of the original pixel corresponding to that thread can be determined using the following expressions:

[0162] h_scale=in_height / out_height

[0163] w_scale = in_width / out_width

[0164] batch_id=global_index / (out_width*in_height*channel_num*k)

[0165] per_batch_index=global_index%(out_width*in_height*channel_num*k);

[0166] channel_id=per_batch_index / (out_width*in_height*k)

[0167] per_channel_index=per_batch_index%(out_width*in_height*k)

[0168] h_id=per_channel_index / out_width

[0169] w_id=per_channel_index%out_width

[0170] start_id=floor(h_id / h_scale)

[0171] end_id=floor((h_id+1) / h_scale)

[0172] Wherein, `in_height` represents the height of the image to be processed, `out_height` represents the height of the target image, `in_width` represents the width of the image to be processed, `out_width` represents the width of the target image, `k` represents the number of pixels processed by this thread (the first quantity), `channel_num` represents the number of channels in the image to be processed, `out_width * in_height * channel_num * k` represents the number of threads required for one image to be processed in the second quantity of threads, and `batch_id` represents the identifier of the batch data to which the thread with the thread index `global_index` belongs, that is, which batch of image data the original pixel data (hereinafter referred to as input data) needs to be processed by this thread belongs to. The pixel data is defined as follows: per_batch_index represents the index of the image to be processed corresponding to the input data of this thread, that is, which image in the batch data identified by batch_id belongs to the image to be processed corresponding to this thread; channel_id represents the identifier of the channel to which the input data of this thread belongs, that is, which channel of the image identified by per_batch_index is the pixel data of this input data; per_channel_index represents the index of the input data of this thread in the corresponding channel of the image; start_id represents the identifier of the starting pixel of each target pixel corresponding to this thread; and end_id represents the identifier of the last pixel of each target pixel corresponding to this thread.

[0173] Furthermore, based on the above relevant information, the coordinate information of each original pixel point required for the interpolation calculation of this thread can be obtained according to the principle of bilinear interpolation algorithm. It is assumed that the coordinates of the four original pixels corresponding to this thread are (h1, w1), (h1, w1+w1p), (h1+h1p, w1) and (h1+h1p, w1+w1p).

[0174] Furthermore, the pixel data of the four original pixels can be read into registers X0, X1, X2, and X3. Based on the coordinates of each original pixel, interpolation weights are calculated, including weights w0lambda and w1lambda for the width dimension and h0lambda and h1lambda for the height dimension. Based on the pixel data of each original pixel and the interpolation weights, the pixel data of each target pixel (i.e., the interpolation result) can be calculated. Optionally, example code for calculating the interpolation weights and interpolation results can be provided below:

[0175] X0=X[idx(batch_id,channel_id,in_height,in_width,h1,w1)];

[0176] X1=X[idx(batch_id,channel_id,in_height,in_width,h1,w1+w1p)];

[0177] X2=X[idx(batch_id,channel_id,in_height,in_width,h1+h1p,w1)];

[0178] X3=X[idx(batch_id,channel_id,in_height,in_width,h1+h1p,w1+w1p)];

[0179] for int i = start_id; i <end_id;i++:

[0180] h1,h1p,h0lambda,h1lambda=bilinear_calcute(i,h_scale);

[0181] Y(batch_id,channel_id,i,w_id,scale)=h0lambda*(w0lambda*X0+w1lambda*X1)+h1lambda*(w0lambda*X2+w1lambda*X3)

[0182] End

[0183] (2) For graph data in NHWC format:

[0184] For a single thread (i.e., thread 1), assuming the original pixels corresponding to this thread are pixel 0, pixel 1, pixel 3, and pixel 4, the pixel data of these four pixels can be read into this thread. Then, this thread can perform pixel interpolation calculations in the H dimension (height dimension) and W dimension (width dimension) based on the pixel data of these four pixels. The target pixels in the target image obtained based on this thread are as follows: Figure 5c As shown in the dashed box of thread 1, each dimension corresponds to 9 target pixels, so the final number of target pixels is 3*3=9 (i.e., k=9). Similarly, for the next thread (thread 2), the pixel data of adjacent pixels 1, 2, 4, and 5 can be obtained. Thread 2 can perform pixel interpolation calculations in the H and W dimensions based on the pixel data of these 4 pixels. At this time, the pixel data of each target pixel in the target image obtained by this thread is as follows: Figure 5c The dashed box indicates thread 2. Following the same logic, once all threads have completed their calculations, the target graph can be obtained (specifically as shown in the image). Figure 5c (As shown in the overall diagram).

[0185] Optionally, in practical applications, when processing at least one image to be processed (which can be divided into multiple batches of data) in NHWC format, for a single thread, assuming the thread's ID index is global_index, the relevant information of the original pixel corresponding to that thread can be determined using the following expressions:

[0186] h_scale=in_height / out_height;

[0187] w_scale=in_width / out_width;

[0188] batch_id=global_index / (in_width*in_height*channel_num*k);

[0189] per_batch_index=global_index%(in_width*in_height*channel_num*k);

[0190] channel_id=per_batch_index / (in_width*in_height*k);

[0191] per_channel_index=per_batch_index%(i_width*in_height*k);

[0192] h_id=per_channel_index*ah / (in_width*k);

[0193] w_id=per_channel_index*ah%(in_width*k);

[0194] start_id_0=floor(h_id / h_scale);

[0195] end_id_0=floor((h_id+1) / h_scale);

[0196] start_id_1=floor(w_id / w_scale);

[0197] end_id_1=floor((w_id+1) / w_scale)

[0198] Wherein, in_width*in_height*channel_num*k represents the number of threads required for one image to be processed in the second number of threads, start_id_0 represents the identifier of the starting pixel of each target pixel in the H dimension of the thread, end_id_0 represents the identifier of the last pixel of each target pixel in the H dimension of the thread, start_id_1 represents the identifier of the starting pixel of each target pixel in the W dimension of the thread, and end_id_1 represents the identifier of the last pixel of each target pixel in the W dimension of the thread.

[0199] Furthermore, based on the above relevant information, the coordinate information of each original pixel point required for the interpolation calculation of this thread can be obtained according to the principle of bilinear interpolation algorithm. It is assumed that the coordinates of the four original pixels corresponding to this thread are (h1, w1), (h1, w1+w1p), (h1+h1p, w1) and (h1+h1p, w1+w1p).

[0200] Furthermore, the pixel data of the four original pixels can be read into registers X0, X1, X2, and X3. Based on the coordinates of each original pixel, interpolation weights are calculated, including weights w0lambda and w1lambda for the width dimension and h0lambda and h1lambda for the height dimension. Based on the pixel data of each original pixel and the interpolation weights, the pixel data of each target pixel (i.e., the interpolation result) can be calculated. Optionally, example code for calculating the interpolation weights and interpolation results can be provided below:

[0201] X0=X[idx(batch_id,channel_id,i_height,i_width,h1,w1)];

[0202] X1=X[idx(batch_id,channel_id,i_height,i_width,h1,w1+w1p)];

[0203] X2=X[idx(batch_id,channel_id,i_height,i_width,h1+h1p,w1)];

[0204] X3=X[idx(batch_id,channel_id,i_height,i_width,h1+h1p,w1+w1p)];

[0205] for int j = start_id_0; j <end_id_0;i++:

[0206] w1,w1p,w0lambda,w1lambda=bilinear_calcute(w_id,w_scale);

[0207] for int i = start_id_1; i <end_id_1;i++:

[0208] h1,h1p,h0lambda,h1lambda=bilinear_calcute(i,h_scale);

[0209] Y(batch_id,channel_id,i,w_id,scale)=h0lambda*(w0lambda*X0+w1lambda*X1)+h1lambda*(w0lambda*X2+w1lambda*X3)

[0210] End

[0211] End

[0212] Step S407: Use other processing methods to process the image to be processed.

[0213] In this example, for the NCHW format image to be processed, multiple pixels are interpolated by one thread in the H dimension, while a single pixel is calculated by a single thread in the W dimension. For the NHWC format image to be processed, multiple pixels can be interpolated and calculated simultaneously in both the H and W dimensions. For example, when interpolating a 3x3 (3 pixels * 3 pixels) image to be processed into a 9x9 (9 pixels * 9 pixels) target image, for the NCHW format, since multiple pixels are calculated in the H dimension and one pixel in the W dimension, a single thread can generate 1x3 pixels, requiring only 27 threads, with each input pixel data read 3 times. For the NHWC format image to be processed, since multiple pixels can be calculated in both the H and W dimensions, a single thread can interpolate and generate 3x3 pixels, requiring a total of 9 threads, with each input data read once. Therefore, the method provided in this embodiment can effectively reduce the amount of data computation, improve computation speed, and reduce computing hardware costs.

[0214] Optionally, in another example, assuming the number of images to be processed is 32 (batch_num = 32), the number of channels in the images to be processed is 3 (i.e., channel = 3), and the image size information is (in_height, in_width, out_height, out_width), the images to be processed are processed 5 times based on TensorRT, PyTorch, and the methods provided in the embodiments of this application (upsampling the 240*240 resolution images to 960*960, 480*480, 1080*080, 1024*1024, and 2080*2080 respectively) to obtain the target image. The time consumed by TensorRT, PyTorch, and the methods provided in the embodiments of this application to process the image to obtain the target image is as follows: Figure 6 As shown (where, Figure 6 The horizontal axis represents different processing methods, and the vertical axis represents the time consumed to obtain the target image, in milliseconds (ms). Based on this... Figure 6 It can be seen that the method provided in this application embodiment has the best time performance and its computational efficiency is higher than that of PyTorch and TensorRT.

[0215] This application provides a graph data processing apparatus, such as... Figure 7 As shown, the image data processing device 70 may include: an image data acquisition module 701, a first quantity determination module 702, a second quantity determination module 703, and a pixel interpolation calculation module 704, wherein...

[0216] The image data acquisition module 701 is used to acquire at least one image to be processed and at least one related information of the image to be processed. The related information includes the image data format and image information. The image information includes the number of images, the number of channels of the image to be processed, and the image size information. The image size information includes the width and height of the image to be processed, and the width and height of the target image corresponding to the image to be processed.

[0217] The first quantity determination module 702 is used to obtain multiple candidate values ​​of the first quantity and device resource information of the electronic device, wherein the first quantity is the number of pixels processed by one thread of the electronic device; based on the device resource information and related information, the device resource utilization rate corresponding to each candidate value is determined, and based on the device resource utilization rate corresponding to each candidate value, the first quantity is determined from each candidate value;

[0218] The second quantity determination module 703 is used to determine, based on the first quantity and relevant information, the second number of threads required to process at least one image to be processed, and to start the second number of threads;

[0219] The pixel interpolation calculation module 704 is used to obtain the pixel data of the original pixel corresponding to the thread in at least one image to be processed for each thread, and perform interpolation calculation based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel, wherein the number of each target pixel is the first quantity.

[0220] Optionally, the graph data format is NCHW(batch_number, channel, height, width, batch number, channel, height, width) or NHWC(batch, height, width, channel, batch number, height, width, channel) format; the first quantity determination module, when obtaining multiple candidate values ​​for the first quantity, is specifically used for:

[0221] If the graph data format is NCHW, determine the first ratio between the height of the target graph and the height of the graph to be processed, and use each integer value that is not greater than the first ratio as multiple candidate values.

[0222] If the image data format is NHWC, determine the second ratio of the height of the target image to the height of the image to be processed, and the third ratio of the width of the target image to the width of the image to be processed; and take integer values ​​that are not greater than the product of the second ratio and the third ratio, and that are multiples of the third ratio, as multiple candidate values.

[0223] Optionally, the graph data format can be NCHW or NHWC.

[0224] The first quantity determination module, when determining the equipment resource utilization rate corresponding to each candidate value based on equipment resource information and related information, is specifically used for:

[0225] Based on the device resource information, determine the number of threads that the electronic device can start;

[0226] If the graph data format is NCHW, for each candidate value, the number of candidate threads corresponding to the candidate value is determined based on the height of the graph to be processed, the number of graphs, the width of the target graph, and the number of channels. The device resource rate corresponding to the candidate value is determined based on the number of candidate threads and the number of threads that can be started. The number of candidate threads refers to the number of threads required to process at least one graph to be processed if the first quantity is a candidate value.

[0227] If the graph data format is NHWC, for each candidate value, the number of candidate threads corresponding to the candidate value is determined based on the height and width of the graph to be processed, the number of graphs, and the number of channels. Based on the number of candidate threads and the number of threads that can be started, the device resource utilization rate corresponding to the candidate value is determined.

[0228] Optionally, the first quantity determination module, when determining the device resource utilization corresponding to each candidate value based on the number of candidate threads and the number of threads that can be started, is specifically used for:

[0229] Determine the absolute value of the difference between the number of candidate threads corresponding to the candidate value and the number of threads that can be started, and use the absolute value as the device resource utilization rate corresponding to the candidate value;

[0230] When determining the first quantity from each candidate value based on the equipment resource utilization rate corresponding to each candidate value, the first quantity determination module is specifically used for:

[0231] The candidate value corresponding to the smallest absolute value among all candidate values ​​is determined as the first quantity.

[0232] Optionally, the device resource information is the number of SMs in the electronic device;

[0233] If the graph data format is NCHW, for any candidate value k, the device resource utilization rate corresponding to the candidate value is determined by the following expression:

[0234] cost=|out_width*in_height*batch_num*channel_num*k-512*sm_num*r|

[0235] Where cost is the device resource utilization rate, out_width is the width of the target graph, in_height is the height of the graph to be processed, channel_num is the number of channels, sm_num is the number of SMs, r is the preset value, batch_num is the number of graphs, out_width*in_height*batch_num*channel_num*k is the number of candidate threads corresponding to the candidate value k, and 512*sm_num*r is the number of threads that can be started.

[0236] If the graph data format is NHWC, for any candidate value k, the equipment resource utilization rate corresponding to the candidate value is determined by the following expression:

[0237] cost=|in_width*in_height*batch_num*channel_num*k-512*sm_num*r|

[0238] Where in_width is the width of the graph to be processed, in_width*in_height*batch_num*channel_num*k is the number of candidate threads corresponding to the candidate value k, and 512*sm_num*r is the number of threads that can be started.

[0239] Optionally, for each thread, when the pixel interpolation calculation module performs interpolation calculations based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel, it is specifically used for:

[0240] If the image data format is NCHW, bilinear interpolation is performed on the pixels in the height dimension based on the pixel data of the original pixels to obtain the pixel data of each target pixel in the height dimension of the target image corresponding to the original pixels.

[0241] If the image data format is NHWC, for each thread, based on the pixel data of the original pixel, bilinear interpolation calculation is performed on the height and width dimensions of the pixels to obtain the pixel data of each target pixel in the target image corresponding to the original pixel in the height and width dimensions.

[0242] Optionally, the apparatus further includes an information determination module for determining, before determining the first quantity from the candidate values, that at least one of the following graphs to be processed satisfies at least one of the following:

[0243] At least one image to be processed must satisfy at least one of the following:

[0244] The ratio of the height of the target image to the height of the image to be processed is greater than or equal to a first set value;

[0245] The ratio of the width of the target image to the width of the image to be processed is greater than or equal to the second set value;

[0246] The total number of pixels corresponding to at least one image to be processed is greater than or equal to the number of threads that the electronic device can start.

[0247] Optionally, for each thread, when the pixel interpolation calculation module obtains pixel data of the original pixel corresponding to the thread in at least one image to be processed, it is specifically used for:

[0248] For any two threads in the second number of threads, if there are identical original pixel data in the pixel data of the original pixel corresponding to the two threads, then obtain the pixel data of the original pixel corresponding to one of the two threads, transfer the pixel data of the identical original pixel in the obtained pixel data to the other thread, and obtain the pixel data of the other original pixel corresponding to the other thread.

[0249] Among them, the pixel data of other original pixels are the pixel data of the original pixels in the pixel data of the original pixels corresponding to another thread, excluding the pixel data of the same original pixels.

[0250] Optionally, the image to be processed can be an image or a feature map.

[0251] The graph data processing apparatus of this application embodiment can execute a graph data processing method provided in this application embodiment, and its implementation principle is similar, so it will not be described again here.

[0252] In some embodiments, the graph data processing apparatus provided in this invention can be implemented using a combination of hardware and software. As an example, the graph data processing apparatus provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the graph data processing method provided in this invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0253] This application provides an electronic device, such as... Figure 8 As shown, Figure 8The illustrated electronic device 2000 includes a processor 2001 and a memory 2003. The processor 2001 and the memory 2003 are connected, for example, via a bus 2002. Optionally, the electronic device 2000 may also include a transceiver 2004. It should be noted that in practical applications, the transceiver 2004 is not limited to one type, and the structure of this electronic device 2000 does not constitute a limitation on the embodiments of this application.

[0254] In this embodiment, the processor 2001 is used to implement... Figure 7 The functions of each module are shown.

[0255] Processor 2001 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 2001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0256] Bus 2002 may include a pathway for transmitting information between the aforementioned components. Bus 2002 may be a PCI bus or an EISA bus, etc. Bus 2002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0257] The memory 2003 may be ROM or other type of static storage device capable of storing static information and computer programs, RAM or other type of dynamic storage device capable of storing information and computer programs, or EEPROM, CD-ROM or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing or storing desired computer programs in the form of data structures and accessible by a computer, but not limited thereto.

[0258] The memory 2003 stores computer programs for executing the application scheme of this application, and its execution is controlled by the processor 2001. The processor 2001 executes the computer programs for the application scheme stored in the memory 2003 to implement… Figure 7 The operation of the graph data processing device provided in the illustrated embodiment.

[0259] This application provides an electronic device including a processor and a memory: the memory is configured to store a computer program, which, when executed by the processor, causes the processor to perform any of the methods described in the above embodiments.

[0260] This application provides a computer-readable storage medium for storing a computer program that, when run on a computer, enables the computer to perform any of the methods described in the above embodiments.

[0261] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

[0262] The specific terms and implementation principles involved in the computer-readable storage medium in this application can be found in a graph data processing method in the embodiments of this application, and will not be repeated here.

[0263] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0264] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for processing graph data, characterized in that, include: Obtain at least one image to be processed and related information of the at least one image to be processed. The related information includes image data format and image information. The image information includes the number of images, the number of channels of the image to be processed, and image size information. The image size information includes the width and height of the image to be processed, and the width and height of the target image corresponding to the image to be processed. Acquire a first number of multiple candidate values ​​and device resource information of the electronic device, wherein the first number is the number of pixels processed by one thread of the electronic device; Based on the equipment resource information and the relevant information, determine the equipment resource utilization rate corresponding to each candidate value, and determine the first quantity from each candidate value based on the equipment resource utilization rate corresponding to each candidate value; Based on the first quantity and the relevant information, determine a second number of threads required to process the at least one graph to be processed, and start the second number of threads; For each thread, the pixel data of the original pixel corresponding to the thread in the at least one image to be processed is obtained through the thread, and interpolation calculation is performed based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel, wherein the number of each target pixel is the first number.

2. The method according to claim 1, characterized in that, The graph data format is either batch quantity channel height and width NCHW format or batch quantity height and width channel NHWC format. The process of obtaining a first number of candidate values ​​includes: If the graph data format is NCHW format, determine a first ratio between the height of the target graph and the height of the graph to be processed, and use each integer value that is not greater than the first ratio as the plurality of candidate values; If the image data format is NHWC format, determine a second ratio between the height of the target image and the height of the image to be processed, and a third ratio between the width of the target image and the width of the image to be processed; and use integer values ​​that are not greater than the product of the second ratio and the third ratio, and that are multiples of the third ratio, as the plurality of candidate values.

3. The method according to claim 1, characterized in that, The graph data format is NCHW or NHWC format; The step of determining the equipment resource utilization rate corresponding to each candidate value based on the equipment resource information and the relevant information includes: Based on the device resource information, determine the number of threads that the electronic device can start; If the graph data format is NCHW format, for each candidate value, the number of candidate threads corresponding to the candidate value is determined according to the height of the graph to be processed, the number of graphs, the width of the target graph, and the number of channels. The device resource rate corresponding to the candidate value is determined according to the number of candidate threads and the number of threads that can be started. The number of candidate threads refers to the number of threads required to process at least one graph to be processed if the first quantity is the candidate value. If the graph data format is NHWC format, for each candidate value, the number of candidate threads corresponding to the candidate value is determined according to the height and width of the graph to be processed, the number of graphs, and the number of channels. The device resource utilization rate corresponding to the candidate value is determined according to the number of candidate threads and the number of threads that can be started.

4. The method according to claim 3, characterized in that, For each candidate value, determining the device resource utilization rate corresponding to the candidate value based on the number of candidate threads and the number of threads that can be started includes: Determine the absolute value of the difference between the number of candidate threads corresponding to the candidate value and the number of threads that can be started, and use the absolute value as the device resource utilization rate corresponding to the candidate value; The step of determining the first quantity from each candidate value based on the equipment resource utilization rate corresponding to each candidate value includes: The candidate value corresponding to the smallest absolute value among the absolute values ​​corresponding to each candidate value is determined as the first quantity.

5. The method according to claim 3, characterized in that, The device resource information refers to the number of stream processors (SMs) in the electronic device. If the graph data format is NCHW, for any candidate value k, the device resource utilization rate corresponding to the candidate value is determined by the following expression: cost=|out_width*in_height*batch_num*channel_num*k-512*sm_num*r| Where cost is the device resource utilization rate, out_width is the width of the target graph, in_height is the height of the graph to be processed, channel_num is the number of channels, sm_num is the number of SMs, r is a preset value, batch_num is the number of graphs, out_width*in_height*batch_num*channel_num*k is the number of candidate threads corresponding to the candidate value k, and 512*sm_num*r is the number of threads that can be started; If the graph data format is NHWC format, for any candidate value k, the equipment resource utilization rate corresponding to the candidate value is determined by the following expression: cost=|in_width*in_height*batch_num*channel_num*k-512*sm_num*r| Wherein, in_width is the width of the graph to be processed, in_width*in_height*batch_num*channel_num*k is the number of candidate threads corresponding to the candidate value k, and 512*sm_num*r is the number of threads that can be started.

6. The method according to claim 2, characterized in that, For each thread, the step of interpolating based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel includes: If the image data format is NCHW format, bilinear interpolation calculation is performed on the pixels in the height dimension based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the height dimension in the target image corresponding to the original pixel. If the image data format is NHWC, for each thread, based on the pixel data of the original pixel, bilinear interpolation calculation is performed on the pixel in the height and width dimensions to obtain the pixel data of each target pixel in the target image corresponding to the original pixel in the height and width dimensions.

7. The method according to claim 1, characterized in that, For each thread, obtaining the pixel data of the original pixel corresponding to the thread in the at least one image to be processed includes: For any two threads in the second number of threads, if there are identical original pixel data in the pixel data of the original pixel points corresponding to the two threads, then obtain the pixel data of the original pixel point corresponding to one of the two threads, transfer the pixel data of the identical original pixel points in the obtained pixel data to the other thread, and obtain the pixel data of other original pixel points corresponding to the other thread. Wherein, the pixel data of the other original pixels are the pixel data of the original pixels other than the pixel data of the same original pixels in the pixel data of the original pixels corresponding to the other thread.

8. The method according to any one of claims 1 to 7, characterized in that, Before determining the first quantity from the candidate values, the method further includes: The at least one image to be processed is determined to satisfy at least one of the following: The ratio of the height of the target image to the height of the image to be processed is greater than or equal to a first set value; The ratio of the width of the target image to the width of the image to be processed is greater than or equal to a second set value; The total number of pixels corresponding to at least one image to be processed is greater than or equal to the number of threads that the electronic device can start.

9. The method according to claim 1, characterized in that, The image to be processed is either an image or a feature map.

10. A graph data processing apparatus, characterized in that, The device includes: The image data acquisition module is used to acquire at least one image to be processed and related information of the at least one image to be processed. The related information includes image data format and image information. The image information includes the number of images, the number of channels of the image to be processed, and image size information. The image size information includes the width and height of the image to be processed, and the width and height of the target image corresponding to the image to be processed. The first quantity determination module is used to obtain multiple candidate values ​​of a first quantity and device resource information of an electronic device, wherein the first quantity is the number of pixels processed by one thread of the electronic device; based on the device resource information and the relevant information, it determines the device resource utilization rate corresponding to each candidate value, and determines the first quantity from each candidate value based on the device resource utilization rate corresponding to each candidate value; The second quantity determination module is used to determine, based on the first quantity and the relevant information, a second quantity of threads required to process the at least one image to be processed, and to start the second quantity of threads; The pixel interpolation calculation module is used to, for each thread, obtain the pixel data of the original pixel corresponding to the thread in the at least one image to be processed, and perform interpolation calculation based on the pixel data of the original pixel to obtain the pixel data of each target pixel in the target image corresponding to the original pixel, wherein the number of each target pixel is the first number.

11. An electronic device, characterized in that, Including the processor and memory: The memory is configured to store a computer program that, when executed by the processor, causes the processor to perform the method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when run on a computer, enables the computer to perform the method described in any one of claims 1-9.

Images