A method and device for scheduling super-resolution image blocks, and a medium

Abstract

This invention discloses a method, apparatus, and medium for scheduling image chunks in ultra-high resolution images. It receives and processes image chunking tasks in real time, obtaining target image chunking task description information. When an image chunking execution command is received, the image chunking algorithm module is instructed to parse the task description information, which is then placed in the task scheduler cache pool, and a parent-child task dependency table and a subtask dependency table are established. When a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool using the task scheduling method until all chunk subtasks are scheduled, thus scheduling the image chunking task to the target hardware device. This invention solves the problems of high hardware bandwidth cost and low efficiency caused by direct processing, as well as the semantic integrity destruction and low accuracy of traditional chunking processing due to ignoring chunk dependencies, thereby improving the efficiency and accuracy of ultra-high resolution image chunking scheduling.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus and medium for block scheduling of ultra-high resolution images. Background Technology

[0002] With the deep application of large visual models and multimodal large models in high-precision scenarios such as remote sensing interpretation, digital pathology, and industrial quality inspection, ultra-high resolution images (e.g., for...) are becoming increasingly important. High-quality scheduling and pre- and post-processing of images (with over 1 billion pixels) has become a core element in building training datasets and ensuring consistency of inference inputs.

[0003] In the process of developing this invention, the inventors discovered the following drawbacks in the existing technology: Currently, directly processing the entire ultra-high resolution image increases the width of the hardware line buffer, resulting in high hardware bandwidth costs and high power consumption. Traditional block-segmentation strategies are usually software solutions implemented at the application layer, leading to long processing times for ultra-high resolution images, excessive consumption of central processing unit and memory resources, and impacting the overall execution efficiency of the business. Furthermore, traditional block-segmentation strategies ignore pixel neighborhood dependencies and cross-block dependencies in image processing, resulting in stitching misalignment, color banding, or edge artifacts, which damage the semantic integrity of the image and directly reduce the accuracy of feature extraction from large models. Summary of the Invention

[0004] This invention provides a method, apparatus, and medium for block scheduling of ultra-high resolution images, so as to improve the efficiency and accuracy of block scheduling of ultra-high resolution images.

[0005] According to one aspect of the present invention, a block scheduling method for ultra-high resolution images is provided, comprising: It receives and parses the target image segmentation task description information in real time according to the image segmentation task to be processed issued by the application layer; Among them, the target image block task description information is stored in the command cache pool inside the management module; When an image segmentation execution command is received, the image segmentation algorithm module is instructed to parse the target image segmentation task description information to obtain the segmentation subtask description information. The segmentation subtask description information is then placed in the task scheduler cache pool, and a parent-child task dependency table and a subtask dependency table are established. When a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all block sub-tasks are scheduled, thereby scheduling the image block task to be processed to the target hardware device.

[0006] According to another aspect of the present invention, a block scheduling apparatus for ultra-high resolution images is provided, comprising: The target image block task description information determination module is used to receive and parse the target image block task description information in real time according to the image block task to be processed issued by the application layer. Among them, the target image block task description information is stored in the command cache pool inside the management module; The sub-task description information determination module is used to, when receiving an image segmentation execution command, instruct the image segmentation algorithm module to parse the target image segmentation task description information, obtain the sub-task description information, put the sub-task description information into the task scheduler cache pool, and establish a parent-child task dependency table and a sub-task dependency table. The scheduling processing module is used to instruct the task scheduling module to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table when a task scheduling command is received, through a preset task scheduling method, until all block sub-tasks are scheduled, so as to schedule the image block task to be processed to the target hardware device.

[0007] According to another aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the block scheduling method for ultra-high resolution images as described in any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the block scheduling method for ultra-high resolution images as described in any embodiment of the present invention.

[0009] The technical solution of this invention receives and parses the target image block task description information based on the image block task to be processed issued by the application layer in real time. The target image block task description information is stored in the command cache pool inside the management module. When an image block execution command is received, the image block algorithm module is instructed to parse the target image block task description information to obtain block sub-task description information, which is then placed in the task scheduler cache pool, and a parent-child task dependency table and a sub-task dependency table are established. When a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all block sub-tasks are scheduled, thus scheduling the image block task to be processed to the target hardware device. This solves the problems of high hardware bandwidth cost and low efficiency caused by direct processing, and the semantic integrity and low accuracy of traditional block processing due to ignoring block dependencies, thereby improving the efficiency and accuracy of ultra-high resolution image block scheduling.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a block scheduling method for ultra-high resolution images provided in Embodiment 1 of the present invention; Figure 2 This is a detailed flowchart of a block scheduling method for ultra-high resolution images provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of a block scheduling device for ultra-high resolution images provided in Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided according to Embodiment 4 of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "target," "current," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] It is worth noting that the information collected in the technical solution of this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse; if the user chooses to refuse, the process will proceed to the expert decision-making process.

[0016] Example 1 Figure 1 The flowchart of a block scheduling method for ultra-high resolution images is provided in Embodiment 1 of the present invention. This embodiment describes the block scheduling of ultra-high resolution images. The method can be executed by a block scheduling device for ultra-high resolution images, which can be implemented in hardware and / or software.

[0017] Correspondingly, such as Figure 1 As shown, the method includes: S110. Receive and parse the target image segmentation task description information in real time according to the image segmentation task to be processed issued by the application layer.

[0018] Specifically, the task description information for segmenting the target image is stored in the command cache pool inside the management module.

[0019] In this embodiment, the application layer can receive command definitions issued to the ultra-high resolution image block scheduling system. These commands can be for various image pre- and post-processing functions, including color gamut conversion, image scaling, image cropping, image edge filling, and image scaling edge filling fusion operators.

[0020] Specifically, the application layer can use the Submit Queue (SQ) and Complete Queue (CQ) provided by the ultra-high resolution image tile scheduling system to receive tasks and provide feedback on results. Specifically, the application layer issues image tile tasks to be processed through the SQ; the ultra-high resolution image tile scheduling system (which can be abbreviated as FW, Video Pre Processing Firmware) provides feedback on task execution results to the application layer through the CQ.

[0021] The target image segmentation task description information may include the source image memory address, the output image memory address, or the row processing step size of the image.

[0022] Optionally, the ultra-high resolution image segmentation scheduling system includes a read command module, a batch processing command management module, an image segmentation algorithm module, a task scheduling module, a hardware abstraction layer, and an interrupt handling module. The real-time reception and parsing of the image segmentation task description information from the application layer includes: receiving the image segmentation task from the application layer in real-time through a task submission queue; instructing the read command module to parse and process the image segmentation task to obtain the target image segmentation task description information, and sending the target image segmentation task description information to the batch processing command management module; instructing the batch processing command management module to store the received target image segmentation task description information in its internal command cache pool, and receiving real-time scheduling from the task scheduling module.

[0023] The command reading module can be used to acquire and parse commands issued by the application layer, and can be a BCF (Batch Command Fetcher). The batch command management module (Batch CommandManager, BQM) is responsible for managing the target image block task description information parsed by BCF, placing commands into the command cache pool inside the management module, and waiting for scheduling by the task scheduling module TS (Task Scheduler).

[0024] The Hardware Abstraction Layer (HAL) enables task transmission with the target hardware device, while the Interrupt Handling Module (IM) receives interrupt requests from the hardware device in real time. The hardware device can be a video image preprocessing device that performs the actual image processing task and reports the task execution status back to the firmware (FW) via interrupt reporting.

[0025] The Slice Algorithm Module (SAM) can be used for image segmentation processing of ultra-high resolution video images by preprocessing operators. It can design corresponding image segmentation methods for different image preprocessing and postprocessing operators, and obtain the input and output resolution, segmentation points, and interpolation pixels and other parameters required for hardware execution for each segment.

[0026] In this embodiment, after receiving the image segmentation task to be processed, it is necessary to determine whether the task belongs to ultra-large resolution image. If so, the task needs to be parsed to obtain the corresponding target image segmentation task description information.

[0027] Specifically, the application layer can send image segmentation tasks to be processed to the FW in real time via SQ. The system can instruct BCF to parse and process the image segmentation tasks to obtain the target image segmentation task description information. Specifically, the target image segmentation task description information may include one or more of the following parameters: source image memory address, output image memory address, and image row processing step size.

[0028] Correspondingly, the source image memory address, output image memory address, and image row processing step size can be sent to the BQM; the BQM is instructed to store the received information in the command cache pool inside the management module and receive scheduling from the task scheduling module TS in real time.

[0029] The advantage of this setup is that the task description information of the target image blocks can be stored in the command cache pool inside the management module. When a task needs to be processed, it can be scheduled through the task scheduling module, avoiding the waste of bandwidth and low efficiency caused by transmitting and scheduling all data.

[0030] S120. When an image segmentation execution command is received, the image segmentation algorithm module is instructed to parse the target image segmentation task description information to obtain the segmentation subtask description information, and the segmentation subtask description information is placed in the task scheduler cache pool, and a parent-child task dependency table and a subtask dependency table are established.

[0031] The segmented subtask description information can be a standardized subtask descriptor, which can encapsulate the complete information required for scheduling and configuring the hardware for each subtask. Specifically, the segmented subtask description information may include, but is not limited to: the segmentation points in the source image, the segmentation points in the output image, image processing operation parameters, the number of segments, or the starting address of the segments.

[0032] In this embodiment, image segmentation execution commands can be obtained from the task cache pool, and the image segmentation algorithm module is instructed to parse the target image segmentation task description information to obtain segmented subtask description information. Furthermore, parent-child task dependency tables and inter-subtask dependency tables can be established based on the description information of each segmented subtask. Correspondingly, the parent task is placed in the task scheduler cache pool for queuing, and the task state machine is adjusted to an executable state, awaiting scheduling.

[0033] In this embodiment, the image segmentation algorithm module parses the target image segmentation task description information, which may include methods such as segmentation data calculation methods and multi-functional image segmentation methods.

[0034] Optionally, when an image segmentation execution command is received, the image segmentation algorithm module is instructed to parse the target image segmentation task description information to obtain segmentation subtask description information, and the segmentation subtask description information is placed in the task scheduler cache pool. A parent-child task dependency table and a subtask dependency table are established. This includes: when an image segmentation execution command is received, calculating the target segmentation number based on the target image segmentation task description information using the segmentation data calculation method in the image segmentation algorithm module; performing image interpolation processing using a preset bilinear filtering interpolation method based on the target image segmentation task description information, and combining this with a preset multi-functional image segmentation method to calculate the segmentation points in the source image and the output image corresponding to each segmentation subtask; placing the segmentation subtask description information and the subtask state machine in the task scheduler cache pool, and establishing a parent-child task dependency table and a subtask dependency table.

[0035] In this embodiment, the number of target segments is first calculated based on the target image segmentation task description information using a segmentation data calculation method.

[0036] Specifically, the method for calculating the segmentation data in the image segmentation algorithm module, based on the target image segmentation task description information, calculates the target segmentation number, including: obtaining a preset reserved processing edge width based on the source image width, expected output image width, maximum image width processed by the hardware device in a single operation, and minimum image width processed by the hardware device in a single operation, as shown in the target image segmentation task description information; when it is determined that the source image width is greater than the maximum image width processed by the hardware device in a single operation, and the source image width is greater than or equal to the expected output image width, the reserved processing edge width is subtracted from the maximum image width processed by the hardware device in a single operation to obtain a first difference, and the source image width is divided by the first difference to calculate the target segmentation number; when it is determined that the expected output image width is greater than the maximum image width processed by the hardware device in a single operation, and the source image width is less than the expected output image width, the expected output image width is divided by the first difference to calculate the target segmentation number.

[0037] For example, the target image segmentation task description information can be set as follows: source image width is srcWidth, expected output image width is dstWidth, maximum image width for single processing by the hardware device is maxWidth, minimum image width for single processing by the hardware device is minWidth, and reserved processing edge width is pWidth (since image processing may use different interpolation modes, a processing edge needs to be reserved with a width of pWidth).

[0038] Furthermore, when it is determined that the source image width srcWidth is greater than the maximum image width maxWidth that the hardware device can process in a single run, and the source image width srcWidth is greater than or equal to the expected output image width dstWidth, then the first difference is obtained by subtracting the reserved processing edge width from the maximum image width that the hardware device can process in a single run. That is, the first difference is... The number of target segments is calculated by dividing the source image width by the first difference. .

[0039] Accordingly, when it is determined that the expected output image width dstWidth is greater than the maximum image width maxWidth that the hardware device can process in a single run, and the source image width srcWidth is less than the expected output image width dstWidth, then the number of target segmented blocks is calculated by dividing the expected output image width by the first difference. .

[0040] In this embodiment, based on the above-mentioned target image segmentation task description information, the image needs to be interpolated using the bilinear filtering interpolation method, and combined with the image multi-functional segmentation method, to calculate the segmentation points in the source image and the segmentation points in the output image corresponding to each segmentation sub-task.

[0041] Before this, it's necessary to determine whether the input target image is in a downsampling format (e.g., YUV42x Planar, YUV42x Semi Planar, YUYV, or UYVY) based on the task description information, to decide whether to enable upsampling or downsampling. Specifically, when upsampling is enabled, the U and V planes of the input image need to be interpolated using bilinear filtering. Specifically, bilinear filtering interpolation selects the mapping of the output image pixels in the source image, and interpolates the pixel values ​​at its right, bottom, and bottom-right points. Upsampling can be considered as doubling the horizontal and vertical dimensions of the source image's U and V planes, ensuring the segmentation points in the source image are aligned to 2, thus avoiding the need to take extra pixels for interpolation in the forward and backward directions. Similarly, the process is similar when downsampling is enabled; the output image width of the previous module needs to be aligned to 2, eliminating the need to take extra pixels for interpolation, which reduces hardware processing load and simplifies the segmentation algorithm.

[0042] Furthermore, the step of combining the preset image multi-functional segmentation method to calculate the segmentation points in the source image and the output image corresponding to each segmentation sub-task includes: using the image edge filling segmentation sub-method in the image multi-functional segmentation method, based on the obtained maximum image width processed by the hardware device in a single operation, minimum image width processed by the hardware device in a single operation, left boundary filling width, right boundary filling width, remaining unsegmented source image width, and remaining unsegmented segment number, to calculate the input width and output width of each segment, the segmentation points in the source image, and the segmentation points in the output image; or using the image multi-functional segmentation method... The image scaling and segmentation sub-method in the method calculates the output width of each segment, the segmentation point in the source image, and the segmentation point in the output image based on the obtained horizontal output-to-input scaling ratio, the image width input to the image scaling module, the total source image width processed from the first segment to the present, and the total output image width of the forward segments. Among them, the image color gamut conversion operator segmentation method segments according to the maximum image width processed by the hardware device in a single operation; and the image cropping operator segmentation method uses the cropping start point as the starting position for reading data from the target hardware device, and takes the average of the actual output image width as the input and output width of the image segment.

[0043] Specifically, the image edge padding sub-method employs a greedy algorithm for block processing, prioritizing maximizing the output width of the preceding blocks while ensuring that the input width of each block is greater than or equal to `minWidth` and the output width is less than or equal to `maxWidth`. The left boundary padding width can be set to `padLeft`, the right boundary padding width to `padRight`, the remaining unsegmented source image width to `remainSrcWidth`, the remaining unsegmented blocks to `remainSlices`, and the input width of each block to `sliceSrcWidth`. The remaining width can also be set to `remainWidth`.

[0044] The sliceSrcWidth can be determined using the formula: when At that time, the sliceSrcWidth calculation result is .when At that time, the sliceSrcWidth calculation result is .

[0045] Furthermore, based on the input width of each block calculated through multiple segmentations, key information parameters such as the output width corresponding to each block input width, the segmentation point position in the source image, and the segmentation point position in the output image can be calculated.

[0046] Furthermore, the image scaling and segmentation method must ensure that all pixels on the right edge of the output image (except for the last segment) can be completely interpolated by the four interpolation points in the input image. This eliminates the dependencies between image segments and simplifies the design complexity of the task scheduling system. Specifically, after image scaling, the actual output width after removing the last column of the right edge should be less than or equal to the input width multiplied by the scaling factor. At the same time, to ensure the correctness of the interpolation of the pixel values ​​on the left edge of the output image, two more columns of pixels need to be taken forward when calculating the width of the source image.

[0047] Specifically, the horizontal output-to-input scaling factor horRatio can be preset, along with the image width rszSrcWidth input to the image scaling module, the total source image width sumWidth processed from the first block to the present, and the total output image width sumPreDstWidth of the forward blocks. These parameters are used to calculate the output width of each block, the segmentation point in the source image, and the segmentation point in the output image. Specifically, the formula for calculating the output width of each block (i.e., the image width after block scaling) rszDstWidth is as follows: The splitting point horStart of this block in the source graph can be calculated as follows: .

[0048] Additionally, by controlling the initial phase of the bilinear filter interpolation, the interpolation result can be ensured to remain unaffected by the block operation. This means the image result after block and stitching processing is consistent with the result of complete processing of a normal image. The initial phase can be obtained by multiplying the last effective output column of the previous block (denoted as preDstWidth) by horRatio, and then subtracting the segmentation point of the source image in this block. That is, if the initial phase is initialPhaseX, then... Based on the above formulas, the output width of each block, the cutting point in the source image, and the cutting point in the output image can be calculated. This information is then filled into the block subtask description information and handed over to the FW for scheduling to hardware execution.

[0049] In this embodiment, since the color gamut conversion operator does not involve pixel interpolation, the image color gamut conversion operator block method divides the image into blocks according to the maximum image width processed by the hardware device in a single operation. For the image cropping operator block method, the cropping start point can be used as the starting position for the target hardware device to read data, and the average value of the actual output image width can be used as the input and output width of the image block. Specifically, the starting position for the hardware RDMA to read data can be used, and the average value of the actual output image width can be used as the input and output width of the image block.

[0050] The advantage of this setup is that by calculating the target number of segments in the image segmentation algorithm module, and by setting parameters such as the output width of each segment, the segmentation point in the source image, or the segmentation point in the output image through image edge filling, image scaling, color gamut conversion, and image cropping operators, the sub-task description information is filled into the sub-task description information. The task segmentation scheduling operation is then performed based on the sub-task description information, the parent-child task dependency table, and the dependency table between sub-tasks. This ensures the semantic integrity of the image segmentation and improves the efficiency and accuracy of ultra-high resolution image segmentation scheduling.

[0051] S130. When a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all block sub-tasks are scheduled, thereby scheduling the image block task to be processed to the target hardware device.

[0052] In this embodiment, the task scheduling module can be instructed to schedule tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table based on the received task scheduling command, until all sub-tasks are scheduled. After all sub-tasks are completed, the corresponding processing results can be integrated, thereby scheduling the image sub-tasks to be processed to the target hardware device, realizing the operation of ultra-high resolution image sub-task scheduling, and achieving accurate, efficient and complete scheduling processing.

[0053] The technical solution of this invention involves receiving and parsing the target image block task description information from the application layer in real time. This target image block task description information is stored in a command cache pool within the management module. When an image block execution command is received, the image block algorithm module is instructed to parse the target image block task description information to obtain sub-task description information. This sub-task description information is then placed in the task scheduler cache pool, and a parent-child task dependency table and a sub-task dependency table are established. When a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all sub-tasks are scheduled, thereby scheduling the image block task to be processed to the target hardware device. It solves the problems of high hardware bandwidth cost and low efficiency caused by direct processing, as well as the problems of semantic integrity destruction and low accuracy caused by traditional block processing due to ignoring block dependencies. It reduces hardware bandwidth cost, ensures the semantic integrity of image blocks, and improves the efficiency and accuracy of block scheduling for ultra-high resolution images.

[0054] Example 2 Figure 2 This invention provides a detailed flowchart of a block scheduling method for ultra-high resolution images, as described in Embodiment 2. This embodiment refines the above embodiment by specifying that when a task scheduling command is received, the task scheduling module is instructed to schedule tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all block sub-tasks are scheduled, thereby scheduling the image block tasks to be processed to the target hardware device for further refinement.

[0055] S210. Receive and parse the target image segmentation task description information in real time according to the image segmentation task to be processed issued by the application layer.

[0056] Specifically, the task description information for segmenting the target image is stored in the command cache pool inside the management module.

[0057] S220. When an image segmentation execution command is received, the image segmentation algorithm module is instructed to parse the target image segmentation task description information to obtain the segmentation subtask description information, and the segmentation subtask description information is placed in the task scheduler cache pool, and a parent-child task dependency table and a subtask dependency table are established.

[0058] S230. When a task scheduling command is received, the task scheduling module is instructed to determine whether the current task to be scheduled is of the parent task type. If not, the current task to be scheduled is scheduled.

[0059] In this embodiment, since the current task to be scheduled is not a parent task type, it is determined to be a child task type, and the current task to be scheduled can be directly scheduled.

[0060] S240. If it is a parent task type, then according to the parent-child task dependency table and the sub-task dependency table, the task scheduling method is used to obtain at least one currently scheduled sub-task corresponding to the currently scheduled task from the task scheduler cache pool.

[0061] S250. Determine whether there are any idle hardware devices in the hardware scheduler list corresponding to the task scheduling module. If so, call the hardware abstraction layer to send the currently scheduled task and each currently scheduled sub-task block to the target hardware device for execution, and switch the current task state machine to the execution state.

[0062] S260. If not, put the currently scheduled task and each currently scheduled subtask into the task cache pool of the hardware scheduler, and set the state of the current task state machine to the waiting hardware execution state.

[0063] The hardware scheduler executes at the level of a parent task. Before the subtasks in the parent-child task dependency table are fully executed, the task scheduler cache pool will not be read again.

[0064] In this embodiment, if the current task is the parent task, the scheduler will scan the parent-child task dependency table and the subtask dependency table, and obtain at least one currently scheduled subtask corresponding to the currently scheduled task from the task scheduler cache pool.

[0065] Furthermore, the system checks if there are any idle hardware devices in the hardware scheduler's list. If so, it calls the HAL to distribute the task (including the currently scheduled task and each currently scheduled subtask) to the hardware device for execution and switches the task state machine to the execution state. If there are no idle hardware devices, the task is placed in the hardware scheduler's task cache pool, and its state is set to waiting for hardware execution. The hardware scheduler's execution granularity is one parent task. The task scheduler cache pool will not be read until all subtasks in the parent-child task dependency table have been fully executed.

[0066] S270. Until all sub-tasks have been scheduled, the image sub-tasks to be processed are scheduled to the target hardware device.

[0067] During task scheduling and execution, the interrupt handling module is instructed to receive interrupt commands from the target hardware device in real time. A task timeout detection mechanism is set in the ultra-high resolution image block scheduling system; if the scheduling time exceeds a preset time threshold, an alarm is sent to the application layer. The target hardware device reads data in steps using RDMA and WDMA, and includes multiple dedicated hardware units with different functions.

[0068] In this embodiment, during task scheduling and execution, IM is used to receive interrupts from hardware devices and obtain task execution status, preventing the hardware scheduler from being in a busy-waiting state after submitting tasks to the hardware. When a subtask execution completion interrupt is received, the task scheduler needs to update the subtask state machine to the point of completion, and simultaneously update the corresponding parent-child task dependency table and the inter-subtask dependency table. Undependent subtasks are scheduled, and the above steps are repeated until all segmented subtasks are scheduled. The hardware scheduler checks the parent-child task dependency table of the parent task. When it determines that all segmented subtasks have finished executing, it integrates the subtask execution status, notifies the user program through the task reply module, and determines that the cached task status in the hardware scheduler queue is unschedulable. This completes the operation, thereby scheduling the image segmentation tasks to be processed to the target hardware device.

[0069] In addition, a task timeout detection mechanism can be set in the ultra-high resolution image block scheduling system. When the scheduling time exceeds the time threshold, the system needs to actively report the exception to the application layer to realize the exception feedback operation, which can improve the timeliness and efficiency of exception feedback.

[0070] Correspondingly, the target hardware device can read data in steps using RDMA (Read Direct Memory Access) and WDMA (Write Direct Memory Access). The target hardware device contains multiple dedicated hardware units with different functions. The dedicated hardware units use RDMA to read source image data and use WDMA to write the processed data back to the specified memory space. RDMA and WDMA support reading data in steps and can accurately read and write data at specified pixel positions, enabling precise extraction or writing back of image blocks at any position from the global image matrix. This provides hardware assurance for image block processing.

[0071] In addition, multiple dedicated hardware units capable of pipelined operation can support various common video and image pre- and post-processing functions, such as image scaling, color gamut conversion, image format upsampling / downsampling, edge filling, and image cropping. The image scaling, image format upsampling, and image format downsampling modules support high-throughput nearest-neighbor interpolation and high-quality bilinear filtering, with computation speeds far exceeding software implementations. The hardware device includes global control registers for startup and configuration, and an interrupt and event manager for reporting task completion status information to the block task scheduling system, forming a complete hardware autonomous execution closed loop.

[0072] The technical solution of this invention receives and parses the target image block task description information based on the image block task to be processed issued by the application layer in real time. The target image block task description information is stored in the command cache pool inside the management module. When an image block execution command is received, the image block algorithm module is instructed to parse the target image block task description information to obtain block sub-task description information, which is then placed in the task scheduler cache pool, and a parent-child task dependency table and a sub-task dependency table are established. When a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all block sub-tasks are scheduled, thus scheduling the image block task to be processed to the target hardware device. This improves the timeliness and efficiency of anomaly feedback, enhances the accuracy of task scheduling, reduces hardware bandwidth costs, ensures the integrity of image block semantics, and improves the efficiency and accuracy of ultra-high resolution image block scheduling.

[0073] Example 3 Figure 3 This is a schematic diagram of a block scheduling device for ultra-high resolution images provided in Embodiment 2 of the present invention. The block scheduling device for ultra-high resolution images provided in this embodiment can be implemented by software and / or hardware, and can be configured in a terminal device or server to implement a block scheduling method for ultra-high resolution images according to an embodiment of the present invention. Figure 3 As shown, the device includes: a target image block task description information determination module 310, a block subtask description information determination module 320, and a scheduling processing module 330.

[0074] The target image block task description information determination module 310 is used to receive and parse the target image block task description information in real time according to the image block task to be processed issued by the application layer. Among them, the target image block task description information is stored in the command cache pool inside the management module; The sub-task description information determination module 320 is used to, when receiving an image segmentation execution command, instruct the image segmentation algorithm module to parse the target image segmentation task description information, obtain the sub-task description information, put the sub-task description information into the task scheduler cache pool, and establish a parent-child task dependency table and a sub-task dependency table. The scheduling processing module 330 is used to instruct the task scheduling module to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table when a task scheduling command is received, through a preset task scheduling method, until all block sub-tasks are scheduled, so as to schedule the image block task to be processed to the target hardware device.

[0075] The technical solution of this invention involves receiving and parsing the target image block task description information from the application layer in real time. This target image block task description information is stored in a command cache pool within the management module. When an image block execution command is received, the image block algorithm module is instructed to parse the target image block task description information to obtain sub-task description information. This sub-task description information is then placed in the task scheduler cache pool, and a parent-child task dependency table and a sub-task dependency table are established. When a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all sub-tasks are scheduled, thereby scheduling the image block task to be processed to the target hardware device. It solves the problems of high hardware bandwidth cost and low efficiency caused by direct processing, as well as the problems of semantic integrity destruction and low accuracy caused by traditional block processing due to ignoring block dependencies. It reduces hardware bandwidth cost, ensures the semantic integrity of image blocks, and improves the efficiency and accuracy of block scheduling for ultra-high resolution images.

[0076] Based on the above embodiments, the ultra-high resolution image block scheduling system includes a command reading module, a batch processing command management module, an image block algorithm module, a task scheduling module, a hardware abstraction layer, and an interrupt handling module.

[0077] Based on the above embodiments, the target image segmentation task description information determination module 310 can be specifically used to: receive image segmentation tasks to be processed from the application layer in real time through the task submission queue; instruct the reading command module to parse and process the image segmentation tasks to be processed to obtain the target image segmentation task description information, and send the target image segmentation task description information to the batch processing command management module; instruct the batch processing command management module to store the received target image segmentation task description information in the command cache pool inside the management module, and receive the scheduling from the task scheduling module in real time.

[0078] Based on the above embodiments, the target image segmentation task description information includes at least one of the following: source image memory address, output image memory address, and image row processing step size; the segmentation subtask description information includes at least one of the following: segmentation point in the source image, segmentation point in the output image, image processing operation parameters, number of segmentation blocks, and segmentation start address.

[0079] Based on the above embodiments, the segmented subtask description information determination module 320 may specifically include: a target segmentation block quantity calculation unit, used to calculate the target segmentation block quantity according to the target image segmentation task description information by using the segmentation block data calculation method in the image segmentation algorithm module when an image segmentation execution command is received; a source image segmentation point and output image segmentation point calculation unit, used to perform image interpolation processing by using a preset bilinear filtering interpolation method according to the target image segmentation task description information, and combined with a preset image multi-functional segmentation method, to calculate the source image segmentation point and output image segmentation point corresponding to each segmented subtask respectively; and a parent-child task dependency table and subtask dependency table establishment unit, used to put the segmented subtask description information and subtask state machine into the task scheduler cache pool, and establish a parent-child task dependency table and a subtask dependency table.

[0080] Based on the above embodiments, the target segmentation block quantity calculation unit can be specifically used to: obtain a preset reserved processing edge width based on the source image width, expected output image width, maximum image width processed by the hardware device in a single operation, and minimum image width processed by the hardware device in a single operation, as described in the target image segmentation task description information; when it is determined that the source image width is greater than the maximum image width processed by the hardware device in a single operation, and the source image width is greater than or equal to the expected output image width, then the reserved processing edge width is subtracted from the maximum image width processed by the hardware device in a single operation to obtain a first difference, and the source image width is divided by the first difference to calculate the target segmentation block quantity; when it is determined that the expected output image width is greater than the maximum image width processed by the hardware device in a single operation, and the source image width is less than the expected output image width, then the expected output image width is divided by the first difference to calculate the target segmentation block quantity.

[0081] Based on the above embodiments, the calculation unit for the segmentation points in the source image and the segmentation points in the output image can be specifically used to: calculate the input width and output width of each block, the segmentation points in the source image and the segmentation points in the output image using the image edge filling segmentation sub-method in the image multi-functional segmentation method, based on the obtained maximum image width processed by the hardware device in a single operation, minimum image width processed by the hardware device in a single operation, left boundary filling width, right boundary filling width, remaining unsegmented source image width and remaining unsegmented blocks; or calculate the input width and output width of each block, the segmentation points in the source image and the segmentation points in the output image using the image scaling sub-method in the image multi-functional segmentation method. The block method calculates the output width of each block, the segmentation point in the source image, and the segmentation point in the output image based on the obtained horizontal output-to-input scaling ratio, the image width input to the image scaling module, the total source image width processed from the first block to the present, and the total image width of the forward block output. Among them, the image color gamut conversion operator block method divides the image into blocks according to the maximum image width processed by the hardware device in a single operation; and the image cropping operator block method uses the cropping start point as the starting position for reading data from the target hardware device, and takes the average of the actual output image width as the input and output width of the image block.

[0082] Based on the above embodiments, the scheduling processing module 330 can be specifically used to: when a task scheduling command is received, instruct the task scheduling module to determine whether the currently scheduled task is a parent task type; if not, perform scheduling processing on the currently scheduled task; if it is a parent task type, obtain at least one currently scheduled subtask corresponding to the currently scheduled task from the task scheduler cache pool according to the parent-child task dependency table and the subtask dependency table, through the task scheduling method; determine whether there is an idle hardware device in the hardware scheduler list corresponding to the task scheduling module; if so, call the hardware abstraction layer to send the currently scheduled task and each currently scheduled subtask to the target hardware device for execution, and switch the current task state machine to the execution state; if not, put the currently scheduled task and each currently scheduled subtask into the hardware scheduler. The task cache pool is used to set the current task state machine to a waiting state for hardware execution. The hardware scheduler's execution granularity is one parent task. The task scheduler cache pool will not be read until all sub-tasks in the parent-child task dependency table have been fully executed. This continues until all sub-tasks are scheduled, at which point the image segmentation task to be processed is scheduled to the target hardware device. During task scheduling, the interrupt handling module is instructed to receive interrupt commands from the target hardware device in real time. A task timeout detection mechanism is set in the ultra-high resolution image segmentation scheduling system. If the scheduling time exceeds a preset time threshold, an alarm is sent to the application layer. The target hardware device reads data in steps using RDMA and WDMA, and includes multiple dedicated hardware units with different functions.

[0083] The image block scheduling device for ultra-high resolution images provided in this embodiment of the invention can execute the image block scheduling method for ultra-high resolution images provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0084] Example 4 Figure 4 A schematic diagram of an electronic device 10, which can be used to implement Embodiment 4 of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0085] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0086] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0087] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the tile scheduling method for ultra-high resolution images.

[0088] In some embodiments, the ultra-high resolution image block scheduling method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the ultra-high resolution image block scheduling method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the ultra-high resolution image block scheduling method by any other suitable means (e.g., by means of firmware).

[0089] The method includes: receiving and parsing the target image segmentation task description information according to the image segmentation task to be processed issued by the application layer in real time; wherein, the target image segmentation task description information is stored in the command cache pool inside the management module; when an image segmentation execution command is received, the image segmentation algorithm module is instructed to parse the target image segmentation task description information to obtain segmentation subtask description information, and the segmentation subtask description information is put into the task scheduler cache pool, and a parent-child task dependency table and a subtask dependency table are established; when a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the subtask dependency table, using a preset task scheduling method, until all segmentation subtasks are scheduled, thereby scheduling the image segmentation task to be processed to the target hardware device.

[0090] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0092] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0094] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0095] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0096] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0097] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0098] Example 5 Embodiment 5 of the present invention also provides a computer-readable storage medium, wherein the computer-readable instructions, when executed by a computer processor, are used to execute a method for scheduling image block partitioning for ultra-high resolution images. The method includes: receiving and parsing target image block task description information based on image block tasks to be processed issued by the application layer in real time; wherein the target image block task description information is stored in a command cache pool within a management module; when an image block execution command is received, the image block algorithm module is instructed to parse the target image block task description information to obtain block sub-task description information, and the block sub-task description information is placed in a task scheduler cache pool, and a parent-child task dependency table and a sub-task dependency table are established; when a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all block sub-tasks are scheduled, thereby scheduling the image block tasks to be processed to the target hardware device.

[0099] Of course, the computer-executable instructions provided in the embodiments of the present invention, which include a computer-readable storage medium, are not limited to the method operations described above, but can also execute related operations in the block scheduling method for ultra-high resolution images provided in any embodiment of the present invention.

[0100] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0101] It is worth noting that in the above embodiments of the block scheduling method for ultra-high resolution images, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0102] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A block scheduling method for ultra-high resolution images, characterized in that, include: It receives and parses the target image segmentation task description information in real time according to the image segmentation task to be processed issued by the application layer; Specifically, the target image block task description information is stored in the command cache pool inside the management module; When an image segmentation execution command is received, the image segmentation algorithm module is instructed to parse the target image segmentation task description information to obtain the segmentation subtask description information. The segmentation subtask description information is then placed in the task scheduler cache pool, and a parent-child task dependency table and a subtask dependency table are established. When a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all block sub-tasks are scheduled, thereby scheduling the image block task to be processed to the target hardware device.

2. The method according to claim 1, characterized in that, The ultra-high resolution image block scheduling system includes a command reading module, a batch processing command management module, an image block algorithm module, a task scheduling module, a hardware abstraction layer, and an interrupt handling module. The real-time reception and parsing of the image segmentation task description information obtained from the image segmentation task to be processed issued by the application layer includes: The system receives image segmentation tasks to be processed from the application layer in real time via a task submission queue. The reading command module is instructed to parse the image block task to be processed, obtain the target image block task description information, and send the target image block task description information to the batch processing command management module; The batch processing command management module is instructed to store the received target image block task description information in the internal command cache pool of the management module, and to receive the scheduling from the task scheduling module in real time.

3. The method according to claim 2, characterized in that, The target image segmentation task description information includes at least one of the following: source image memory address, output image memory address, and image row processing step size; The segmentation subtask description information includes at least one of the following: segmentation points in the source image, segmentation points in the output image, image processing operation parameters, number of segments, and segmentation start address.

4. The method according to claim 3, characterized in that, When an image segmentation execution command is received, the image segmentation algorithm module is instructed to parse the target image segmentation task description information to obtain segmentation subtask description information. This subtask description information is then placed in the task scheduler cache pool, and a parent-child task dependency table and a subtask dependency table are established, including: When an image segmentation execution command is received, the number of target segments is calculated based on the target image segmentation task description information using the segmentation data calculation method in the image segmentation algorithm module. Based on the target image segmentation task description information, the image is interpolated using a preset bilinear filtering interpolation method, and combined with a preset multi-functional image segmentation method, the segmentation points in the source image and the segmentation points in the output image corresponding to each segmentation sub-task are calculated respectively. The segmented subtask description information and subtask state machine are placed into the task scheduler cache pool, and parent-child task dependency tables and subtask dependency tables are established.

5. The method according to claim 4, characterized in that, The method for calculating the segmentation data in the image segmentation algorithm module, based on the target image segmentation task description information, calculates the number of target segments, including: Based on the source image width, expected output image width, maximum image width processed by the hardware device in a single operation, and minimum image width processed by the hardware device in a single operation, as described in the target image segmentation task description information; Obtain the preset reserved processing edge width; When it is determined that the width of the source image is greater than the maximum image width that the hardware device can process at one time, and the width of the source image is greater than or equal to the width of the expected output image, the first difference is obtained by subtracting the reserved processing edge width from the maximum image width that the hardware device can process at one time, and the target number of segments is calculated by dividing the width of the source image by the first difference. When it is determined that the width of the expected output image is greater than the maximum image width that the hardware device can process in a single operation, and the width of the source image is less than the width of the expected output image, the number of target segmentation blocks is calculated by dividing the width of the expected output image by the first difference.

6. The method according to claim 5, characterized in that, The method, combined with a preset multi-functional image segmentation method, calculates the segmentation points in the source image and the output image corresponding to each segmentation sub-task, including: The image edge filling segmentation sub-method in the image multi-functional segmentation method calculates the input and output width of each segment, the segmentation point in the source image, and the segmentation point in the output image based on the obtained maximum image width processed by the hardware device in a single operation, minimum image width processed by the hardware device in a single operation, left boundary filling width, right boundary filling width, width of the remaining unsegmented source image, and number of remaining unsegmented segments. Alternatively, the image scaling and segmentation sub-method in the image multi-functional segmentation method can be used to calculate the output width of each segment, the segmentation point in the source image, and the segmentation point in the output image based on the obtained horizontal output to input scaling ratio, the image width input to the image scaling module, the total source image width processed from the first segment to the present, and the total image width of the forward segment output. Among them, the image color gamut conversion operator block segmentation method divides the image into blocks according to the maximum image width processed by the hardware device in a single operation. The image cropping operator block method uses the cropping start point as the starting position for reading data from the target hardware device, and takes the average of the actual output image width as the input and output width of the image block.

7. The method according to claim 6, characterized in that, When a task scheduling command is received, the task scheduling module is instructed to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table, using a preset task scheduling method, until all segmented sub-tasks are scheduled, thereby scheduling the image segmentation task to be processed to the target hardware device, including: When a task scheduling command is received, the task scheduling module is instructed to determine whether the current task to be scheduled is of the parent task type. If not, the current task to be scheduled is scheduled. If it is a parent task type, then according to the parent-child task dependency table and the sub-task dependency table, the task scheduling method is used to obtain at least one currently scheduled sub-task corresponding to the currently scheduled task from the task scheduler cache pool. Determine if there are any idle hardware devices in the hardware scheduler list corresponding to the task scheduling module. If so, call the hardware abstraction layer to send the currently scheduled task and each currently scheduled sub-task block to the target hardware device for execution, and switch the current task state machine to the execution state. If not, the current task to be scheduled and each of the current subtasks to be scheduled are placed into the task cache pool of the hardware scheduler, and the state of the current task state machine is set to the waiting state for hardware execution. The hardware scheduler executes at the level of a parent task. Before the subtasks in the parent-child task dependency table are fully executed, the task scheduler cache pool will not be read again. Until all sub-tasks have been scheduled, the image sub-tasks to be processed are scheduled to the target hardware device. During the task scheduling and execution process, the interrupt handling module is instructed to receive interrupt commands issued by the target hardware device in real time. In the ultra-high resolution image block scheduling system, a task timeout detection mechanism is set up. If the scheduling time of a task exceeds a preset time threshold, an alarm is sent to the application layer. The target hardware device reads data in steps using RDMA and WDMA, and contains multiple dedicated hardware units with different functions.

8. A block scheduling device for ultra-high resolution images, characterized in that, include: The target image block task description information determination module is used to receive and parse the target image block task description information in real time according to the image block task to be processed issued by the application layer. Specifically, the target image block task description information is stored in the command cache pool inside the management module; The sub-task description information determination module is used to, when receiving an image segmentation execution command, instruct the image segmentation algorithm module to parse the target image segmentation task description information, obtain the sub-task description information, put the sub-task description information into the task scheduler cache pool, and establish a parent-child task dependency table and a sub-task dependency table. The scheduling processing module is used to instruct the task scheduling module to schedule the tasks in the task scheduler cache pool according to the parent-child task dependency table and the sub-task dependency table when a task scheduling command is received, through a preset task scheduling method, until all block sub-tasks are scheduled, so as to schedule the image block task to be processed to the target hardware device.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a block scheduling method for ultra-high resolution images as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute a block scheduling method for ultra-high resolution images as described in any one of claims 1-7.

Images