Eye image processing method and eye image processing device

Abstract

The invention relates to an eye image processing method and an eye image processing device. The method comprises the following steps: acquiring target reference coefficients corresponding to a plurality of execution engines; the execution engine comprises at least two of a WebGPU-based image processing execution engine, a WebGL-based image processing execution engine and a central processing unit (CPU)-based image processing execution engine; inputting the target reference coefficients corresponding to the plurality of execution engines into an explicit prediction model for time consumption prediction to obtain current task time consumption of each execution engine; selecting a target execution engine from the plurality of execution engines according to the current task time consumption of each execution engine and a preset time threshold; and performing image processing on the to-be-processed eye image through the target execution engine. By adopting the method, the problems occurring in the image processing process by adopting a fixed and single execution engine can be avoided, so that the processing efficiency and the processing effect of the eye image can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for processing eye images. Background Technology

[0002] With the development of image processing technology, eye imaging technology has emerged. Eye imaging technology can acquire eye images, and then perform image processing such as rendering and filtering on the eye images to obtain eye images with better image quality.

[0003] In related technologies, a preset execution engine is usually used for image processing such as rendering and filtering.

[0004] However, current methods for processing eye images suffer from low processing efficiency. Summary of the Invention

[0005] Therefore, it is necessary to provide a method and apparatus for processing eye images to address the aforementioned technical problems, thereby improving the processing efficiency of eye images.

[0006] In a first aspect, this application provides a method for processing eye images, comprising:

[0007] Obtain target baseline coefficients corresponding to multiple execution engines; the execution engines include at least two of the following: WebGPU-based image processing execution engines, WebGL-based image processing execution engines, and CPU-based image processing execution engines.

[0008] The target baseline coefficients corresponding to the multiple execution engines are input into the explicit prediction model to predict the time consumption, thereby obtaining the current task consumption of each execution engine;

[0009] Based on the current task time and preset time threshold of each execution engine, a target execution engine is selected from the plurality of execution engines;

[0010] The target execution engine performs image processing on the eye image to be processed.

[0011] In one embodiment, obtaining the target baseline coefficients corresponding to multiple execution engines includes:

[0012] Obtain the initial baseline coefficients fitted from historical test results based on the standard execution engine;

[0013] The benchmark eye image was tested by rendering tasks using each of the aforementioned execution engines, and the test time results of each execution engine were obtained.

[0014] The engine calibration factor of each execution engine is determined based on the test time results of each execution engine;

[0015] The initial benchmark coefficients are calibrated according to the engine calibration factors of each execution engine to obtain the target benchmark coefficients corresponding to each execution engine.

[0016] In one embodiment, the explicit prediction model includes a pixel load estimation term and a filter complexity estimation term. The step of inputting the target baseline coefficients corresponding to the plurality of execution engines into the explicit prediction model for time consumption prediction, to obtain the current task consumption of each execution engine, includes:

[0017] The initial pixel load is determined based on the image resolution and the number of pixels processed; the number of pixels processed is calculated based on the rendering viewport size and the device pixel ratio.

[0018] For each execution engine, the pixel load sensitivity coefficient in the target baseline coefficient and the initial pixel load are input into the load estimation term to calculate the load time.

[0019] The initial filter complexity is determined based on the filter type, the number of filters, the convolution kernel radius, and the number of iterations.

[0020] For each execution engine, the complexity sensitivity coefficient in the target baseline coefficient and the initial filter complexity are input into the filter complexity estimation term to calculate the complexity time.

[0021] The current task time of each execution engine is determined based on the load time, the complexity time, and the task overhead time in the target baseline coefficient.

[0022] In one embodiment, selecting a target execution engine from the plurality of execution engines based on the current task duration and a preset time threshold of each execution engine includes:

[0023] Determine whether the current task time is less than the preset time threshold;

[0024] If the current task time is less than the preset time threshold, the execution engine corresponding to the current task time is determined as a candidate execution engine, and the target execution engine is determined based on the candidate execution engine.

[0025] In one embodiment, determining the target execution engine based on the candidate execution engines includes:

[0026] If the candidate execution engine includes one candidate execution engine, then the candidate execution engine is used as the target execution engine;

[0027] If the candidate execution engine includes multiple candidate execution engines, then the candidate execution engine with the shortest current task time among the multiple candidate execution engines is determined as the target execution engine.

[0028] In one embodiment, the image processing of the eye image to be processed through the target execution engine includes:

[0029] The eye image to be processed is parsed to obtain the standard data structure of the eye image to be processed;

[0030] Based on the standard data structure, the rendering context and execution language of the eye image to be processed are determined;

[0031] The unified parameter set of the task to be processed corresponding to the eye image to be processed is mapped to the engine parameter set that each execution engine can execute;

[0032] Based on the set of executable engine parameters of the target execution engine, the standard data structure, the rendering context, and the execution language, the eye image to be processed is rendered and / or filtered in the target execution engine to obtain the target eye image.

[0033] In one embodiment, the method further includes:

[0034] Based on relevant information of the target eye image, image compensation is performed on the target eye image; the relevant information includes at least one of the quality assessment result of the target eye image, scan type, and scan protocol, and the image compensation includes adjusting at least one of the brightness, contrast, gamma, and sharpening parameters.

[0035] In one embodiment, the target execution engine is the WebGPU-based image processing execution engine, and the method further includes:

[0036] Every preset time interval, the target execution engine is checked to see if it meets preset stability conditions. The preset stability conditions include no device loss event detected, and / or successful command submission, and / or successful health check of the WebGPU, and / or the test time of the WebGPU does not exceed a preset time threshold.

[0037] If the target execution engine meets the preset stability condition, then the target execution engine continues to render and filter the eye image to be processed;

[0038] If the target execution engine does not meet the preset stability conditions, the GPU's storage resources are recreated, the rendering pipeline is reinitialized, and the target execution engine is re-checked to see if it meets the preset stability conditions. If the target execution engine does not meet the preset stability conditions, the target execution engine is re-determined according to engine priority.

[0039] In one embodiment, the step of redetermining the target execution engine according to engine priority includes:

[0040] Determine whether the WebGL is available and whether the WebGL is in software rasterization rendering state;

[0041] If the WebGL is available and the WebGL is not in the software rasterization rendering state, then the WebGL is determined as the target execution engine;

[0042] If the WebGL is unavailable, or if the WebGL is in the software rasterization rendering state, then the CPU is identified as the target execution engine.

[0043] Secondly, this application also provides an eye image processing apparatus, comprising:

[0044] An acquisition module is used to acquire target baseline coefficients corresponding to multiple execution engines; the execution engines include at least two of the following: a WebGPU-based image processing execution engine, a WebGL-based image processing execution engine, and a CPU-based image processing execution engine.

[0045] The prediction module is used to input the target baseline coefficients corresponding to the multiple execution engines into the explicit prediction model to predict the time consumption, and obtain the current task consumption of each execution engine.

[0046] The selection module is used to select a target execution engine from the plurality of execution engines based on the current task consumption time and preset time threshold of each execution engine;

[0047] The image processing module is used to perform image processing on the eye image to be processed through the target execution engine.

[0048] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method in any of the embodiments of the first aspect described above.

[0049] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.

[0050] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.

[0051] The aforementioned eye image processing method and apparatus acquire target reference coefficients corresponding to multiple execution engines; input these coefficients into an explicit prediction model for time consumption prediction to obtain the current task consumption of each execution engine; select a target execution engine from among the multiple execution engines based on the current task consumption of each execution engine and a preset time threshold; and perform image processing on the eye image to be processed using the target execution engine. Since the execution engines include at least two of the following: WebGPU-based image processing execution engines, WebGL-based image processing execution engines, and CPU-based image processing execution engines, this embodiment can automatically and intelligently select a target execution engine from among the multiple execution engines in a pure front-end environment based on the current task consumption of each execution engine and a preset time threshold. This allows the use of the best-performing target execution engine for image processing, avoiding the problems that occur when using a fixed and single execution engine for image processing, thus improving the processing efficiency and effect for eye images. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is an application environment diagram of an eye image processing method in one embodiment;

[0054] Figure 2 This is a flowchart illustrating a method for processing eye images in one embodiment;

[0055] Figure 3 This is a flowchart illustrating the steps for determining the target baseline coefficient in one embodiment;

[0056] Figure 4 This is a flowchart illustrating the steps for determining the target execution engine in one embodiment;

[0057] Figure 5 This is a flowchart illustrating the image processing steps in one embodiment;

[0058] Figure 6 This is a schematic diagram illustrating the real-time adjustment of the fundus camera filter in one embodiment;

[0059] Figure 7 This is a flowchart illustrating the step of re-determining the target execution engine in one embodiment;

[0060] Figure 8 This is a flowchart illustrating a method for processing eye images in one optional embodiment.

[0061] Figure 9 This is a structural block diagram of an eye image processing device in one embodiment. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0064] With the development of image processing technology, eye imaging technology has emerged. Eye imaging technology can acquire eye images, and then perform image processing such as rendering and filtering on the eye images to obtain eye images with better image quality.

[0065] However, several issues exist with current technologies. First, access to eye images involves both the acquisition end and the remote end (browser and Google's shell app). Some browser versions cannot enable hardware acceleration, resulting in significant differences in device performance and inconsistent hardware acceleration availability. Second, current real-time rendering and complex filters for eye images on the Google kernel typically employ a single, pre-defined execution engine for rendering, filtering, and other image processing, leading to insufficient compatibility. Third, during image processing of eye images using a single execution engine, software rendering (such as SwiftShader) can degrade image processing performance, easily causing interactive stuttering and export failures under high-resolution and multi-filter tasks. Fourth, current filter outputs from different execution engines are inconsistent, lacking medical consistency guarantees. In summary, current methods for processing eye images suffer from low processing efficiency and poor processing capabilities.

[0066] After introducing the background technology of the eye image processing method provided in the embodiments of this application, the implementation environment involved in the eye image processing method provided in the embodiments of this application will be briefly described below.

[0067] The eye image processing method provided in this application embodiment can be applied to, for example, Figure 1 The computer device shown can be a terminal, or it can be a server. Its internal structure diagram can be as follows: Figure 1 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for processing eye images. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0068] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0069] In one embodiment, such as Figure 2 As shown, a method for processing eye images is provided, which can be applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0070] S201, obtain the target baseline coefficients corresponding to multiple execution engines.

[0071] The execution engine includes at least two of the following: a WebGPU-based image processing execution engine, a WebGL-based image processing execution engine, and a CPU-based image processing execution engine. WebGPU allows direct control of the GPU via the web, enabling direct GPU operation on the web interface. This leverages the GPU's high concurrency capabilities to achieve high-performance hardware acceleration and high-performance rendering of 5K fundus images. The target baseline coefficients may include α, β, and γ. α is the pixel load sensitivity coefficient; for easier interpretation, the initial pixel load P can be converted to megapixels (MP), and α can be expressed as ms / MP. β is the complexity sensitivity coefficient, measured in ms / complexity unit. γ is a constant representing task overhead time, such as scheduling, binding, and submission.

[0072] In this embodiment, optionally, the computer device can directly obtain the target benchmark coefficients corresponding to multiple execution engines from a preset database. Alternatively, the computer device can first obtain initial benchmark coefficients fitted from a large number of historical test results, and then calibrate the initial benchmark coefficients according to the test time results of each execution engine to obtain the target benchmark coefficients corresponding to multiple execution engines. Alternatively, the computer device can also directly calculate the target benchmark coefficients in real time through online linear regression analysis based on the time consumption data of historical rendering tasks on the local machine, without the need to preset initial benchmark coefficients, thereby achieving complete edge-side adaptation. Of course, this embodiment does not limit the specific implementation method for obtaining the target benchmark coefficients.

[0073] S202, input the target baseline coefficients corresponding to multiple execution engines into the explicit prediction model to predict the time consumption, and obtain the current task consumption of each execution engine.

[0074] The explicit prediction model is used to estimate the time taken to process the current frame image processing task on different execution engines. The image processing task may include rendering task or filtering task.

[0075] In this embodiment, the computer device can pre-build or obtain an explicit prediction model. Therefore, for each of the multiple execution engines, the computer device can input the target baseline coefficients (i.e., α, β, γ) corresponding to the execution engine into the explicit prediction model to predict the time consumption, thereby obtaining the current task consumption t_hat_engine for each execution engine.

[0076] S203: Select the target execution engine from multiple execution engines based on the current task time and preset time threshold of each execution engine.

[0077] The preset time threshold can be set according to actual conditions; however, this application embodiment does not limit the specific value of the preset time threshold. The target execution engine refers to the execution engine currently used to perform the image processing task.

[0078] In this embodiment, optionally, the computer device can determine the execution engine with the shortest current task time and directly select the execution engine with the shortest current task time from multiple execution engines as the target execution engine. Alternatively, the computer device can first determine whether the current task time of each execution engine is less than a preset time threshold, and then further select the target execution engine from the execution engines whose current task time is less than the preset time threshold. Of course, this embodiment does not limit the specific implementation method for selecting the target execution engine.

[0079] S204 performs image processing on the eye image to be processed through the target execution engine.

[0080] Among them, the eye image to be processed refers to the eye image that needs to be rendered or filtered in real time. For example, the eye image may include a 4K–8K resolution, four-channel (including No-Red mode) fundus color image.

[0081] In this embodiment, the computer device can create a buffer storage resource on the GPU side and upload the eye image to be processed to the buffer storage resource. Thus, based on the eye image to be processed in the buffer storage resource, resources and configurations executable by the target execution engine can be generated, and rendering tasks and / or filter tasks can be performed on the eye image to be processed in the target execution engine to obtain the target eye image.

[0082] In the above-described method for processing eye images, target baseline coefficients corresponding to multiple execution engines are obtained; these coefficients are input into an explicit prediction model for time prediction to obtain the current task time of each execution engine; a target execution engine is selected from the multiple execution engines based on the current task time of each execution engine and a preset time threshold; and image processing of the eye image to be processed is performed using the target execution engine. Since the execution engines include at least two of the following: WebGPU-based image processing execution engines, WebGL-based image processing execution engines, and CPU-based image processing execution engines, this embodiment can automatically and intelligently select a target execution engine from the multiple execution engines in a pure front-end environment based on the current task time of each execution engine and a preset time threshold. This allows the use of the best-performing target execution engine for image processing, avoiding the problems that occur when using a fixed and single execution engine for image processing, thus improving the processing efficiency and effect for eye images.

[0083] In one embodiment, such as Figure 3 As shown, an implementation method for obtaining target baseline coefficients corresponding to multiple execution engines is provided, namely, "obtaining target baseline coefficients corresponding to multiple execution engines" in S201 above, including:

[0084] S301, obtain the initial baseline coefficients fitted based on historical test results from the standard execution engine.

[0085] In this embodiment, multiple minimum rendering tasks can be performed in advance on the standard execution engine of a fixed / standard configuration machine. This allows initial baseline coefficients to be fitted offline based on a large amount of data. The fixed / standard configuration machine can then send the fitted initial baseline coefficients to the computer device. In this way, the computer device can obtain the initial baseline coefficients fitted based on historical test results from the standard execution engine.

[0086] The minimum rendering task refers to the rendering task performed on the fundus thumbnail. The image size of the fundus thumbnail is generally at least less than 2K, for example, the image size of the fundus thumbnail can be 1024×1024, so that the minimum rendering task can be executed faster. A large amount of data can include the quantified performance parameters of the device under a standard machine engine. Initial baseline coefficients can include α0, β0, and γ0, where α0 is the initial pixel load sensitivity coefficient, β0 is the initial complexity sensitivity coefficient, and γ0 is the initial constant term.

[0087] S302 tests the rendering tasks of the benchmark eye image using each execution engine, and obtains the test time results for each execution engine.

[0088] In this embodiment of the application, the computer device may pre-collect: hardwareConcurrency (the number of logical cores of the device's CPU), deviceMemory (the total memory size of the device), DPR (Device Pixel Ratio), screen / viewport size; and capability information: WebGPU: adapter.features / limits (the set of additional features supported by the GPU adapter or hardware limitation parameters); WebGL: WEBGL_debug_renderer_info (extended interface), MAX_TEXTURE_SIZE (the maximum texture size supported by the GPU).

[0089] Therefore, the computer device can perform a rendering task test on the benchmark eye image on each execution engine to obtain the test time result of each execution engine, bench_engine (ms). The benchmark eye image is the aforementioned fundus image. For example, a single rendering task test can include multiple rounds of rendering task tests.

[0090] S303 determines the engine calibration factor for each execution engine based on the test time results of each execution engine.

[0091] In this embodiment of the application, for each of the multiple execution engines, the computer device can calculate the corresponding engine calibration factor s(engine) based on the test time result of the execution engine. For example, if the performance of the execution engine is higher than that of the standard execution engine, the engine calibration factor is determined to be greater than 1; if the performance of the execution engine is lower than that of the standard execution engine, the engine calibration factor is determined to be less than 1; if the performance of the execution engine is equal to that of the standard execution engine, the engine calibration factor is determined to be equal to 1.

[0092] S304, calibrate the initial baseline coefficients according to the engine calibration factor of each execution engine to obtain the target baseline coefficients corresponding to each execution engine.

[0093] In this embodiment of the application, the computer device can perform scaling calibration processing on the initial reference coefficients (β0, γ0, α0) according to the engine calibration factor of each execution engine, so as to adjust the initial reference coefficients (β0, γ0, α0) and obtain more accurate target reference coefficients (i.e., α, β, γ) corresponding to each execution engine.

[0094] In one embodiment, the explicit prediction model includes a pixel load estimation term and a filter complexity estimation term. Based on this, a method for time consumption prediction is provided, namely, the "inputting the target baseline coefficients corresponding to multiple execution engines into the explicit prediction model for time consumption prediction to obtain the current task consumption of each execution engine" in S202 above, includes:

[0095] The initial pixel load is determined based on the image resolution and the number of pixels processed; the number of pixels processed is calculated based on the rendering viewport size and the device pixel ratio.

[0096] For each execution engine, the pixel load sensitivity coefficient and the initial pixel load in the target baseline coefficient are input into the load estimation term to calculate the load time.

[0097] The initial filter complexity is determined based on the filter type, number of filters, convolution kernel radius, and number of iterations.

[0098] For each execution engine, the complexity sensitivity coefficient and the initial filter complexity in the target baseline coefficients are input into the filter complexity estimation term to calculate the complexity time.

[0099] The current task time of each execution engine is determined based on the load time, complexity time, and task overhead time in the target baseline coefficient.

[0100] For example, the explicit prediction model can be expressed as equation (1):

[0101] t_hat_engine = α · P + β · C + γ (1)

[0102] Where t_hat_engine is the prediction time in milliseconds. P is the pixel load, P = W × H × DPR 2 Where W (width) and H (height) are the image resolution, and DPR is the device pixel ratio. 2 This reflects the increase in pixel area with the square of DPR. C is the filter complexity, C = r × N_ops, where r is the radius parameter (e.g., the radius of blur / sharpening), the larger r is, the more time-consuming a single operation; N_ops is the number of equivalent operations (e.g., the number of filters or rendering passes or a weighted count), the more times the image is processed, the greater the workload of the image processing task.

[0103] N_ops can be determined in two ways: First, simple counting (unweighted), where each enabled filter or rendering pass is recorded as 1, and N_ops is the sum of the number of enabled operations. A rendering pass refers to a complete rendering pipeline operation performed by the graphics processor. Generally, one filter may correspond to multiple rendering passes, or multiple filters may be combined into one rendering pass. Second, weighted counting, where different weights are assigned to different filters or rendering passes. For example, the weight of radius-related operations is greater than the weight of linear color transformations, and N_ops is the weighted sum based on these weights.

[0104] In this embodiment, the computer device can pre-calculate the number of pixels to be processed based on the render viewport size and device pixel ratio (DPR), and determine the initial pixel load P based on the image resolution (W, H) and the number of pixels to be processed. Therefore, for each execution engine, the computer device can input the pixel load sensitivity coefficient α from the target baseline coefficients and the initial pixel load P into the load estimation term (α·P) to calculate the load duration.

[0105] The computer device can also pre-determine the initial filter complexity C based on the filter type, number of filters, convolution kernel radius, and number of iterations. Therefore, for each execution engine, the computer device can input the complexity sensitivity coefficient β from the target baseline coefficients and the initial filter complexity C into the filter complexity estimation term (β·C) to calculate the complexity time.

[0106] Furthermore, the computer device can sum the load time, complexity time, and task overhead time γ in the target baseline coefficient to accurately determine the current task time t_hat_engine of each execution engine based on the explicit prediction model.

[0107] In one embodiment, such as Figure 4 As shown, an implementation method for selecting a target execution engine from multiple execution engines is provided, namely, "selecting a target execution engine from multiple execution engines based on the current task time and preset time threshold of each execution engine" in S203 above, including:

[0108] S401, determine whether the current task time is less than a preset time threshold. If the current task time is less than the preset time threshold, then execute S402.

[0109] S402, determine the execution engine corresponding to the current task time as a candidate execution engine, and determine the target execution engine based on the candidate execution engine.

[0110] For example, the preset time threshold τ_in can be 16.7ms (60FPS).

[0111] In this embodiment, the computer device can determine in real time or periodically whether the current task duration of each execution engine is less than a preset time threshold. If the current task duration is not less than the preset time threshold, the preset time threshold τ_in can be adjusted. For example, the preset time threshold τ_in can be relaxed to 33ms (30FPS) until at least one execution engine has a current task duration less than the preset time threshold. If at least one execution engine has a current task duration less than the preset time threshold, the execution engine with a current task duration less than the preset time threshold can be determined as a candidate execution engine.

[0112] Therefore, the target execution engine can be further determined based on the candidate execution engines. Optionally, any one of the candidate execution engines can be selected as the target execution engine; or, the candidate execution engine with the shortest current task execution time can be selected as the target execution engine.

[0113] In one optional embodiment, "determining the target execution engine based on the candidate execution engines" in S402 includes: if the candidate execution engines include one candidate execution engine, then the candidate execution engine can be directly used as the target execution engine. If the candidate execution engines include multiple candidate execution engines, then the candidate execution engine with the shortest current task time among the multiple candidate execution engines can be determined as the target execution engine.

[0114] In addition, hysteresis control logic can be set, which can be understood as follows: if it is necessary to switch back to the upper-level execution engine, the current task time t_hat_engine ≤ τ_out must be satisfied for 3 consecutive frames, where τ_out = τ_in − 2ms, in order to avoid image jitter.

[0115] In this embodiment, the target execution engine for the current task time can be intelligently selected from multiple execution engines based on the current task time obtained from the automatic collection of hardware and software parameters and micro-benchmark evaluation, combined with a preset time threshold. Therefore, it can be ensured that the target execution engine is the execution engine with the best performance.

[0116] In one embodiment, such as Figure 5 As shown, an implementation method for image processing of an eye image to be processed is provided, namely, "image processing of the eye image to be processed through the target execution engine" in S204 above, including:

[0117] S501, parse the eye image to be processed to obtain the standard data structure of the eye image to be processed.

[0118] In this embodiment, the computer device can perform encoding and decoding processing on the eye image to be processed during the image preprocessing stage, that is, parse the eye image to be processed into a standard data structure that WebGPU can understand. Here, the standard data structure refers to a data format compatible with the graphics API (Application Programming Interface), such as a texture object, buffer object, or bitmap data compatible with the graphics API.

[0119] S502, based on standard data structures, determines the rendering context and execution language of the eye image to be processed.

[0120] In this embodiment, the computer device can process standard data structures, such as through image reflection or decoding processes, to obtain processed standard data structures. Based on these processed standard data structures, a rendering context and execution language for the eye image to be processed can be generated. Furthermore, if the target execution engine is WebGPU or WebGL, the eye image to be processed, the standard data structure, the rendering context, and the execution language can all be written into the GPU's storage resources; if the target execution engine is CPU, the eye image to be processed, the standard data structure, the rendering context, and the execution language can all be written into the CPU's memory.

[0121] S503 maps the unified parameter set of the task to be processed corresponding to the eye image to be processed to the engine parameter set that each execution engine can execute.

[0122] The unified parameter mapping can be represented as: {brightness, contrast, gamma, radius, weight, saturation, channel mode, No-Red∈{Color,Red, Green, Blue, NoRed}}.

[0123] In this embodiment, the computer device can map the unified parameter set of the task corresponding to the eye image to be processed to the engine parameter set executable by each execution engine. This unified mapping of parameters across execution engines ensures that the output of the same eye image processed by different execution engines (WebGPU / WebGL / CPU) under the same parameters typically meets the following requirements: PSNR (Peak Signal-to-Noise Ratio) ≥ 40 dB, or ΔE00 (CIE DE2000 Color Difference Formula) ≤ 3.0. In a preferred embodiment, the output processed by different execution engines satisfies ΔE00 ≤ 3.0. This improves the consistency of the image processing process. It should be noted that the specific values ​​(such as 40 dB, 3.0, etc.) in this embodiment are merely examples and not mandatory numerical limitations.

[0124] S504, based on the set of engine parameters, standard data structures, rendering context and execution language that the target execution engine can execute, renders and / or filters the eye image to be processed in the target execution engine to obtain the target eye image.

[0125] In this embodiment of the application, the computer device can perform rendering and / or filtering tasks on the eye image to be processed in the target execution engine based on the set of engine parameters executable by the target execution engine, the eye image to be processed in the GPU's storage resources, standard data structures, rendering context and execution language, so as to obtain the target eye image.

[0126] In this embodiment, by uniformly mapping the parameters of each execution engine, it can be ensured that the filter performance is the same across engines, thereby improving the image performance of the target eye image obtained after rendering and / or filtering.

[0127] In one embodiment, an image compensation implementation method is provided, namely, the above-mentioned eye image processing method further includes:

[0128] Image compensation is performed on the target eye image based on relevant information from the target eye image.

[0129] The relevant information includes at least one of the following: quality assessment results of the target eye image, scan type, and scan protocol. Image compensation includes adjusting at least one of the following: brightness, contrast, gamma, and sharpening parameters.

[0130] In the embodiments of this application, such as Figure 6 As shown, Figure 6This is a schematic diagram illustrating real-time adjustment of a fundus camera filter in one embodiment. The computer device can further perform image compensation on the target eye image based on at least one of the following: quality assessment results of the target eye image, scan type (e.g., infrared image / black and white image), and scan protocol. For example, if the quality assessment results of the target eye image are poor, the computer device can further fine-tune at least one of the brightness, contrast, gamma, and sharpening parameters of the target eye image. Optionally, image compensation can be performed simultaneously on the R, G, and B channels of the target eye image (e.g.,...). Figure 6 (as shown in the image). Alternatively, image compensation can be performed on a single channel in the target eye image, for example, image compensation can be performed on the G channel of the target eye image alone. In this way, image compensation can further improve the image quality of the final output target eye image.

[0131] In one embodiment, such as Figure 7 As shown, the target execution engine is a WebGPU-based image processing execution engine. Based on this, a method for redefining the target execution engine is provided, namely, the above-mentioned eye image processing method further includes:

[0132] S601, at preset time intervals, check whether the target execution engine meets preset stability conditions. If the target execution engine meets the preset stability conditions, execute S602; if the target execution engine does not meet the preset stability conditions, execute S603.

[0133] S602 continues to render and filter the eye image under processing through the target execution engine.

[0134] S603 recreates the GPU's storage resources, reinitializes the rendering pipeline, and re-checks whether the target execution engine meets the preset stability conditions. If the target execution engine does not meet the preset stability conditions, the target execution engine is re-determined according to engine priority.

[0135] For example, the preset time period can be 1 second, or it can be 1 minute. Preset stability conditions include no detected device lost event, and / or successful command submission, and / or a successful health probe of the WebGPU, and / or the WebGPU test time not exceeding a preset time threshold. Engine priority is: WebGPU > WebGL, WebGL > CPU.

[0136] The "device lost" event refers to a situation during WebGPU operation where the current device becomes unavailable due to reasons such as underlying graphics driver reset, GPU process abnormality, resource exhaustion, or system power / graphics policy switching. This state sends a device loss notification to the application via `device.lost` (or an equivalent mechanism), indicating that the current device session has expired and graphics processing commands can no longer be reliably submitted. It's important to note that "device lost" does not equate to the GPU hardware disappearing, but rather the current WebGPU device session becoming invalid.

[0137] In this embodiment of the application, if the target execution engine is a WebGPU-based image processing execution engine, then during the execution of rendering tasks and / or filtering tasks, optionally, the computer device can detect whether the target execution engine meets the preset stability conditions at preset time intervals, that is, it can periodically detect whether the target execution engine meets the preset stability conditions. Alternatively, the computer device can also detect whether the target execution engine meets the preset stability conditions based on event triggering.

[0138] For example, a computer device can determine whether a device lost event was not detected, and / or whether a command was successfully submitted, and / or whether a health probe of the WebGPU passed, and / or whether the test time of the WebGPU did not exceed a preset time threshold. For instance, a device lost event can be detected every 1 second, and a health probe of the WebGPU can be performed.

[0139] If the target execution engine meets the preset stability conditions, indicating that no device loss event (devicelost) has been detected, and / or the command has been successfully submitted, and / or the health probe of the WebGPU has passed, and / or the test time of the WebGPU has not exceeded the preset time threshold, it means that the target execution engine performs well in executing rendering and / or filtering tasks. At this time, the target execution engine can continue to perform rendering and / or filtering tasks on the eye image to be processed.

[0140] If the target execution engine does not meet the preset stability conditions, it indicates that a device loss event (device elost) has been detected, and / or a command was not successfully submitted, and / or the WebGPU health probe failed, and / or the WebGPU test time exceeded the preset time threshold. This means that the target execution engine's performance in rendering and / or filtering tasks is poor. In this case, the GPU's storage resources can be recreated, and the rendering pipeline can be reinitialized. After a preset delay (e.g., 200ms), the target execution engine can be re-tested to see if it meets the preset stability conditions. If the target execution engine still does not meet the preset stability conditions, the target execution engine can be re-determined according to the engine priority of WebGPU, WebGL, and CPU.

[0141] In this embodiment, by detecting whether the target execution engine meets the preset stability conditions, the reconstruction of the device lost event and the health probe are realized, which can provide stability assurance for image processing tasks and avoid problems such as device hang-up or hibernation.

[0142] In one embodiment, an implementation method for redetermining the target execution engine is provided, namely, "redetermining the target execution engine according to engine priority" in S603 above, including:

[0143] Determine if WebGL is available and if WebGL is in software rasterization rendering mode.

[0144] If WebGL is available and WebGL is not in software rasterization rendering state, then WebGL is determined as the target execution engine.

[0145] If WebGL is unavailable, or if WebGL is in software rasterization rendering mode, then the CPU is identified as the target execution engine.

[0146] In this embodiment, if the target execution engine is a WebGPU-based image processing execution engine, the computer device can determine whether WebGL is available and whether it is in a software rasterization rendering state according to the engine priority of WebGPU, WebGL, and CPU. For example, if WebGPU performance is poor, it determines whether WebGL is available; if WebGL is available, it can then call the specific rendering acceleration flag of WebGL and determine whether WebGL is in a software rasterization rendering state based on the acceleration flag. For instance, the UNMASKED_RENDERER_WEBGL constant can be read, and the acceleration flag in the constant can be obtained by calling WebGL in Google Chrome. If the acceleration flag contains keywords such as swiftshader / llvmpipe / soft, it indicates that the current state is not true GPU acceleration (i.e., it is in a pseudo-acceleration state), indicating that WebGL is in a software rasterization rendering state. The software rasterization rendering state is generally understood as a pseudo-acceleration state in software.

[0147] If WebGL is available and not in software rasterization rendering, the computer device can designate WebGL as the target execution engine. If WebGL is unavailable, or if WebGL is in software rasterization rendering, the computer device can disable WebGL and designate the CPU as the target execution engine.

[0148] Furthermore, it can detect changes in the image size or resolution of the input eye image to be processed in real time or periodically. If the image size or resolution of the input eye image to be processed changes, the target execution engine can be re-determined according to engine priority. Additionally, it can detect whether the execution time of the image processing task exceeds a preset time threshold in real time or periodically. If the execution time of the image processing task exceeds the preset time threshold, it indicates that the task execution process is experiencing lag, and the target execution engine can also be re-determined according to engine priority.

[0149] In this embodiment, the target execution engine with better performance can be automatically and accurately reselected based on engine priority and the judgment process of soft rendering interception. Thus, during the execution of image processing tasks, the target execution engine with better performance can be selected in real time according to the execution status of the image processing tasks, thereby further improving the processing efficiency and effect for eye images.

[0150] In summary, all the above embodiments, such as Figure 8 As shown, a method for processing eye images is provided, which is applied to... Figure 1 The computer equipment shown includes:

[0151] S701, obtain the initial baseline coefficients fitted based on historical test results from the standard execution engine;

[0152] S702 performs rendering task tests on the benchmark eye image using each execution engine, and obtains the test time results for each execution engine.

[0153] S703 determines the engine calibration factor for each execution engine based on the test time results of each execution engine;

[0154] S704, calibrate the initial baseline coefficients according to the engine calibration factor of each execution engine to obtain the target baseline coefficients corresponding to each execution engine;

[0155] S705 determines the initial pixel load based on the image resolution and the number of pixels to be processed; the number of pixels to be processed is calculated based on the rendering viewport size and the device pixel ratio.

[0156] S706: For each execution engine, the pixel load sensitivity coefficient and the initial pixel load in the target baseline coefficient are input into the load estimation term to calculate the load time.

[0157] S707 determines the initial filter complexity based on filter type, number of filters, convolution kernel radius, and number of iterations;

[0158] S708: For each execution engine, the complexity sensitivity coefficient and the initial filter complexity in the target baseline coefficient are input into the filter complexity estimation term to calculate the complexity time.

[0159] S709 determines the current task time of each execution engine based on the load time, complexity time, and task overhead time in the target baseline coefficient;

[0160] S710, determine whether the current task time is less than a preset time threshold;

[0161] S711, If ​​the current task time is less than the preset time threshold, then the execution engine corresponding to the current task time is determined as the candidate execution engine;

[0162] S712, if the candidate execution engine includes a candidate execution engine, then the candidate execution engine is used as the target execution engine;

[0163] S713, If the candidate execution engine includes multiple candidate execution engines, then the candidate execution engine with the shortest current task time among the multiple candidate execution engines shall be determined as the target execution engine;

[0164] S714 parses the eye image to be processed to obtain the standard data structure of the eye image to be processed;

[0165] S715 determines the rendering context and execution language of the eye image to be processed based on standard data structures;

[0166] S716, map the unified parameter set of the task to be processed corresponding to the eye image to be processed to the engine parameter set that each execution engine can execute;

[0167] S717, based on the set of executable engine parameters, standard data structures, rendering context and execution language of the target execution engine, renders and / or filters the eye image to be processed in the target execution engine to obtain the target eye image;

[0168] S718, Based on relevant information of the target eye image, perform image compensation on the target eye image; the relevant information includes at least one of the following: the quality assessment result of the target eye image, the scan type, and the scan protocol; the image compensation includes adjusting at least one of the following: brightness, contrast, gamma, and sharpening parameters.

[0169] S719, every preset time interval, detect whether the target execution engine meets the preset stability conditions; the preset stability conditions include no device loss event detected, and / or, successful command submission, and / or, the health probe of WebGPU is passed, and / or, the test time of WebGPU does not exceed the preset time threshold;

[0170] S720: If the target execution engine meets the preset stability conditions, the target execution engine will continue to render and filter the eye image to be processed.

[0171] S721, If ​​the target execution engine does not meet the preset stability conditions, then recreate the GPU's storage resources, reinitialize the rendering pipeline, and re-check whether the target execution engine meets the preset stability conditions. If the target execution engine does not meet the preset stability conditions, then execute S722.

[0172] S722 determines whether WebGL is available and whether WebGL is in software rasterization rendering mode;

[0173] S723, if WebGL is available and WebGL is not in software rasterization rendering state, then WebGL is determined as the target execution engine;

[0174] S724: If WebGL is unavailable, or if WebGL is in software rasterization rendering mode, then the CPU is identified as the target execution engine.

[0175] It should be noted that the embodiments in this application are not general GPU adaptations for any network image, but are specifically designed for the characteristics of fundus medical images, mainly reflected in the following aspects:

[0176] First, the image type and resolution are specific to ophthalmology. The system processes multi-channel fundus images at 4K–8K resolution, including color, FAF (fundus autofluorescence imaging), FAF-G (green light-excited autofluorescence), FAF-B (blue light-excited autofluorescence), IR (infrared reflectance imaging), and "No-Red" ophthalmic imaging modes, rather than ordinary photographic images. Different channels are used to observe different lesions (such as macular degeneration, hemorrhage, exudation, etc.), so the filter parameter design and execution engine selection in this application are all based on the premise of "ensuring the visibility of lesion details."

[0177] Second, the filter type and consistency constraints are derived from the needs of ophthalmic diagnosis. The solution specifically supports commonly used ophthalmic observation modes such as "red light-free grayscale image" and sets image consistency thresholds (such as PSNR and ΔE00) across execution engines. The purpose is not "as long as the appearance is similar", but to ensure that the brightness and color differences of the lesion area under different execution engines are within the acceptable range for doctors, so as to avoid affecting the diagnostic judgment due to different graphics card environments.

[0178] Third, the pre-export health detection, self-healing, and degradation mechanisms are for the purpose of ensuring accountability in medical reports. Before performing high-resolution screenshots / exporting reports, this application's embodiments add WebGPU device health detection, reconstruction after failure, and degradation mechanisms because the exported images will be directly used as part of the medical records / reports. If blank images or abnormal color casts appear, it will affect clinical use, which is not a rigid requirement in ordinary image browsing scenarios.

[0179] Fourth, the triggering conditions for selecting multiple execution engines also depend on the diagnostic needs of fundus images. This application's embodiments use the interaction of "real-time dragging, zooming, and filter adjustment" under 4K–8K fundus images as a baseline, and combine the actual update frequency and continuous operation time used by ophthalmologists to design frame rate thresholds and hysteresis thresholds. The aim is to ensure that doctors can maintain a smooth, stable, and unobtrusive experience when magnifying lesion areas for observation.

[0180] In summary, the embodiments of this application are designed based on the acquisition mode, diagnostic process and reporting requirements of fundus medical images, and are not simply applying existing multi-rendering backend strategies to medical scenarios.

[0181] Based on the above embodiments, this application provides a method for adaptive rendering and image filtering acceleration of large color images (4K–8K) of the fundus based on the Google kernel (Chromium / WebView) using multiple execution engines. This method can adaptively select and stably control the operation of multiple execution engines such as WebGPU, WebGL, and CPU (OpenCV.js) according to the rendering method, device parameters or performance, browser version, etc., and can achieve the following effects:

[0182] First, by combining soft rendering recognition, device-lost self-healing reconstruction, exporting health detection retry, and cross-engine filter consistency threshold, the real-time performance, stability, and consistency of 4K–8K fundus color images can be guaranteed across different channels / devices.

[0183] Secondly, it can run on both the acquisition end and complex remote models, automatically selecting the optimal execution engine, which can improve the versatility and coverage of the equipment.

[0184] Third, the target execution engine is automatically selected based on the current task time of each execution engine, and the engine with the best performance is used for rendering, which can ensure low latency in 4K-8K level rendering and filtering processes, thus improving the real-time performance of image processing.

[0185] Fourth, device-lost or software rendering-induced faults can be identified, self-healed, or downgraded, thus ensuring the stability of the image processing process.

[0186] Fifth, low copy input and CPU fallback ensure a good experience for both powerful and weak machines, and are resource-friendly for various devices.

[0187] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0188] Based on the same inventive concept, this application also provides an eye image processing apparatus for implementing the eye image processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more eye image processing apparatus embodiments provided below can be found in the limitations of the eye image processing method described above, and will not be repeated here.

[0189] In one exemplary embodiment, such as Figure 9 As shown, an eye image processing device is provided, including: an acquisition module 31, a prediction module 32, a selection module 33, and an image processing module 34, wherein:

[0190] The acquisition module 31 is used to acquire target baseline coefficients corresponding to multiple execution engines; the execution engines include at least two of the following: a WebGPU-based image processing execution engine, a WebGL-based image processing execution engine, and a CPU-based image processing execution engine.

[0191] The prediction module 32 is used to input the target baseline coefficients corresponding to multiple execution engines into the explicit prediction model to predict the time consumption and obtain the current task consumption of each execution engine.

[0192] The selection module 33 is used to select a target execution engine from multiple execution engines based on the current task time and preset time threshold of each execution engine.

[0193] Image processing module 34 is used to perform image processing on the eye image to be processed through the target execution engine.

[0194] In one embodiment, the acquisition module 31 includes:

[0195] The initial baseline coefficient acquisition unit is used to acquire the initial baseline coefficients fitted based on historical test results from the standard execution engine.

[0196] The rendering task test unit is used to perform rendering task tests on the benchmark eye image through each execution engine, and obtain the test time results of each execution engine.

[0197] The engine calibration factor determination unit is used to determine the engine calibration factor of each execution engine based on the test time results of each execution engine.

[0198] The calibration unit is used to calibrate the initial reference coefficients according to the engine calibration factors of each execution engine, so as to obtain the target reference coefficients corresponding to each execution engine.

[0199] In one embodiment, the explicit prediction model includes a pixel load estimate and a filter complexity estimate, and the prediction module 32 includes:

[0200] The initial pixel load determination unit is used to determine the initial pixel load based on the image resolution and the number of pixel processes; the number of pixel processes is calculated based on the rendering viewport size and the device pixel ratio.

[0201] The first input unit is used to input the pixel load sensitivity coefficient and the initial pixel load from the target baseline coefficient into the load estimation term for each execution engine, and calculate the load consumption time.

[0202] The initial filter complexity determination unit is used to determine the initial filter complexity based on the filter type, number of filters, convolution kernel radius, and number of iterations.

[0203] The second input unit is used to input the complexity sensitivity coefficient and the initial filter complexity from the target baseline coefficient into the filter complexity estimation term for each execution engine, and calculate the complexity time consumption.

[0204] The current task time determination unit is used to determine the current task time of each execution engine based on the load time, complexity time, and task overhead time in the target baseline coefficient.

[0205] In one embodiment, the selection module 33 includes:

[0206] The first determining unit is used to determine whether the current task time is less than a preset time threshold.

[0207] The second determining unit is used to determine the execution engine corresponding to the current task time as a candidate execution engine if the current task time is less than a preset time threshold, and to determine the target execution engine based on the candidate execution engine.

[0208] In one embodiment, the second determining unit is specifically used for:

[0209] If the candidate execution engine includes a candidate execution engine, then the candidate execution engine is used as the target execution engine;

[0210] If there are multiple candidate execution engines, the candidate execution engine with the shortest current task time will be determined as the target execution engine.

[0211] In one embodiment, the image processing module 34 includes:

[0212] The parsing unit is used to parse the eye image to be processed and obtain the standard data structure of the eye image to be processed;

[0213] The third determining unit is used to determine the rendering context and execution language of the eye image to be processed based on the standard data structure;

[0214] The mapping unit is used to map the unified parameter set of the task to be processed corresponding to the eye image to be processed to the engine parameter set that each execution engine can execute.

[0215] The execution unit is used to render and / or filter the eye image to be processed in the target execution engine according to the set of engine parameters, standard data structures, rendering context and execution language that the target execution engine can execute, so as to obtain the target eye image.

[0216] In one embodiment, the above-mentioned eye image processing apparatus further includes:

[0217] The image compensation module is used to perform image compensation on the target eye image based on relevant information of the target eye image; the relevant information includes at least one of the following: the quality assessment result of the target eye image, the scan type, and the scan protocol; the image compensation includes adjusting at least one of the following: brightness, contrast, gamma, and sharpening parameters.

[0218] In one embodiment, the target execution engine is a WebGPU-based image processing execution engine, and the aforementioned eye image processing apparatus further includes:

[0219] The detection module is used to detect whether the target execution engine meets the preset stability conditions at preset time intervals. The preset stability conditions include no device loss event detected, and / or successful command submission, and / or successful health probe of WebGPU, and / or the test time of WebGPU does not exceed the preset time threshold.

[0220] The continue execution module is used to continue rendering and filtering the eye image to be processed through the target execution engine if the target execution engine meets the preset stability conditions;

[0221] The re-determination module is used to recreate the GPU's storage resources and reinitialize the rendering pipeline if the target execution engine does not meet the preset stability conditions. It also re-detects whether the target execution engine meets the preset stability conditions. If the target execution engine does not meet the preset stability conditions, the target execution engine is re-determined according to engine priority.

[0222] In one embodiment, the re-determination module includes:

[0223] The fourth determining unit is used to determine whether WebGL is available and whether WebGL is in software rasterization rendering state;

[0224] The fifth determining unit is used to determine WebGL as the target execution engine if WebGL is available and WebGL is not in a software rasterization rendering state;

[0225] The sixth determining unit is used to determine the CPU as the target execution engine if WebGL is unavailable or if WebGL is in software rasterization rendering state.

[0226] Each module in the aforementioned eye image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0227] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 1 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a method for processing eye images. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0228] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0229] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the methods in the above-described method embodiments.

[0230] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the methods in the above-described method embodiments.

[0231] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the methods in the above-described method embodiments.

[0232] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis such as eye images, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0233] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0234] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0235] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing eye images, characterized in that, The method includes: Obtain target baseline coefficients corresponding to multiple execution engines; the execution engines include at least two of the following: WebGPU-based image processing execution engines, WebGL-based image processing execution engines, and CPU-based image processing execution engines. The target baseline coefficients corresponding to the multiple execution engines are input into the explicit prediction model to predict the time consumption, thereby obtaining the current task consumption of each execution engine; Based on the current task time and preset time threshold of each execution engine, a target execution engine is selected from the plurality of execution engines; The target execution engine performs image processing on the eye image to be processed.

2. The method according to claim 1, characterized in that, The process of obtaining the target baseline coefficients corresponding to multiple execution engines includes: Obtain the initial baseline coefficients fitted from historical test results based on the standard execution engine; The benchmark eye image was tested by rendering tasks using each of the aforementioned execution engines, and the test time results of each execution engine were obtained. The engine calibration factor of each execution engine is determined based on the test time results of each execution engine; The initial benchmark coefficients are calibrated according to the engine calibration factors of each execution engine to obtain the target benchmark coefficients corresponding to each execution engine.

3. The method according to claim 1, characterized in that, The explicit prediction model includes a pixel load estimation term and a filter complexity estimation term. The step of inputting the target baseline coefficients corresponding to the multiple execution engines into the explicit prediction model for time consumption prediction, to obtain the current task consumption of each execution engine, includes: The initial pixel load is determined based on the image resolution and the number of pixels processed; the number of pixels processed is calculated based on the rendering viewport size and the device pixel ratio. For each execution engine, the pixel load sensitivity coefficient in the target baseline coefficient and the initial pixel load are input into the load estimation term to calculate the load time. The initial filter complexity is determined based on the filter type, the number of filters, the convolution kernel radius, and the number of iterations. For each execution engine, the complexity sensitivity coefficient in the target baseline coefficient and the initial filter complexity are input into the filter complexity estimation term to calculate the complexity time. The current task time of each execution engine is determined based on the load time, the complexity time, and the task overhead time in the target baseline coefficient.

4. The method according to any one of claims 1-3, characterized in that, The step of selecting a target execution engine from the plurality of execution engines based on the current task duration and preset time threshold of each execution engine includes: Determine whether the current task time is less than the preset time threshold; If the current task time is less than the preset time threshold, the execution engine corresponding to the current task time is determined as a candidate execution engine, and the target execution engine is determined based on the candidate execution engine.

5. The method according to claim 4, characterized in that, The step of determining the target execution engine based on the candidate execution engines includes: If the candidate execution engine includes one candidate execution engine, then the candidate execution engine is used as the target execution engine; If the candidate execution engine includes multiple candidate execution engines, then the candidate execution engine with the shortest current task time among the multiple candidate execution engines is determined as the target execution engine.

6. The method according to any one of claims 1-3, characterized in that, The image processing of the eye image to be processed through the target execution engine includes: The eye image to be processed is parsed to obtain the standard data structure of the eye image to be processed; Based on the standard data structure, the rendering context and execution language of the eye image to be processed are determined; The unified parameter set of the task to be processed corresponding to the eye image to be processed is mapped to the engine parameter set that each execution engine can execute; Based on the set of executable engine parameters of the target execution engine, the standard data structure, the rendering context, and the execution language, the eye image to be processed is rendered and / or filtered in the target execution engine to obtain the target eye image.

7. The method according to claim 6, characterized in that, The method further includes: Based on relevant information of the target eye image, image compensation is performed on the target eye image; the relevant information includes at least one of the quality assessment result of the target eye image, scan type, and scan protocol, and the image compensation includes adjusting at least one of the brightness, contrast, gamma, and sharpening parameters.

8. The method according to any one of claims 1-3, characterized in that, The target execution engine is the WebGPU-based image processing execution engine, and the method further includes: Every preset time interval, the target execution engine is checked to see if it meets preset stability conditions. The preset stability conditions include no device loss event detected, and / or successful command submission, and / or successful health check of the WebGPU, and / or the test time of the WebGPU does not exceed a preset time threshold. If the target execution engine meets the preset stability condition, then the target execution engine continues to render and filter the eye image to be processed; If the target execution engine does not meet the preset stability conditions, the GPU's storage resources are recreated, the rendering pipeline is reinitialized, and the target execution engine is re-checked to see if it meets the preset stability conditions. If the target execution engine does not meet the preset stability conditions, the target execution engine is re-determined according to engine priority.

9. The method according to claim 8, characterized in that, The step of redetermining the target execution engine according to engine priority includes: Determine whether the WebGL is available and whether the WebGL is in software rasterization rendering state; If the WebGL is available and the WebGL is not in the software rasterization rendering state, then the WebGL is determined as the target execution engine; If the WebGL is unavailable, or if the WebGL is in the software rasterization rendering state, then the CPU is identified as the target execution engine.

10. An apparatus for processing eye images, characterized in that, The device includes: An acquisition module is used to acquire target baseline coefficients corresponding to multiple execution engines; the execution engines include at least two of the following: a WebGPU-based image processing execution engine, a WebGL-based image processing execution engine, and a CPU-based image processing execution engine. The prediction module is used to input the target baseline coefficients corresponding to the multiple execution engines into the explicit prediction model to predict the time consumption, and obtain the current task consumption of each execution engine. The selection module is used to select a target execution engine from the plurality of execution engines based on the current task consumption time and preset time threshold of each execution engine; The image processing module is used to perform image processing on the eye image to be processed through the target execution engine.

Images