Real-time color adjustment method and device for fluorescent slices based on WebGL shader and channel mixing matrix, and readable storage medium thereof

Abstract

This paper proposes a real-time color grading method for fluorescent slices based on WebGL shaders and channel blending matrices, a device, and a readable storage medium. This method employs a three-layer processing pipeline built on OpenSeaDragon. By monitoring tile rendering events, a WebGL shader supporting 16-bit images is injected. A 4×4 channel blending matrix is ​​used to achieve multi-channel cross-blending. This method incorporates S-curve nonlinear enhancement and employs tile-level parallelism and incremental update strategies to optimize performance. This method overcomes the problems of low color grading efficiency, 16-bit signal loss, and interaction lag associated with traditional methods, achieving real-time interaction, precise color control, and efficient processing.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology in the field of digital pathology, and in particular to a method and device for real-time color adjustment of fluorescent slices based on a WebGL shader and a channel mixing matrix, and a readable storage medium thereof. Background Art

[0002] In digital pathology diagnosis, multi-channel staining imaging of fluorescent sections provides a key basis for disease diagnosis. Its processing needs to achieve functions such as channel separation, precise color adjustment, and weak signal enhancement. However, existing technologies in this field have significant limitations:

[0003] Traditional desktop software (such as ImageJ and Fiji) uses an offline processing mode, which is cumbersome to operate. The color processing delay for 4K resolution slices often exceeds 30 seconds. It also cannot be integrated with the web-based digital pathology platform, seriously affecting the real-time diagnostic efficiency of doctors.

[0004] The Canvas2D-based front-end processing solution relies on single-threaded CPU computing. When processing 4K slices, full-image color adjustment takes 500ms to 2 seconds, which is prone to interactive lag. It also only supports 8-bit / channel image processing, resulting in the loss of weak signals in 16-bit fluorescent slices. It also lacks multi-channel cross-mixing capabilities, making it unable to meet the analysis requirements of complex staining signals.

[0005] OpenSeadragon, a commonly used web image rendering engine, only provides basic zooming, panning, and simple filtering functions. It lacks a multi-channel independent control interface, cannot load 16-bit high dynamic range images, and has low color adjustment efficiency, making it difficult to achieve accurate channel separation and nonlinear enhancement.

[0006] The above technical defects lead to problems such as poor real-time performance, insufficient accuracy, single function and performance bottlenecks in the current fluorescence slice color adjustment, which restricts the diagnostic efficiency and accuracy of the web-based digital pathology platform. Summary of the Invention

[0007] The embodiments of the present invention provide a method, device and readable storage medium for real-time color adjustment of fluorescent slices based on WebGL shaders and channel mixing matrices. The methods address the defects of the existing technologies in fluorescent slice color adjustment, such as poor real-time interactivity (high processing delay), inability to support weak signal retention of 16-bit images, lack of multi-channel cross-mixing function, and significant performance bottlenecks (stuttering and low efficiency) when processing high-resolution slices.

[0008] The core technology of this invention is mainly to build a three-layer processing pipeline based on the OpenSeadragon rendering engine, inject WebGL shaders by monitoring rendering events, combine 4×4 channel mixing matrix to realize multi-channel cross-mapping, and use S-shaped curves to achieve nonlinear enhancement. It also adopts optimization strategies such as tile-level parallel computing, incremental update and 16-bit texture compression to achieve real-time color adjustment and precise signal enhancement of fluorescence slices on the Web.

[0009] In a first aspect, the present invention provides a method for real-time color adjustment of fluorescent slices based on a WebGL shader and a channel mixing matrix, the method comprising the following steps:

[0010] A three-layer processing pipeline is built based on the OpenSeadragon rendering engine. The three-layer processing pipeline includes OpenSeadragon tile rendering, WebGL shader injection, and color processing engine operation.

[0011] Listen to the tile drawing events of the OpenSeadragon rendering engine and inject a custom WebGL shader when drawing each image tile. The custom WebGL shader supports the processing of 16-bit depth images.

[0012] Perform multi-pass color adjustments with custom WebGL shaders, including:

[0013] A 4×4 channel mixing matrix is ​​used to perform cross-mixing processing on the multi-channel signals of the fluorescent slice. The diagonal parameters of the 4×4 channel mixing matrix control the independent gain of each channel, and the off-diagonal parameters realize the cross-mapping of signals between channels.

[0014] Apply nonlinear enhancement algorithm to enhance multi-channel signals;

[0015] Real-time interaction is achieved using a performance optimization strategy, which includes splitting high-resolution fluorescence slices into multiple independent tiles and performing parallel computation on each tile using a WebGL shader. The parameters of the custom WebGL shader are updated and the corresponding tiles are redrawn only when the parameters of the 4×4 channel mixing matrix or the nonlinear enhancement algorithm change by more than a preset threshold.

[0016] Furthermore, the nonlinear enhancement algorithm is an S-type response curve algorithm, which controls curve characteristics through a sensitivity parameter and a midpoint parameter. The sensitivity parameter is used to control the steepness of the curve, and the midpoint parameter is used to set a signal enhancement threshold.

[0017] Furthermore, the sensitivity parameter has a value range of 5.0-15.0, and the midpoint parameter has a value range of 0.3-0.7.

[0018] Furthermore, the coefficients of the 4×4 channel mixing matrix range from [-1.0, 2.0].

[0019] Furthermore, the performance optimization strategy also includes: converting the 16-bit depth image into a compressed texture and transmitting it to the GPU to reduce the data transmission bandwidth.

[0020] Furthermore, the high-resolution fluorescence slice is segmented into a plurality of independent tiles, including: segmenting the high-resolution fluorescence slice into a plurality of independent tiles of 256×256 pixels, and setting overlapping areas between adjacent tiles to eliminate boundary artifacts.

[0021] Furthermore, the method also includes a preset color scheme reuse step: calculating the image fingerprint of the fluorescent slice, retrieving the color scheme of similar fluorescent slices in the preset library based on the image fingerprint, and applying the color scheme to the current fluorescent slice through a custom WebGL shader.

[0022] In a second aspect, the present invention provides a real-time color adjustment device for fluorescent slices based on a WebGL shader and a channel mixing matrix, comprising:

[0023] The OpenSeadragon rendering module is used to implement tiled rendering of fluorescent slices and provide basic zooming and panning functions;

[0024] The shader injection module is used to monitor the tile drawing events of the OpenSeadragon rendering module and inject custom WebGL shaders when each image tile is drawn. The custom WebGL shader supports the processing of 16-bit depth images.

[0025] The channel mixing processing module is used to call a 4×4 channel mixing matrix through a custom WebGL shader to perform cross-mixing processing on the multi-channel signals of the fluorescence slice. The diagonal parameters of the 4×4 channel mixing matrix control the independent gain of each channel, and the off-diagonal parameters realize the cross-mapping of signals between channels.

[0026] Nonlinear enhancement module, which is used to apply nonlinear enhancement algorithms through custom WebGL shaders to enhance multi-channel signals;

[0027] A performance optimization module is used to achieve real-time interaction through tile-level parallel computing and an incremental update strategy. Tile-level parallel computing involves dividing high-resolution fluorescence slices into multiple independent tiles and performing parallel computing on each tile through WebGL shaders. The incremental update strategy updates the parameters of the custom WebGL shader and redraws the corresponding tiles only when the change amplitude of the 4×4 channel mixing matrix or the parameters of the nonlinear enhancement algorithm exceeds a preset threshold.

[0028] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the above-mentioned real-time color adjustment method for fluorescent slices based on WebGL shader and channel mixing matrix.

[0029] In a fourth aspect, the present invention provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes the real-time color adjustment method of fluorescent slices based on the above-mentioned WebGL shader and channel mixing matrix.

[0030] The main contributions and innovations of the present invention are as follows:

[0031] 1. Significantly improved real-time interactivity: Through GPU parallel computing and tile-level processing of WebGL shaders, interactive latency for 4K slice color grading is controlled within 15ms, eliminating the lag issues associated with traditional solutions (such as Canvas2D processing taking over 500ms and offline software delays exceeding 30 seconds), achieving smooth real-time adjustment.

[0032] 2. Accurately preserve high dynamic range signals: Supports 16-bit deep image processing (65536 grayscale levels), avoiding the loss of weak signals caused by 8-bit processing. Combined with the nonlinear enhancement of the S-shaped curve, it can accurately enhance weak fluorescence signals.

[0033] 3. Flexible multi-channel mixing: By using the diagonal parameters (independent gain) and off-diagonal parameters (cross-mapping) of the 4×4 channel mixing matrix, this technology fills the gap in the lack of channel cross-mixing in existing technologies and meets the analysis requirements of complex staining signals.

[0034] 4. Efficient resource utilization: Incremental updates (updating only areas where parameter changes exceed 2%), 16-bit texture compression (reducing bandwidth by 50%), and tile-level parallel computing achieve zero data redundancy and reduce memory and bandwidth usage.

[0035] 5. Improved functional integrity: Integrating multi-channel control, nonlinear enhancement, and preset scheme reuse (based on image fingerprint retrieval), it breaks through the functional limitations of engines such as OpenSeadragon and provides a comprehensive color adjustment tool for web-based digital pathology diagnosis.

[0036] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0038] Figure 1 This is a flowchart of a method for real-time color adjustment of fluorescent slices based on a WebGL shader and a channel mixing matrix according to an embodiment of the present invention;

[0039] Figure 2 is a flowchart of GPU acceleration and intelligent preset processing for real-time color adjustment of fluorescent slices according to an embodiment of the present invention;

[0040] Figure 3 is a user interaction flow chart according to an embodiment of the present invention;

[0041] Figure 4 is a multi-strategy processing flow chart for 16-bit slice GPU load optimization according to an embodiment of the present invention;

[0042] Figure 5 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0044] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0045] Existing technologies have defects in fluorescence slice color adjustment, such as poor real-time interactivity (high processing delay), inability to support weak signal retention of 16-bit images, lack of multi-channel cross-mixing function, and significant performance bottlenecks (stuttering and low efficiency) when processing high-resolution slices.

[0046] Based on this, the present invention constructs a three-layer processing pipeline based on the OpenSeadragon rendering engine to solve the problems existing in the prior art.

[0047] Example 1

[0048] The present invention aims to propose a real-time color adjustment method for fluorescent slices based on WebGL shader and channel mixing matrix. Figures 1-4 , the method comprises the following steps:

[0049] Step 1: Build a three-layer processing pipeline based on the OpenSeadragon rendering engine. The three-layer processing pipeline includes OpenSeadragon tile rendering, WebGL shader injection, and color processing engine operation.

[0050] In this embodiment, a three-layer processing pipeline is constructed based on the OpenSeadragon rendering engine. The original high-resolution fluorescence slice (DCM format) is sequentially processed through OSD tile rendering, WebGL shader injection, and color processing engine operations, completing 4×4 channel blending matrix, S-curve enhancement, and 16-bit precision processing. For example:

[0051] First layer: OSD tile rendering

[0052] DCM (Digital Imaging and Communications in Medicine, also known as DICOM) is a pyramidal slice format that divides high-resolution images into tiles at different zoom levels (for example, 256×256 pixels), facilitating efficient loading and scaling on the web. The core function of the OSD rendering engine is to perform the basic rendering of these tiles, including loading tiles at different zoom levels, enabling user interactions such as zooming and panning, and converting raw DCM slices (raw high-resolution fluorescence slices) into tiled image streams that can be displayed on the web, providing the foundation for subsequent processing.

[0053] Second layer: WebGL shader injection

[0054] During the OSD tile rendering process, by listening to (or registering for) its "tile-drawn" event, a custom WebGL shader program is injected into the rendering process of each tile. WebGL is a GPU-based web graphics library. Shaders (GLSL programs) can be parallelized on the GPU, replacing traditional single-threaded CPU processing.

[0055] The key to this step is to transfer image processing from "CPU inefficient calculation" to "GPU parallel acceleration". At the same time, by declaring high-precision floating-point numbers (precision highp float) to support the processing of 16-bit depth images (65536 levels of grayscale), the problem of weak signal loss caused by traditional Canvas2D only supporting 8-bit images is solved.

[0056] Layer 3: Color processing engine operations

[0057] This layer is the specific image color grading and enhancement logic implementation layer, which executes the core algorithms through the injected WebGL shader, including:

[0058] 4×4 channel mixing matrix operation: The diagonal parameters of the matrix control the independent gains of the red, green, and blue channels, while the off-diagonal parameters realize cross-mapping of signals between channels (such as superimposing the red channel signal on the green channel), achieving precise multi-channel mixing;

[0059] S-curve enhancement: The image signal is nonlinearly enhanced using the sigmoid function, adjusting the sensitivity (controlling contrast) and midpoint (controlling brightness center) parameters to enhance weak fluorescence signals while avoiding overexposure.

[0060] 16-bit precision maintenance: Calculations are performed throughout the entire process based on 16-bit data precision, ensuring that weak signals in fluorescent slices (such as low-intensity fluorescent markers) are not lost.

[0061] Through the above three-layer pipeline, the original DCM slices complete the entire process of "tiled rendering → GPU acceleration preparation → precise color grading and enhancement" from loading to final display, ultimately achieving real-time interactive color grading of high-resolution fluorescence slices (delay ≤ 15ms), complete 16-bit signal preservation and flexible multi-channel control, solving the problems of low efficiency and insufficient precision of traditional technologies.

[0062] Step 2: Listen to the tile drawing events of the OpenSeadragon rendering engine and inject a custom WebGL shader when drawing each image tile. The custom WebGL shader supports the processing of 16-bit depth images.

[0063] In this embodiment, the method for implementing WebGL shader injection is as follows:

[0064] Listen for OSD rendering events: When OSD draws each image tile, inject a custom shader. The following is a code example:

[0065] viewer.addHandler('tile-drawn', (tile) =>{

[0066] const gl = tile.context;

[0067] / / Compile the GLSL shader containing the color processing logic

[0068] const shader = createShader(gl, fragmentShaderSource);

[0069] bindShaderParameters(shader, colorMatrix, curveParams); / / pass dynamic parameters

[0070] });

[0071] The above code uses OpenSeadragon's event monitoring mechanism to implement "monitoring" of the tile drawing process and inject custom shaders. It is the key bridge connecting the OSD rendering engine and WebGL GPU computing.

[0072] Shader Precision Extension: Declare high-precision floating-point numbers to support 16-bit images. The following is a code example:

[0073] precision highp float; / / Support 65,536 gray levels

[0074] uniform sampler2D u_image; / / Input tile texture

[0075] Declare high-precision floating-point calculation mode in the GLSL shader. A 16-bit fluorescence slice contains 65,536 grayscale levels (0-65,535). Low-precision modes (such as mediump or lowp) result in data truncation and loss of weak signals. Highp float supports high-precision floating-point arithmetic, fully preserving the grayscale levels of a 16-bit image. This solves the problem of weak signal loss caused by the traditional Canvas2D model's limited 8-bit (256 grayscale) support. In the shader, declare an input texture variable, u_image, to receive the image data of the current tile (stored as a WebGL texture). Through this variable, the shader can directly read the tile's pixel information, providing raw data for subsequent calculations such as channel blending and curve enhancement.

[0076] Step 3: Perform multi-channel color adjustment through custom WebGL shaders, including:

[0077] A 4×4 channel mixing matrix is ​​used to perform cross-mixing processing on the multi-channel signals of the fluorescent slice. The diagonal parameters of the 4×4 channel mixing matrix control the independent gain of each channel, and the off-diagonal parameters realize the cross-mapping of signals between channels.

[0078] Apply nonlinear enhancement algorithm to enhance multi-channel signals;

[0079] In this embodiment, the color transformation matrix operation is specifically as follows:

[0080] A 4×4 channel mixing matrix is ​​used to implement color transformation. The diagonal parameters control the R / G / B channel gains, and the off-diagonal parameters implement channel signal mixing. The GPU matrix is ​​updated in real time through JavaScript when the user operates. The code example is as follows:

[0081] function updateMatrix(a11, a12, a13, ...) {

[0082] gl.uniformMatrix4fv(matrixLocation, false,

[0083] new Float32Array([a11, a12, a13, 0, ...]) / / Pass to GPU );

[0085] updateMatrix(a11,a12,a13,...) is a callback function that responds to user channel adjustment operations (such as adjusting channel gain and cross-blending ratio). It is used to generate a new 4×4 channel blending matrix based on the matrix element values ​​entered by the user and pass it to the WebGL shader in real time, ensuring that the GPU uses the latest color parameters when processing the image.

[0086] In this way, this code implements a closed loop of "user interaction parameters → CPU calculation → GPU real-time application": after the user adjusts the channel mixing parameters, the matrix elements are updated in real time, quickly converted into a format recognizable by the GPU through this function and passed to the shader, so that subsequent image tile rendering can immediately apply the new mixing rules, thereby realizing real-time adjustment of the multi-channel colors of fluorescent slices, solving the problems of parameter update delays and interaction jams in traditional solutions.

[0087] In this embodiment, the nonlinear enhancement algorithm is specifically as follows:

[0088] In the shader, the sensitivity and midpoint of the S-shaped curve (sigmoid function) enhancement algorithm using a 16-bit grayscale image are the core parameters that control the curve characteristics. By adjusting these two parameters, precise enhancement and contrast adjustment of weak signals in fluorescent slices can be achieved. The specific code example is as follows:

[0089] float sigmoid(float value, float sensitivity, float midpoint) {

[0090] return 1.0 / (1.0 + exp(-sensitivity * (value - midpoint)));

[0091] }

[0092] void main() {

[0093] vec4 color = texture2D(u_image, v_texCoord);

[0094] color.r = sigmoid(color.r, u_sensitivity, u_midpoint); / / Enhance the R channel

[0095] ... / / Process G / B channels similarly

[0096] }

[0097] Sensitivity: controls the steepness (contrast) of the curve, the default value is 10.0;

[0098] midpoint: Set the signal enhancement threshold (brightness center), the default value is 0.5;

[0099] The above code in the shader's main function performs pseudo-color conversion by combining the grayscale image with the wavelength of the emitted light, and applies the sigmoid function to the R, G, and B channels of the image respectively:

[0100] The pixel value of each channel (color.r / color.g / color.b) is used as the input value and is nonlinearly transformed by u_sensitivity (the sensitivity parameter passed) and u_midpoint (the midpoint parameter passed);

[0101] Finally, the converted channel values ​​are output to achieve independent enhancement of different channels (such as enhancing the signal of the red fluorescent marker alone while maintaining a soft transition of the green channel).

[0102] In this way, the combination of sensitivity and midpoint can flexibly adjust the contrast and enhancement range of the fluorescence signal, retaining the weak signal details of the 16-bit slices while highlighting the key diagnostic areas, solving the problem that traditional linear enhancement cannot take into account both strong and weak signals.

[0103] Preferably, this step further includes:

[0104] Preset system: Generate a fingerprint by calculating the Perceptual Hash of the image, and retrieve the color palette of similar images in the preset library to achieve fast color adjustment. The preset system is an intelligent reuse mechanism designed to achieve fast color adjustment of fluorescence slices. Through image fingerprint generation and similarity retrieval, it quickly matches the historical color palette, reducing the manual adjustment cost of users. Its core logic includes two parts: "image fingerprint generation" and "parameter matching", which are as follows:

[0105] 1. Image fingerprint generation:

[0106] / / Calculate the Perceptual Hash of the image (simplified version)

[0107] const imageData = downscaleTo8x8(slice); / / Downsample to 8x8

[0108] const avg = averagePixel(imageData);

[0109] let hash = 0; <00​​​​​​​​​​​​​​​​​​​​​2. Parameter matching: Retrieve color schemes with similar hash values ​​(e.g., within ±5 bits of difference) from the preset library. The preset library stores a large number of "image fingerprint-color scheme" correspondences for historical fluorescence slices (color schemes include 4×4 channel mixing matrix parameters, S-curve sensitivity and midpoint, etc.). When processing a new slice:

[0118] Calculate the image fingerprint of the current slice;

[0119] Retrieve historical fingerprints from the preset library that differ by ≤5 bits from the current fingerprint (i.e., the number of binary bits that differ between the two 64-bit hash values ​​is ≤5), and consider the corresponding sections to be of similar type (same tissue, same staining scheme);

[0120] Directly reuse the color scheme of similar slices retrieved and apply it to the current slice through WebGL shaders, achieving fast color adjustment with "zero manual adjustment" or "fine-tuning".

[0121] In this way, the preset system avoids repeated color adjustment operations on similar fluorescent slices through efficient comparison of image fingerprints. Especially in medical scenarios, the staining characteristics of the same type of tissue slices are often similar. This mechanism can significantly reduce the interaction cost of doctors. Combined with the basic technologies of real-time color adjustment (such as shader acceleration and matrix operations), it further improves the efficiency and convenience of fluorescent slice processing on the Web.

[0122] Step 4: Implement real-time interaction using a performance optimization strategy. This strategy includes: dividing the high-resolution fluorescence slice into multiple independent tiles and performing parallel computation on each tile using a WebGL shader. Only when the change in the parameters of the 4×4 channel mixing matrix or the nonlinear enhancement algorithm exceeds a preset threshold, the parameters of the custom WebGL shader are updated and the corresponding tiles are redrawn.

[0123] In this embodiment, if Figure 2 As shown, the user can adjust the parameters by dragging the slider, setting the mixing ratio, clicking the preset, etc. The system only re-executes the shader for the tiles in the visible area to reduce the GPU load. The details are as follows:

[0124] 1. User operation (slider / mixing ratio / preset)

[0125] Trigger condition: The user initiates a color adjustment operation through interface controls (such as the channel gain slider, the mixing ratio input box, and the preset scheme button). The corresponding modified parameters include:

[0126] 4×4 channel mixing matrix (control channel independent gain, cross mapping);

[0127] S-curve parameters (sensitivity controls contrast, midpoint controls brightness center);

[0128] A full set of color grading parameters reused in preset schemes (directly load historical matrix and curve parameters).

[0129] 2. Detection parameter changes

[0130] Core function: Filter out "invalid operations" (such as users accidentally touching but parameters remain unchanged), triggering subsequent rendering only when parameters actually change, avoiding wasted GPU computing power.

[0131] Implementation logic: Use parameter monitoring (such as JavaScript Proxy and event callback) to compare the differences between new and old parameters (such as whether matrix elements have changed or whether curve parameters have changed).

[0132] 3. Update only the visible area tile shader

[0133] The definition of "visible area" is: the tiles currently displayed within the OpenSeadragon viewport (not the full-image tiles). High-resolution tiles are split into 256×256 pixel tiles, and tiles outside the viewport are not processed.

[0134] "Update shader" means passing new parameters (such as channel matrix and curve parameters) to the WebGL shader corresponding to the visible tile:

[0135] Update the 4×4 matrix via gl.uniformMatrix4fv;

[0136] Update scalar parameters such as sensitivity and midpoint through gl.uniform1f.

[0137] 4. Re-execute the visible area tile shader

[0138] GPU parallel computing: Based on the new parameters, the shader performs channel blending (4×4 matrix operation) and curve enhancement (sigmoid function) on each pixel of the visual tile, leveraging the parallel architecture of the GPU (thousands of threads computing simultaneously) to achieve millisecond-level processing.

[0139] Incremental update feature: Only visible tiles are recalculated instead of the entire image, reducing the computational effort by up to 90% (e.g., for a full image of 1000 tiles, only 100 tiles within the viewport need to be processed).

[0140] 5. Output the color-graded image

[0141] Tile synthesis: WebGL reassembles the processed visual tiles into a complete viewport image, renders it to the browser interface through Canvas, and users can see the color grading effect in real time.

[0142] In this embodiment, the performance optimization strategy includes tile-level parallelism and incremental update strategy, which can convert 16-bit slices into RGBA4444 texture transmission, improve computing efficiency and reduce bandwidth consumption.

[0143] Tile-level parallelism: Each 256×256 tile independently triggers GPU computation, leveraging WebGL's fragment-level parallelism. Specifically, tile-level parallelism is defined as follows:

[0144] In the OpenSeadragon (OSD) rendering pipeline, high-resolution slices are divided into several independent tiles (usually 256×256 pixels). Each tile triggers an independent WebGL shader instance on the GPU, leveraging the GPU's multi-core architecture to achieve parallel color calculations without data dependencies or communication between tiles. The following is a code example of tile-level parallelism:

[0145] viewer.addHandler('tile-drawn', (tile) =>{

[0146] / / Get the WebGL context of the current tile (separate context for each tile)

[0147] const gl = tile.context;

[0148] / / Tile-level shader isolation: compile dedicated shaders when drawing for the first time (avoid multi-tile state conflicts)

[0149] if (!gl._shader) {

[0150] gl._shader = compileShader(gl); / / Isolate compilation, each tile has an independent shader instance

[0151] }

[0152] / / Pass tile global offset parameters (used to calculate continuous texture coordinates across tiles)

[0153] gl.uniform2fv(u_tileOffset, tile.position);

[0154] / / Trigger tile drawing: parallel execution of GPU calculations based on independent shaders

[0155] drawTile(gl, tile);

[0156] });

[0157] It also includes medical imaging adaptation design:

[0158] Tile boundary processing: expand the 2-pixel overlap area (gl.texParameteri(gl.TEXTURE_WRAP_S,gl.CLAMP_TO_EDGE)) to eliminate seam artifacts.

[0159] 16-bit support: Each tile is loaded independently as gl.TEXTURE_2D, with the format set to gl.RGBA16UI (extend EXT_texture_norm16 is required).

[0160] The code example of the incremental update strategy is as follows:

[0161] let lastMatrix = null;

[0162] function onSliderChange() {

[0163] const newMatrix = calculateMatrix(); / / Generate a new 4×4 channel mixing matrix in real time based on the parameters currently adjusted by the user (such as channel gain, mixing ratio)

[0164] if (matrixChangeRate(newMatrix, lastMatrix)>0.02) { / / Calculate the parameter change rate of the new matrix and the previous matrix (such as the average difference rate of each matrix element) through the matrixChangeRate function. Only when the change rate exceeds 2%, perform subsequent update operations.

[0165] gl.uniformMatrix4fv(..., newMatrix); / / If the change reaches the target, call the WebGL uniformMatrix4fv method to pass the new matrix parameters to the GPU shader program and update the color calculation logic

[0166] lastMatrix = newMatrix; / / Save the current new matrix as the "last matrix" for comparison when the parameters change next time

[0167] }

[0168] }

[0169] The above code is the performance valve of the "user interaction → GPU parameter update" link. By filtering out minor changes, the GPU computing power is focused on parameter updates that truly affect visual effects. Together with the "update only visible area tiles" strategy, this achieves dual optimization and jointly ensures the smoothness of real-time color grading.

[0170] Specifically, the definition of incremental update is as follows:

[0171] When the user adjusts parameters (such as channel gain), the system only re-renders tiles whose changes exceed the threshold, and locally updates the shader parameters through WebGL's uniform mechanism to avoid redrawing the entire image.

[0172] Among them, 16-bit texture compression: convert the original 16-bit slices into RGBA4444 textures and transmit them to the GPU, reducing bandwidth by 50%.

[0173] Example 2

[0174] Based on the same concept, the present invention also proposes a real-time color adjustment device for fluorescent slices based on WebGL shaders and channel mixing matrices, comprising:

[0175] The OpenSeadragon rendering module is used to implement tiled rendering of fluorescent slices and provide basic zooming and panning functions;

[0176] The shader injection module is used to monitor the tile drawing events of the OpenSeadragon rendering module and inject custom WebGL shaders when each image tile is drawn. The custom WebGL shader supports the processing of 16-bit depth images.

[0177] The channel mixing processing module is used to call a 4×4 channel mixing matrix through a custom WebGL shader to perform cross-mixing processing on the multi-channel signals of the fluorescence slice. The diagonal parameters of the 4×4 channel mixing matrix control the independent gain of each channel, and the off-diagonal parameters realize the cross-mapping of signals between channels. The coefficient value range of the 4×4 channel mixing matrix is ​​[-1.0, 2.0].

[0178] Nonlinear enhancement module, which is used to apply nonlinear enhancement algorithms through custom WebGL shaders to enhance multi-channel signals;

[0179] A performance optimization module that enables real-time interaction through tile-level parallel computing and an incremental update strategy. Tile-level parallel computing involves dividing high-resolution fluorescence slices into multiple independent tiles and performing parallel computing on each tile using a WebGL shader. The incremental update strategy updates the parameters of the custom WebGL shader and redraws the corresponding tile only when the change in the parameters of the 4×4 channel mixing matrix or the nonlinear enhancement algorithm exceeds a preset threshold (such as 2%).

[0180] The preset retrieval module is used to calculate the image fingerprint of the fluorescent slice and retrieve the color scheme of similar fluorescent slices in the preset library based on the image fingerprint to achieve fast color adjustment.

[0181] Preferably, the nonlinear enhancement algorithm is an S-type response curve algorithm, which controls the curve characteristics through a sensitivity parameter and a midpoint parameter. The sensitivity parameter is used to control the steepness of the curve, and the midpoint parameter is used to set the signal enhancement threshold.

[0182] Preferably, the performance optimization module further includes a texture compression unit for converting the 16-bit depth image into a compressed texture and transmitting it to the GPU to reduce data transmission bandwidth.

[0183] Preferably, the size of the multiple independent tiles is 256×256 pixels, and an overlapping area is provided between adjacent tiles to eliminate boundary artifacts.

[0184] Example 3

[0185] This embodiment also provides an electronic device, referring to Figure 4 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0186] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0187] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Where appropriate, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0188] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0189] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the methods for real-time color adjustment of fluorescent slices based on a WebGL shader and a channel mixing matrix in the above embodiments.

[0190] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0191] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0192] The input / output device 408 is used to input or output information.

[0193] Example 4

[0194] This embodiment also provides a readable storage medium, which stores a computer program. The computer program includes program code for controlling a process to execute a process, and the process includes the real-time color adjustment method for fluorescent slices based on a WebGL shader and a channel mixing matrix according to Example 1.

[0195] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0196] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0197] The embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components that are configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, it should be noted at this point that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0198] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0199] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A real-time color adjustment method for fluorescent slices based on WebGL shader and channel mixing matrix, characterized in that: The following steps are involved: Building a three-layer processing pipeline based on the OpenSeadragon rendering engine, which includes OpenSeadragon tile rendering, WebGL shader injection, and color processing engine operations; Listening to tile drawing events of the OpenSeadragon rendering engine, injecting a custom WebGL shader when drawing each image tile, wherein the custom WebGL shader supports processing of 16-bit depth images; Multi-pass color adjustment is performed via the custom WebGL shader, including: A 4×4 channel mixing matrix is ​​used to perform cross-mixing processing on the multi-channel signals of the fluorescent slice. The diagonal parameters of the 4×4 channel mixing matrix control the independent gain of each channel, and the off-diagonal parameters realize signal cross-mapping between channels. Applying a nonlinear enhancement algorithm to perform enhancement processing on the multi-channel signal; Real-time interaction is achieved by adopting a performance optimization strategy, which includes: dividing the high-resolution fluorescence slice into multiple independent tiles, and performing parallel calculations on each tile through the WebGL shader; updating the parameters of the custom WebGL shader and redrawing the corresponding tiles only when the change amplitude of the parameters of the 4×4 channel mixing matrix or the nonlinear enhancement algorithm exceeds a preset threshold.

2. The method for real-time color adjustment of fluorescent slices based on WebGL shader and channel mixing matrix according to claim 1, characterized in that: The nonlinear enhancement algorithm is an S-type response curve algorithm, which controls the curve characteristics through a sensitivity parameter and a midpoint parameter. The sensitivity parameter is used to control the steepness of the curve, and the midpoint parameter is used to set the signal enhancement threshold.

3. The method for real-time color adjustment of fluorescent slices based on WebGL shader and channel mixing matrix according to claim 2, characterized in that: The sensitivity parameter has a value range of 5.0-15.0, and the midpoint parameter has a value range of 0.3-0.

7.

4. The method for real-time color adjustment of fluorescent slices based on WebGL shader and channel mixing matrix according to claim 1, characterized in that: The coefficient value range of the 4×4 channel mixing matrix is ​​[-1.0, 2.0].

5. The method for real-time color adjustment of fluorescent slices based on WebGL shader and channel mixing matrix according to claim 1, characterized in that: The performance optimization strategy also includes: converting the 16-bit depth image into a compressed texture and transmitting it to the GPU to reduce data transmission bandwidth.

6. The method for real-time color adjustment of fluorescent slices based on WebGL shader and channel mixing matrix according to claim 1, characterized in that: The high-resolution fluorescence slice is divided into a plurality of independent tiles, comprising: dividing the high-resolution fluorescence slice into a plurality of independent tiles of 256×256 pixels, and setting overlapping areas between adjacent tiles to eliminate boundary artifacts.

7. A method for real-time color adjustment of fluorescent slices based on WebGL shader and channel mixing matrix according to any one of claims 1 to 6, characterized in that: The method also includes a preset color scheme reuse step: calculating an image fingerprint of a fluorescent slice, retrieving a color scheme of a similar fluorescent slice in a preset library based on the image fingerprint, and applying the color scheme to the current fluorescent slice through the custom WebGL shader.

8. A real-time color adjustment device for fluorescent slices based on WebGL shader and channel mixing matrix, characterized in that: include: The OpenSeadragon rendering module is used to implement tiled rendering of fluorescent slices and provide basic zooming and panning functions; A shader injection module is used to monitor the tile drawing events of the OpenSeadragon rendering module and inject a custom WebGL shader when each image tile is drawn. The custom WebGL shader supports the processing of 16-bit depth images. A channel mixing processing module is used to call a 4×4 channel mixing matrix through the custom WebGL shader to perform cross-mixing processing on the multi-channel signals of the fluorescent slice, wherein the diagonal parameters of the 4×4 channel mixing matrix control the independent gain of each channel, and the off-diagonal parameters realize cross-mapping of signals between channels; A nonlinear enhancement module, configured to apply a nonlinear enhancement algorithm through the custom WebGL shader to perform enhancement processing on the multi-channel signal; A performance optimization module is configured to achieve real-time interaction through tile-level parallel computing and an incremental update strategy. The tile-level parallel computing involves dividing a high-resolution fluorescence slice into multiple independent tiles and performing parallel computing on each tile using the WebGL shader. The incremental update strategy involves updating the parameters of the custom WebGL shader and redrawing the corresponding tiles only when the change in the parameters of the 4×4 channel mixing matrix or the nonlinear enhancement algorithm exceeds a preset threshold.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the real-time color adjustment method for fluorescent slices based on WebGL shader and channel mixing matrix according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, wherein the process includes the real-time color adjustment method for fluorescent slices based on a WebGL shader and a channel mixing matrix according to any one of claims 1 to 7.

Images