A distributed cluster endoscope network aggregation method and endoscope system

Abstract

The application discloses a kind of distributed cluster endoscope network aggregation method and endoscope system, belong to medical equipment technical field, the method is based on the system implementation, the method includes determining dispatcher and establishing the self-organizing network of system network and dispatcher election step, complete each endoscope host available resource evaluation available resource value evaluation step, determine applicant and sharer resource allocation step, applicant and sharer data interaction to realize the data transmission step of computing power sharing.This scheme can be implemented in the environment without central server, realize the dynamic computing power aggregation and task scheduling between different performance endoscope host.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a distributed cluster endoscope network aggregation method and endoscope system. Background Technology

[0002] An endoscope host typically consists of core modules such as a power supply module, a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), and a GPU (Graphics Processing Unit). The power supply module is responsible for supplying power to the power-consuming modules inside the host. The CPU completes tasks such as task scheduling, resource allocation, control software, and human-machine interface. The FPGA performs preprocessing of the raw images and then hands them over to the GPU for subsequent image processing via the CPU. The processed images are then sent to the monitor for display at a certain frame rate.

[0003] With the development of medical technology, advanced imaging technologies such as 4K, 3D, fluorescence imaging, and HDR output are becoming increasingly popular. These technologies place extremely high demands on the image processing capabilities of endoscope hosts, especially GPU computing power.

[0004] Currently, hospitals typically procure one endoscope unit per operating room. However, the endoscope units installed in operating rooms may face the following problems: 1. Resource silos and waste: High-difficulty surgeries (such as hepatobiliary and pancreatic surgeries) require high-performance hosts to handle tasks such as 3D reconstruction, while low-difficulty surgeries (such as routine cholecystectomy) only require ordinary 2D images. If the endoscope host in the operating room is a high-performance host, its image processing computing power will be idle when performing non-high-difficulty surgeries. If the endoscope host in the operating room is a general-performance host, it will be unable to meet the requirements of surgery or examination when faced with complex cases.

[0005] 2. High procurement costs: For single operating rooms, in order to cope with the occasional demand for complex surgeries or detailed examinations, if the hospital requires that the endoscope main unit in each operating room meet all the needs of the surgery or examination, then each operating room needs to be equipped with a high-performance endoscope main unit (such as an endoscope main unit that integrates a variety of advanced imaging technologies and functions and has excellent hardware performance, such as a full-function endoscope main unit that can realize 4K ultra-high-definition imaging, 3D stereoscopic imaging, special light imaging, image enhancement, picture-in-picture, etc. Such a full-function endoscope main unit is usually equipped with high-performance hardware as standard), which leads to a significant increase in the hospital's endoscope main unit procurement costs.

[0006] In the prior art, patent application number CN202110302393.1, entitled "A Task Scheduling Method and Apparatus, Electronic Device and Storage Medium", provides a technical solution that uses the remaining computing resources of a terminal device to execute tasks to be processed, thereby alleviating the computing pressure on the cloud service area; and patent application number CN202111027775.4, entitled "A Task Processing System and Its Computing Resource Allocation Method", provides a technical solution for medical imaging equipment that involves the allocation of computing resources.

[0007] As those skilled in the art know, the above solutions are all centralized / cloud-based computing power solutions that rely on cloud servers or fixed processing equipment. Further optimizing the application of endoscope hosts in the medical field will undoubtedly promote the further development of the medical field. Summary of the Invention

[0008] To address the aforementioned technical challenges in optimizing the application of endoscope hosts in the medical field, this invention provides a distributed cluster endoscope network aggregation method and endoscope system. This solution enables dynamic computing power aggregation and task scheduling among endoscope hosts with varying performance levels in an environment without a central server.

[0009] To address the above problems, this invention provides a distributed cluster endoscope network aggregation method and endoscope system that solves the problems through the following technical points: A distributed cluster endoscope network aggregation method, which is used to realize computing power sharing among multiple endoscope hosts through a network, includes the following steps: S1. Self-organizing network and scheduler election: When there is no scheduler in the network, the first endoscope host to join the network automatically becomes the scheduler; when there is already a scheduler in the network, the newly joined endoscope host calculates its own recommendation score, obtains the recommendation score of the current scheduler, and elects the endoscope host with the highest recommendation score as the scheduler; the recommendation score is positively correlated with the CPU idle rate and network bandwidth idle rate of the endoscope host. S2. Available resource value assessment: The scheduler periodically obtains the available resource value of the image processing computing power of each endoscope host in the network. The available resource value is obtained based on the number of available cores of the graphics processor, available video memory, and available network bandwidth of the endoscope host. The available resource value is then corrected based on the real-time temperature of each endoscope host and the ratio of the current image processing time to the maximum allowable processing time determined based on the image frame rate. S3. Resource allocation: When the image processing computing power required for the local image processing task of any endoscope host exceeds its own available resource value, the endoscope host, as the applicant, sends a resource request to the scheduler. The scheduler selects the endoscope host from the network whose corrected available resource value is closest to the estimated value of the required image processing computing power as the sharer, and establishes an image transmission data channel between the applicant and the sharer. S4. Data transmission: The applicant sends the image to be processed to the sharer through the data channel. The sharer receives and processes the image to be processed and returns the processed image data to the applicant through the data channel.

[0010] In one specific embodiment, the recommendation score is calculated using the following formula: S = 1 × 1 𝑐𝑝𝑢 +2×1 𝑏𝑎𝑛d𝑤𝑖d𝑡ℎ ; Where S is the nomination score, and 𝑈 𝑐𝑝𝑢 and 𝑈 𝑏𝑎𝑛d𝑤𝑖d𝑡ℎ These are the normalized idle rate of the central processing unit and the normalized available network bandwidth, respectively. 𝑘1 and 𝑘2 are the weight parameters, 𝑘... 1+ 𝑘2=1; The available resource value is calculated using the following formula: R = (1 × 1) 𝑔𝑝𝑢 +2×1 gpuddr +𝜔3×U band𝑤𝑖d𝑡ℎ )×(1−P); Where R is the available resource value, 𝑈 𝑔𝑝𝑢 This represents the number of cores currently available for parallel graphics processing in the graphics processor, specifically the difference between the total number of cores in the graphics processor and the number of cores currently in use, denoted as 𝑈. gpuddr U represents the current remaining available memory value of the graphics processor. band𝑤𝑖d𝑡ℎ represents the currently available bandwidth of the endoscope host, where 𝜔1, 𝜔2, and 𝜔3 are weighting factors, and P is the penalty factor; The penalty factor P is calculated using the following formula: P = 1 × P tmp +2×P 𝑡𝑖𝑚𝑒 ; Among them, P tmp P is the temperature penalty factor for the endoscope host. 𝑡𝑖𝑚𝑒 To handle the time penalty factor, 𝛼1 and 𝛼2 are the weighting coefficients, respectively; The estimated image processing computing power required is calculated using the following formula: R image =𝜆1×S×F×(𝐴1+𝐴2+⋯+𝐴 𝑛 )+k2×k 𝑔𝑝𝑢ddr+3×U band𝑤𝑖d𝑡ℎ ; Where S is the size of the image to be processed, F is the frame rate of the image to be processed, and n is the number of algorithms for the image to be processed, from n1 to n. 𝑛 Here are the estimated number of graphics processor cores required for each of the n images to be processed, and n is the number of images to be processed. 𝑔𝑝𝑢ddr To estimate the required graphics processor memory, U band𝑤𝑖d𝑡ℎ To estimate the required network bandwidth, R1, R2, and R3 are weighting coefficients.

[0011] In one specific embodiment, P tmp P 𝑡𝑖𝑚𝑒 Calculate according to the following formulas respectively: ,in, t The optimal operating temperature for the endoscope host is calculated by weighted averaging the temperatures of the CPU, GPU, FPGA, and motherboard, taking into account the overall performance of the core components. T This represents the current image processing time of the endoscope host. T farme This is the maximum allowed processing time determined based on the image frame rate.

[0012] In one specific embodiment, the method by which the scheduler selects a sharer from the network is as follows: Calculate the absolute value of the difference between the corrected available resource value of each endoscope host in the network and the estimated value of the required image processing computing power, and select the endoscope host with the smallest absolute value of the difference as the sharer; If the number of endoscope hosts with the smallest absolute difference is greater than 1, the endoscope host with the largest available resource value before the available resource value correction is selected as the sharer.

[0013] In one specific embodiment, the resource allocation and data transmission are implemented as follows: After the applicant sends a resource request to the scheduler, the scheduler identifies a unique sharer from the network. The applicant maintains a data connection with the sharer throughout the entire image processing task until the task is completed. After the image processing task is completed, the applicant actively disconnects from the sharer and notifies the scheduler to release the sharer's available resources. In the applicant's image processing task based on the sharer, the applicant transmits the first frame of image data to the sharer along with complete image processing parameters. When the image processing parameters do not change, only image data is transmitted subsequently, and the sharer performs image processing according to the current image processing parameters. When the image processing parameters change, the applicant transmits the updated image processing parameters to the sharer, and the sharer uses the most recently received image processing parameters for image processing. Image transfer between the applicant and the sharer is performed using RDMA or GPU Direct RDMA technology.

[0014] In one specific embodiment, in step S2, after the scheduler periodically obtains the available resource value of the image processing computing power of each endoscope host in the network, it generates a resource information table. The information of each endoscope host recorded in the resource information table includes: The identification and network address of the endoscope host; Available resource values ​​before and after the correction; Network access time and network disconnection time; The current operating mode, which characterizes the role of the endoscope host in the current system, includes: The scheduler pattern as a scheduler, the applicant pattern as an applicant, the sharer pattern as a sharer, and the idler pattern as neither an applicant nor a sharer. Endoscope host association information, which is used to record the correspondence between the current applicant and the sharer; A shared password is used for secure authentication when establishing a data connection between the applicant and the sharer. The offline time is configured as follows: when the endoscope host is normally shut down, the endoscope host actively sends an offline notification to the scheduler before shutting down. After receiving the notification, the scheduler sets the offline time of the endoscope host to the current time. For endoscope hosts that do not receive an offline notification, after the scheduler sends a set number of query requests to the endoscope host and does not receive a response to the query requests, the offline time of the endoscope host is set to the moment when no query request response is received for the first time. In step S3, the scheduler responds to the applicant's resource request according to the resource information table and excludes hosts with offline time greater than 0 when selecting a sharer.

[0015] In one specific embodiment, when the scheduler detects an endoscope host with an offline time greater than 0, the scheduler checks the association information of the offline endoscope host to determine whether the offline endoscope host is currently participating in the applicant's image processing task as a sharer. If it is determined that the offline endoscope host is currently acting as the sharer, the scheduler notifies the applicants associated with the offline endoscope host and re-triggers the resource allocation step to reselect a sharer for the associated applicants; If it is determined that the offline endoscope host is not currently acting as a sharer, the scheduler marks the available resource value of the endoscope host as unavailable and stops sending periodic available resource value query requests to it.

[0016] In one specific embodiment, when an applicant sends a resource request to the scheduler, and the scheduler is unable to find a sharer for the applicant on the network according to the resource information table, the scheduler returns a response to the applicant that no sharer is available; after receiving the response, the applicant uses its own image processing computing power to complete the image processing task. During the process of the endoscope host acting as a sharer to execute the applicant's image processing task, if the sharer starts a new image processing task locally, and it is determined that the sharer's current available resources are insufficient to simultaneously satisfy the local task and the currently executing shared task, the sharer sends a notification to the scheduler to cancel image processing computing power sharing and switches to idle mode to prioritize the processing of the new local task; after receiving the notification, the scheduler reselects a sharer for the associated applicant according to the resource information table.

[0017] This solution also relates to a distributed cluster endoscope system, including multiple endoscope hosts, each endoscope host comprising: An image acquisition unit is used to connect to the endoscope camera and acquire raw image data; Local display unit, used to connect to a monitor and output processed images; The transmission module is used for data transmission with other endoscope hosts; The task proxy module is used to execute the method according to any one of claims 1 to 8; A high-speed network switch is used to enable network connections between multiple endoscope hosts.

[0018] In one specific embodiment, among the multiple endoscope hosts, some endoscope hosts are not configured with a graphics processor as an image processing unit. Endoscope hosts without a graphics processor access the system in requester mode and complete image processing tasks by calling the graphics processors of other endoscope hosts. Some endoscope hosts are configured with a graphics processor as an image processing unit. Endoscope hosts with a graphics processor are configured to act as schedulers, requesters, and sharers in the system.

[0019] The present invention has the following beneficial effects: For image processing tasks where the available resources are insufficient to meet the required computing power, this solution achieves the following through dynamic computing power aggregation and task scheduling among endoscope hosts with different performance levels: It can reduce the procurement cost of hospital endoscopes: there is no need to equip every operating room or examination room with a high-performance endoscope host, thereby reducing the overall procurement cost of hospital endoscope hosts; Improve the utilization rate of high-performance hospital resources (sharers in endoscopic systems); Facilitates the enhancement of the hospital's overall endoscopy capabilities: When a hospital needs to upgrade the functionality of its endoscopy host, it only needs to add at least one corresponding high-performance host to the network. By calling these high-performance hosts, the hospital's overall endoscopy processing capabilities can be enhanced. To solve the problem that low-performance endoscope main units cannot handle complex image processing tasks in operating rooms and examination rooms; The system operates in a serverless environment, reducing manual configuration and application costs through autonomous resource scheduling. The mode adopted by the scheduler to identify the unique sharer in the network is a long connection binding mode, based on RDMA transmission technology, and uses an available resource value correction mode, which can effectively ensure the low latency and reliable response requirements of real-time video streaming applications for endoscopy. Attached Figure Description

[0020] Figure 1 This is a distributed cluster network topology diagram used in a specific embodiment of the endoscope system described in this solution. Figure 2 This is a schematic diagram of the functional modules of the endoscope host in a specific embodiment of the endoscope system described in this solution; Figure 3 This is a schematic diagram illustrating the switching of working modes between endoscope main units in a specific embodiment of the endoscope system described in this solution; Figure 4 This is a flowchart illustrating the power-on and network connection process of the endoscope host in a specific embodiment of the endoscope system described in this solution. Figure 5 The scheduler's workflow is shown in a specific embodiment of the endoscope system described in this solution. Figure 6 The applicant's workflow diagram is shown in a specific embodiment of the endoscope system described in this solution; Figure 7 This is a flowchart of the workflow of the sharer in a specific embodiment of the endoscope system described in this solution; Figure 8 This is a time sequence diagram of resource request data transmission between the applicant, sharer, and scheduler in a specific embodiment of the endoscope system described in this solution. Figure 9 A schematic diagram of the data flow of the endoscope host without resource application in a specific embodiment of the endoscope system described in this solution; Figure 10 A schematic diagram of the data flow of the endoscope host that requires resource application in a specific embodiment of the endoscope system described in this solution; Figure 11 This is a flowchart of a specific embodiment of the distributed cluster endoscope network aggregation method described in this solution. Detailed Implementation

[0021] The following description, in conjunction with the following embodiments and appendices, Figure 1 To the attached Figure 11 The present invention will be further described in detail below, but the present invention is not limited to the following embodiments: Example 1: like Figure 11 A distributed cluster endoscope network aggregation method, which is used to realize computing power sharing among multiple endoscope hosts through a network, includes the following steps: S1. Self-organizing network and scheduler election: When there is no scheduler in the network, the first endoscope host to join the network automatically becomes the scheduler; when there is already a scheduler in the network, the newly joined endoscope host calculates its own recommendation score, obtains the recommendation score of the current scheduler, and elects the endoscope host with the highest recommendation score as the scheduler; the recommendation score is positively correlated with the CPU idle rate and network bandwidth idle rate of the endoscope host. S2. Available resource value assessment: The scheduler periodically obtains the available resource value of the image processing computing power of each endoscope host in the network. The available resource value is obtained based on the number of available cores of the graphics processor, available video memory, and available network bandwidth of the endoscope host. The available resource value is then corrected based on the real-time temperature of each endoscope host and the ratio of the current image processing time to the maximum allowable processing time determined based on the image frame rate. S3. Resource allocation: When the image processing computing power required for the local image processing task of any endoscope host exceeds its own available resource value, the endoscope host, as the applicant, sends a resource request to the scheduler. The scheduler selects the endoscope host from the network whose corrected available resource value is closest to the estimated value of the required image processing computing power as the sharer, and establishes an image transmission data channel between the applicant and the sharer. S4. Data transmission: The applicant sends the image to be processed to the sharer through the data channel. The sharer receives and processes the image to be processed and returns the processed image data to the applicant through the data channel.

[0022] As those skilled in the art, the core concept of the above technical means includes: executing the applicant's image processing tasks using the available resources of the sharer. Therefore, there is a difference in image processing task capabilities between the applicant and the sharer. From a technical necessity perspective, the multiple endoscope hosts interconnected by the network in this solution have performance differences, that is: the endoscope hosts used to form the endoscope system have heterogeneous performance (including the case where there are two or more endoscope hosts with the same performance in the system). This is consistent with the general practice of hospitals purchasing endoscope hosts in batches and purchasing endoscope hosts from different manufacturers and models. It should also be noted that: regarding the aforementioned performance heterogeneity, in order to achieve resource allocation for the purpose of network aggregation, it refers to the system having at least two endoscope hosts with heterogeneous performance, such as low-performance endoscope hosts, medium-performance endoscope hosts, and high-performance endoscope hosts. The above low, medium, and high are only the results of internal system comparison of the image processing performance of different endoscope hosts in the system, and do not involve a performance level classification quantified by specific performance parameters.

[0023] The following provides a specific application of the method, which relies on a distributed cluster endoscope system, such as... Figure 1 The system comprises M high-performance endoscope hosts, N medium-performance endoscope hosts, and K low-performance endoscope hosts. All endoscope hosts are interconnected via a high-speed network switch. Upon initial system startup, the first endoscope host to connect to the network (assuming it is a medium-performance host compared to subsequent hosts) detects that there is no scheduler in the network and automatically becomes the scheduler. This endoscope host initiates scheduler mode in the task agent module and begins listening for network entry requests. Subsequently, a high-performance endoscope host (compared to the performance of previously connected endoscope hosts) connects to the network and sends a network entry request broadcast packet. Upon receiving this request, the current scheduler (the medium-performance host) and the endoscope host calculate their respective recommendation scores using the formula: S = Σ1 × Σ. 𝑐𝑝𝑢 +2×1 𝑏𝑎𝑛d𝑤𝑖d𝑡ℎ Where, both 𝑘1 and 𝑘2 are 0.5, and the 𝑈 of the high-performance host (the endoscope host subsequently connected to the network) is 0.5. 𝑐𝑝𝑢 and 𝑈 𝑏𝑎𝑛d𝑤𝑖d𝑡ℎ The scores are 0.9 and 0.8 respectively, with a total recommendation score of 0.85. This represents the N-score for the medium-performance host (the endoscope host that was connected to the network first). 𝑐𝑝𝑢 and 𝑈 𝑏𝑎𝑛d𝑤𝑖d𝑡ℎThey are 0.6 and 0.7 respectively, and the calculated result of the recommendation score is 0.65. Since the recommendation score of the newly connected endoscope host is higher, the current scheduler is replaced by the newly connected endoscope host. In other embodiments, if there are other endoscope hosts in the system, the endoscope host that was previously the scheduler sends a broadcast. After notifying other endoscope hosts in the network through the broadcast, the scheduling right is transferred to the high-performance host, and itself switches to the idle mode, and the high-performance host becomes the new scheduler.

[0024] Regarding the evaluation of available resource values, in this embodiment, for a 3D reconstruction surgery in the operating room that depends on the endoscope, the endoscope host in this operating room is a low-performance host that cannot meet the 3D image processing requirements (even without a GPU configured). The computing power required for the local image processing task of this endoscope host exceeds its own available resource value, so it sends a resource application to the scheduler. After receiving the resource application, the scheduler queries the resource information table it maintains. The resource information table records the available resource values of all online endoscope hosts in the network. The scheduler calculates the absolute value of the difference between the corrected available resource value R of each candidate sharer (including the scheduler itself) and the task resource pre-estimation value R image and selects the endoscope host with the smallest absolute value of the difference as the sharer. In this embodiment, assume that there is a high-performance host currently idle in the network, and its corrected available resource value R = 0.85, while the task resource pre-estimation value R image = 0.80, and the absolute value of the difference is 0.05, which is less than other candidate endoscope hosts, so this high-performance host is selected as the sharer.

[0025] The scheduler respectively informs the applicant and the sharer of the IP address and sharing password of the sharer. A direct RDMA long connection data channel is established between the applicant and the sharer. The applicant sends the 4K endoscope images collected in real time to the sharer through this channel. After the sharer uses its high-performance GPU to complete the 3D reconstruction, it returns the processed images to the applicant, and the applicant displays them in real time on the local monitor.

[0026] During the endoscope surgery, the temperature of the sharer host that works under high load for a long time will gradually increase. According to the temperature penalty factor formula P = 𝛼1×P tmp + 𝛼2×P 𝑡𝑖𝑚𝑒 , where P tmp = 0.04×( t -60) (when 60 < t < 85). Assume that the working temperature of a certain sharer host rises from 60°C to 75°C, and its temperature penalty factor increases from 0 to 0.6. At the same time, assume that other parameters of this host remain unchanged, and as the penalty factor P increases, the corrected available resource value R decreases.

[0027] When a new resource request is made, the scheduler calculates the available resource value of each host and finds that the R value of the host is lower than that of other hosts with lower temperatures. Therefore, the task will be assigned to other hosts. This mechanism allows each host to work in turn, avoiding the problem of some hosts running at high load for a long time and causing the temperature to become too high, thus ensuring the stability of the equipment and the safety of the operation.

[0028] In this embodiment, when a high-performance host that is performing an image processing task as a sharer suddenly disconnects from the network due to a network failure, after the scheduler sends three consecutive query requests to the host without receiving a response, the scheduler sets the host's offline time to the moment when no response is received for the first time and marks the offline time as greater than 0.

[0029] The scheduler checks the association information of the offline host and finds that the host is currently participating in the image processing task of applicant A as a sharer. The scheduler immediately notifies applicant A that the sharer has gone offline and triggers the resource reallocation step. Then, a new sharer is selected for applicant A from the resource information table. After applicant A establishes a connection with the new sharer, the image processing task continues from the point of interruption, ensuring the continuity of the operation.

[0030] In this embodiment, when an endoscope host is currently acting as a sharer to execute the image processing task of applicant B, if the endoscope host starts a local image processing task (e.g., a doctor needs to use an endoscope in the operating room), and the endoscope host determines, based on the task type input by the doctor (e.g., 3D reconstruction), that the remaining available resources are insufficient to simultaneously satisfy the local task and the currently executing shared task, the endoscope host acting as the sharer proactively sends a notification to the scheduler to cancel image processing computing power sharing and switches to idle mode to prioritize processing the new local task. Upon receiving this notification, the scheduler selects a new sharer for applicant B according to the resource information table and notifies applicant B to establish a connection with the new sharer. After completing its local task, the original sharer reconnects to the system in idle mode, awaiting subsequent resource requests.

[0031] In this embodiment, when the overall system load is high (endoscope host usage is relatively saturated), after a requester with a low-performance endoscope host sends a resource request to the scheduler, and the scheduler queries the resource information table and finds that the corrected available resource values ​​of all on-network endoscope hosts are less than the estimated resource value for the task, the scheduler returns a response to the requester stating that there are no available sharers. Upon receiving this response, the requester no longer requests resources from the scheduler and instead independently completes the image processing task using its limited image processing computing power. Although the processing effect may not be as ideal as calling a high-performance sharer, this method ensures that the basic needs of the surgery can be met, preventing the surgery from being completely impossible to perform.

[0032] In this embodiment, after the applicant and the sharer establish a long connection, the applicant transmits the first frame of image data to the sharer, carrying complete image processing parameters (such as contrast, brightness, color enhancement coefficient, etc.). When the image processing parameters of subsequent image frames do not change, the applicant only transmits the image data. After retransmitting the image processing parameters, for example, when the doctor adjusts the image processing parameters (e.g., adjusts the contrast from 1.2 to 1.5), the applicant encodes the updated parameters and writes them into the edge pixel area outside the effective display area of ​​the image frame (endoscopes ultimately display mostly circular images, and the four corner areas of rectangular images are non-observation areas; writing parameters there will not affect the doctor's observation). The sharer extracts the parameters from the edge pixel area of ​​the received image frame and performs image processing with a contrast of 1.5 to save transmission bandwidth and improve processing efficiency.

[0033] In this embodiment, regarding data transmission, to further reduce the bandwidth requirements, the applicant compresses the image data using the JPEG-LS lossless compression algorithm before sending it. Upon receiving the compressed data, the sharer decompresses it before performing image processing. In the currently prevalent gigabit network environment, transmitting a 4K raw image (approximately 12MB / frame) takes about 0.1 milliseconds. After JPEG-LS compression, the data volume can typically be reduced to 1 / 2 to 1 / 3 of the original. Combined with RDMA transmission technology, the additional computational overhead of compression and decompression is less than the saved transmission time, thus reducing end-to-end image transmission latency overall. The JPEG-LS lossless compression algorithm ensures no image information loss while effectively reducing data volume, making it particularly suitable for operating room environments with limited network bandwidth. Lossless compression ensures that image quality is not affected in any way, meeting the stringent fidelity requirements of medical image processing.

[0034] Example 2: like Figure 1 and Figure 4 This embodiment is a further refinement based on Embodiment 1: After the scheduler was re-determined based on the recommendation scores of 0.85 and 0.65, during subsequent use, an endoscope host connected to the network. The endoscope host sent a network access request broadcast packet to the network. After receiving the request, the current scheduler calculated that the recommendation score of the newly joined endoscope host was 0.45, which was lower than the current scheduler's recommendation score of 0.85. Therefore, the scheduler notified the low-performance host to remain in the idle role, and the low-performance host joined the network as a member.

[0035] Through the nomination process provided in this embodiment, the system can achieve self-organizing network without a central server. All endoscope hosts automatically determine the scheduler according to the nomination score, without the need for manual configuration.

[0036] Example 3: like Figure 3 , Figure 6 as well as Figure 7 This embodiment is a further refinement based on Embodiment 1: This embodiment is used to describe in detail the dynamic switching process of the endoscope host between scheduler mode, requester mode, sharer mode and idler mode.

[0037] like Figure 3 As shown, the working modes of the endoscope host in the system (scheduler mode, requester mode, sharer mode, and idler mode) can be switched between each other.

[0038] Scenario 1: From available job seeker to applicant, and then back to available job seeker Assuming that based on the calculated available resource values ​​and the current image processing task requirements of the system, a low-performance endoscope host is currently in idle mode, when the endoscope host locally starts a 4K fluorescence imaging 3D image processing task, the endoscope host first assesses whether the available local resources meet the current task requirements. If the assessment result indicates that the local GPU computing power is insufficient to complete the task independently, the endoscope host will proceed according to... Figure 6 The applicant's workflow, as shown, involves sending a resource request to the scheduler. After the scheduler configures a high-performance endoscope host as a sharer, the endoscope host switches to applicant mode. Upon completion of the task, the endoscope host proactively disconnects from the sharer and notifies the scheduler that the resource request is complete, then switches back to idler mode.

[0039] Scenario 2: From idle person to sharer, and then back to idle person Assuming that based on the calculated available resource values ​​and the current image processing task requirements of the system, a high-performance endoscope host is currently in idle mode, after the scheduler receives a resource request, it selects this endoscope host as the sharer. Upon receiving notification from the scheduler, the endoscope host switches to sharer mode, establishes a data connection with the requester, and executes the requester's image processing task. After the task is completed, the requester actively disconnects and notifies the scheduler, at which point the host's sharer mode ends, and it switches back to idle mode.

[0040] Scenario 3: From sharer to free user, and then to applicant Suppose an endoscope host is currently acting as a sharer to execute an applicant's image processing task. During this process, the endoscope host locally starts the image processing task. The endoscope host first determines whether the remaining available resources meet the local work task requirements based on the task type input by the human-computer interaction. If not, the endoscope host sends a notification to the scheduler to cancel resource sharing and switches to idle mode. Then, the endoscope host again evaluates whether the available resources meet the local work task requirements. If not, it sends a resource request to the scheduler and enters applicant mode, whereby the scheduler assigns a new sharer for it.

[0041] Example 4: like Figure 2 and Figure 5 As shown, this embodiment is a further refinement based on embodiment 1: Each endoscope host includes a task agent module, which is responsible for functions such as network access, scheduler election, estimation of required image processing computing power, resource application, and resource scheduling. In step S2, the scheduler follows... Figure 5 The workflow shown involves periodically querying the available resource values ​​of each endoscope host under its management and generating a periodic resource information table based on the periodic query results. The resource information table includes the host's identity (such as host serial number), IP address and available resource value, as well as network access time, network disconnection time, current working mode, associated host information (the applicant and the sharer are associated with each other) and shared password information.

[0042] Normal shutdown and disconnection from the network: When an endoscope host is shut down normally, it actively sends a disconnection notification to the scheduler before shutdown. After receiving the notification, the scheduler sets the disconnection time of the endoscope host to the current time and immediately marks the available resource value of the host as unavailable. In subsequent resource allocation, the scheduler excludes hosts with a disconnection time greater than 0 when selecting sharers.

[0043] Abnormal Network Disconnection: When an endoscope host disconnects from the network due to abnormal reasons such as network failure or system crash, after the scheduler sends multiple (e.g., 3 times in the system) query requests to one or more endoscope hosts without receiving a response, the scheduler sets the disconnection time of these endoscope hosts to the moment the first no response is received. The scheduler then checks the association information of these endoscope hosts: if the host is currently performing an image processing task as a sharer, the scheduler immediately notifies the associated applicants and triggers a resource reallocation step; if these endoscope hosts are not currently acting as sharers, the scheduler marks the available resource values ​​of these endoscope hosts as unavailable and stops sending periodic available resource value query requests to them.

[0044] Example 5: like Figure 6 and Figure 7 As shown, this embodiment is a further refinement based on embodiment 1: This embodiment aims to provide an image processing mechanism after an applicant's resource request fails: When an endoscope host sends a resource request to the scheduler as the applicant, if the scheduler cannot match a suitable sharer for it (e.g., high system utilization, resulting in high network load, or all available resources of endoscope hosts on the network after correction are less than the estimated image processing computing power required by the applicant), the scheduler returns a response to the applicant that there is no available sharer. After receiving this response, the applicant will no longer request resources from the scheduler and will instead use its own image processing computing power to independently complete the image processing task, so as to avoid the surgery being delayed due to the inability to obtain external computing power.

[0045] This embodiment aims to provide an image processing mechanism for the sharing of computing resources after the sharing user's local task begins, which is then occupied by local image processing. During the sharing user's execution of the applicant's image processing task, after obtaining the type of local image processing task to begin through a human-computer interaction interface (e.g., a doctor selects the type of local image analysis to be performed on the sharing user via an endoscope host input device and a human-computer interaction dialog box), the sharing user first determines whether its remaining available resources are sufficient for the local task. If insufficient, the sharing user sends a notification to the scheduler to release resource sharing and switches to idle mode to prioritize local tasks. Upon receiving the notification, the scheduler immediately selects a new sharing user for applicant A from the resource information table and notifies the applicant to establish a connection with the new sharing user. After the scheduler receives confirmation from the applicant that they are allowed to change sharing users, the new sharing user provides image processing to the applicant via network connection and provides feedback on the processed image data to prompt the doctor to respond to (e.g., stop the operation and wait for smooth real-time image data display after the switch is complete) the brief display delay or image stuttering introduced during the sharing user switching process.

[0046] Example 6: like Figure 8 , Figure 9 as well as Figure 10 This embodiment is a further refinement based on Embodiment 1: Regarding data transmission, such as Figure 8 As shown, after the applicant requests resources from the scheduler, the scheduler assigns a sharer to the applicant. The applicant maintains a connection with the same sharer throughout the entire image processing task until the task is completed. This data transmission and processing method aims to achieve: The number of scheduling requests by the scheduler is reduced from requests per frame of image to requests per task, which can effectively reduce the CPU performance and bandwidth requirements of the scheduler. It eliminates the extra time overhead of scheduling each frame of image in short connection mode (which uses each frame or multiple frames of images collected in each task as the image dataset and uses different sharers to process different datasets), thus achieving the goal of ensuring image display frame rate and reducing display latency. Image processing parameters do not need to be carried in every dataset. When the image processing parameters do not change, there is no need to repeatedly configure and transmit them, thereby saving the bandwidth required for data transmission and reducing transmission time.

[0047] In this embodiment, after the applicant and the sharer establish a long connection, the applicant transmits complete image processing parameters (including contrast, brightness, color enhancement coefficient, sharpening intensity, etc.) to the sharer when transmitting the first frame of image data. If the image processing parameters do not change in subsequent image frames, the applicant only transmits the image processing parameters once. When the doctor adjusts the image processing parameters according to surgical needs (e.g., adjusting the contrast from 1.2 to 1.5, enabling fluorescence imaging mode, etc.), the applicant sends the updated parameters to the sharer through the data channel. Upon receiving the new image processing parameters, the sharer performs image processing using the latest parameters.

[0048] Regarding the embedding method of image processing parameters, the applicant adopts the following approach: After encoding the updated image processing parameters, they are written into the edge pixel region outside the effective display area of ​​the image frame. Since the final display of an endoscope is mostly a circular image, and the corner region of the rectangular image acquired by the applicant's image sensor (i.e., the region between the rectangular region and its inner circular observation area) is a non-observation area, writing parameters into the pixels of this non-observation area will not affect the final image processing result. The applicant extracts the parameters from the edge pixel region of the received image frame and then processes the image data within the effective display area, thereby achieving synchronous transmission of image processing parameters and the image without the need for additional data types. For image shapes and styles acquired by other image sensors (such as circles), since image edge features are usually non-observation features and image acquisition is continuous (video streams are usually transmitted at a rate of 30 to 60 frames per second; even if the effective features in the current image frame are covered by encoding, the effective features can still be reflected in subsequent image frames since no encoding is required), the method of writing the encoding representing the image processing parameters at the edge of the image frame can also be used to achieve synchronous transmission of image processing parameters and the image.

[0049] exist Figure 9The diagram illustrates the data flow of the endoscope host that does not require resource allocation: as indicated by the red arrow, after the image acquisition unit completes image acquisition, the image processing unit of the endoscope host directly processes the image and then sends it to the image display unit for display. This data flow is downward, and no task agent module or transmission module needs to perform related tasks.

[0050] exist Figure 10 The diagram illustrates the image data flow for resource allocation and allocation as follows: The data flow within each sharer and sharer is shown by the red arrows. After acquiring the image from the image sensor via the image acquisition unit, the sharer sends it to the sharer through the task proxy module and transmission module. The sharer, via network connection, receives the image to be processed through the transmission module and then sends it to the image processing unit via the task proxy module. After processing, the image processing unit transmits the processed image data to its transmission module, which then transmits it to the sharer's transmission module via network connection. Finally, the sharer's task proxy module transmits the data to the image display unit for display. Throughout this process, the sharer sends the raw image data directly to the sharer via network connection, and the sharer, after processing the image, sends the processed image data directly to the sharer via network connection. The scheduler is only responsible for resource matching on the control plane and does not participate in image transmission. This direct data transmission between the sharer and sharer not only shortens the data transmission path and reduces end-to-end transmission latency, but also avoids the scheduler's network bandwidth and CPU resources being occupied by the image data stream, thus supporting larger-scale clusters and higher-frequency scheduling requests.

[0051] In the above embodiments of the distributed cluster endoscopic network aggregation method, the selection criteria for the scheduler are intended to achieve the following: Using CPU idle rate and network bandwidth idle rate as core indicators for electing the scheduler, the scheduler needs to periodically send query requests to all endoscope hosts in the network to obtain available resource values, receive and process the load status of each endoscope host, periodically maintain the resource information table, and respond to concurrent resource request requests that may occur in the system. These control plane tasks themselves consume CPU computing resources and network bandwidth resources. By positively correlating the election score with these two resources, it can ensure that the node with the strongest control plane processing capability in the system becomes the scheduler, and avoid the scheduler itself becoming a system performance bottleneck. In actual hospital endoscope host applications, endoscope hosts with strong central processing unit performance and network bandwidth are usually also high-performance hosts with strong graphics processing unit computing power. This means that while such endoscope hosts are elected as schedulers, they can still participate in computing power sharing as sharers, providing image processing services to other applicants, thereby maximizing the overall resource utilization of the system. In the specific calculation scheme of the nomination score, by configuring weight parameters, the system administrator can adjust the weight of the CPU idle rate and network bandwidth idle rate in the nomination score calculation according to the actual network environment (such as wired network or wireless network) and the specific needs of the operating room, so that the scheduler election mechanism can adapt to different deployment scenarios.

[0052] In the above embodiments of the distributed cluster endoscopic network aggregation method, the introduction of a penalty factor to modify the available resource value aims to achieve the following: The introduction of the temperature penalty factor aims to enable each endoscope host to take turns acting as a sharer to undertake the tasks of the applicant, thereby avoiding the overheating caused by some endoscope hosts operating at high loads for a long time. This prevents the image processing performance from being affected by overheating in endoscopic surgery scenarios, and even avoids the equipment from shutting down for protection or being damaged, directly threatening the continuity of the surgery and the safety of the patient.

[0053] In the processing time penalty factor used in this scheme, min( T, T farme This represents the current image processing time T and the maximum allowable processing time determined based on the image frame rate. T farme The smaller value in the equation; more specifically, T is the actual processing time used by the endoscope host to process one frame of image. T farme The maximum allowable processing time calculated based on the current image frame rate (e.g., at 60 frames / second). T farme =1000 / 60≈16.67 milliseconds, min( T, T farme ) indicates taking T and T farme The smaller value in the above processing time penalty factor, whose value is the normalized value of the time taken for the current image processing, in the unprocessed image ( T farmeWhen the processing time penalty factor is 0 (including applications processing static images), no time penalty is applied to the available resource value, reflecting the targeted optimization of this solution for real-time video streaming scenarios. In non-real-time scenarios, the penalty strategy is automatically adjusted. When the processing time is close to or greater than the maximum processing time, the penalty factor is close to or equal to 1. This processing time penalty factor aims to solve the problem of resource mismatch. For example, an endoscope host may have limited video memory resources and a small number of available cores, but its available bandwidth may be very large. This results in a large available resource value without a processing time penalty factor, but a small number of available cores and video memory leads to a corresponding increase in processing time. As mentioned above, by increasing the calculated value of the processing time penalty factor, the final result can reduce its available resource value, avoiding the endoscope host being allocated as a sharer after overestimating its available resource value.

[0054] The above P tmp P 𝑡𝑖𝑚𝑒 The calculation method is also used to normalize the host temperature penalty factor and processing time penalty factor. Meanwhile, P... 𝑡𝑖𝑚𝑒 The squared function used is intended to increase the sensitivity of the penalty factor to processing time: for example, when the processing time reaches 50% of the maximum allowable processing time, P... 𝑡𝑖𝑚𝑒 The calculated result is 0.25. When the processing time reaches 80% of the maximum allowable processing time, P... 𝑡𝑖𝑚𝑒 The calculated result is 0.64. When the processing time reaches 100% of the maximum allowable processing time, P... 𝑡𝑖𝑚𝑒 The calculation result is 1. This non-linear function setting makes the penalty more significant as the processing time gets closer to the maximum allowed time, ensuring that the scheduler prioritizes the host with higher processing efficiency as the sharer.

[0055] In the revised formula for calculating available resource value, the available resource value is maximized when the penalty factor is 0; when the penalty factor is 1, the available resource value is 0, and such endoscope hosts cannot become sharers. This available resource value design ensures that endoscope hosts with excessively high temperatures or excessively long processing times can be removed from the sharer candidate pool, guaranteeing the overall performance of the system and the safety of the surgery.

[0056] In this scheme, regarding offline time, the system defines any offline endoscope host that does not receive an offline notification or does not receive a response to the first query request as an abnormal offline. When an abnormal offline occurs, the scheduler makes a judgment by querying the response to multiple consecutive requests, avoiding the endoscope host being mistakenly judged as offline due to instantaneous network jitter. At the same time, the design of multiple consecutive (e.g., 3) query requests is used to achieve a balance between timeliness and accuracy: it will not be misjudged due to a single packet loss, nor will it delay task reassignment due to excessive waiting time.

[0057] In this scheme, if the applicant fails to comply with the requirements during the process of receiving image processing computing power support from the sharer... Figure 8 The provided resource request data transmission timing is such that if the applicant fails to receive the sharer's image data after the set time has elapsed before receiving a cancellation response, the applicant's task agent module outputs an image reception stop display signal to the image display unit to prompt the applicant's doctor to pause the surgery to ensure surgical safety. After the network transmission is restored or the sharer is changed and the process is completed automatically, the task agent module outputs an image reception normal display signal to the image display unit based on the real-time data acquisition and recognition results from the applicant's task agent module to prompt the applicant's doctor to continue the surgery.

[0058] In this solution, when the sharer starts an image processing task locally and has insufficient remaining available resources, the sharer proactively sends a notification to the scheduler to cancel resource sharing and switches to idle mode to prioritize local tasks. This aims to ensure that doctors can obtain immediate image processing responses when operating directly on the endoscope host through the principle of prioritizing local tasks. This configuration enables the system to achieve optimal performance in different usage scenarios and avoids local operation lag caused by shared tasks consuming computing power. This ensures the effectiveness of doctors' use of the endoscope when the system's computing power allows.

[0059] After receiving a notification from the sharer to cancel resource sharing, the scheduler immediately selects a new sharer for the associated applicant and completes the switch after receiving confirmation from the applicant that they are allowed to change the sharer. This is intended to enable the applicant (i.e., the operating room that is relying on external computing power for image processing) to switch sharers at appropriate times (such as during breaks in surgical procedures), avoiding screen lag or delays during critical operations (such as tissue cutting, hemostasis, etc.) and enabling more refined consideration of surgical safety.

[0060] Example 7: This embodiment, based on embodiment 1, provides a specific embodiment for calculating the available resource value and the estimated value of the required image processing computing power.

[0061] In this embodiment, the values ​​of available resource values ​​𝜔1, 𝜔2, and 𝜔3 are 0.5, 0.3, and 0.2 respectively (following the decreasing weight of available GPU cores, available GPU memory, and endoscope host network bandwidth in resource evaluation). Taking an NVIDIA RTX 4090 GPU as an example, its total CUDA cores are 16384. With 2048 cores currently in use, 𝑈 𝑔𝑝𝑢 The value is 14336, and after normalization, it becomes 1.43 (divided by the base value of 10000), which corresponds to 14336 in available video memory. gpuddrThe total video memory is 16384MB (total video memory is 24GB, 8GB is used), and the normalized value is 1.64 (divided by the baseline value of 10000). band𝑤𝑖d𝑡ℎ The normalized value is 0.8 (divided by the baseline value of 1000) for a speed of 800MB / s. Simultaneously, the penalty factor is calculated to be 0.126 (because overheating of the host may cause the sharer to crash, posing a direct threat to surgical safety; α1 is set to 0.8 and α2 to 0.2; the current temperature of the endoscope host is 62.5℃; for an image frame rate of 60 frames / second, the maximum allowed processing time is 1000 / 60≈16.67 milliseconds, while the actual processing time for one frame by the endoscope host is 8 milliseconds, hence the penalty factor is actually 0.126). Therefore, the available resource value of the endoscope host is calculated to be 1.19, which serves as the basis for resource matching by the scheduler. It is easy to understand that the normalization process in this embodiment aims to address the performance differences between hardware products of different generations, models, and brands, making the resource evaluation model universal across hardware platforms and models.

[0062] For the estimated image processing computing power required, all parameters adopt the same normalization benchmark principle as those used in the calculation of available resource values. Specifically, R1, R2, and R3 are set to 0.6, 0.3, and 0.1, respectively. For a 4K resolution image to be processed, the image size is set to 0.83 based on the pixel benchmark (10 megapixels corresponds to an image size normalization benchmark of 1.0, and 4K resolution corresponds to 8.29 × 10⁻⁶ pixels). 6 Pixels, 8.29×10 6 / 10 7 (≈0.83), with a frame rate of 60 frames per second, and a normalized value of 1.0. This requires executing three image algorithms: 3D reconstruction, color enhancement, and noise reduction. The estimated number of GPU cores for each algorithm, after normalization, is 1.0 for R1. ， 𝐴2 is 0.5 ， R3 is 0.5, the estimated required graphics processing unit (GPU) memory is normalized to 0.49 (estimated GPU memory required is 8000MB, 8000 / 16384≈0.49), the estimated required network bandwidth is normalized to 0.5, and the calculated estimated image processing power is 1.193. After completing the above calculations, the scheduler compares this value with the absolute value of the available resources of each endoscope host in the network, and selects the host with the smallest absolute value of the difference as the sharer. For example, if the available resource value R of a high-performance host in the network is 1.19, the absolute value of the difference is 0.003; if the available resource value R of a medium-performance host is 0.85, the absolute value of the difference is 0.343. Since the difference of the high-performance host is smaller, it is selected as the sharer.

[0063] Example 8: This embodiment, based on Embodiment 1, provides a distributed cluster endoscope system. The system includes multiple endoscope hosts, each of which includes: An image acquisition unit is used to connect to the endoscope camera and acquire raw image data; Local display unit, used to connect to a monitor and output processed images; The transmission module is used for data transmission with other endoscope hosts; The task proxy module is used to execute the method described in Example 1; A high-speed network switch is used to enable network connections between multiple endoscope hosts.

[0064] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, other embodiments derived without departing from the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for aggregation of distributed clustered endoscopic networks, characterized in that, This method is used to achieve computing power sharing among multiple endoscope hosts via a network, and includes the following steps: S1. Self-organizing network and scheduler election: When there is no scheduler in the network, the first endoscope host to join the network automatically becomes the scheduler; when there is already a scheduler in the network, the newly joined endoscope host calculates its own recommendation score, obtains the recommendation score of the current scheduler, and elects the endoscope host with the highest recommendation score as the scheduler; the recommendation score is positively correlated with the CPU idle rate and network bandwidth idle rate of the endoscope host. S2. Available resource value assessment: The scheduler periodically obtains the available resource value of the image processing computing power of each endoscope host in the network. The available resource value is obtained based on the number of available cores of the graphics processor, available video memory, and available network bandwidth of the endoscope host. The available resource value is then corrected based on the real-time temperature of each endoscope host and the ratio of the current image processing time to the maximum allowable processing time determined based on the image frame rate. S3. Resource allocation: When the image processing computing power required for the local image processing task of any endoscope host exceeds its own available resource value, the endoscope host, as the applicant, sends a resource request to the scheduler. The scheduler selects the endoscope host from the network whose corrected available resource value is closest to the estimated value of the required image processing computing power as the sharer, and establishes an image transmission data channel between the applicant and the sharer. S4. Data transmission: The applicant sends the image to be processed to the sharer through the data channel. The sharer receives and processes the image to be processed and returns the processed image data to the applicant through the data channel.

2. The distributed cluster endoscope network aggregation method according to claim 1, characterized in that, The recommendation score is calculated using the following formula: S=𝑘1×𝑈 𝑐𝑝𝑢 +𝑘2×𝑈 𝑏𝑎𝑛d𝑤𝑖d𝑡ℎ ; Where S is the nomination score, and 𝑈 𝑐𝑝𝑢 and 𝑈 𝑏𝑎𝑛d𝑤𝑖d𝑡ℎ These are the normalized idle rate of the central processing unit and the normalized available network bandwidth, respectively. 𝑘1 and 𝑘2 are the weight parameters, 𝑘... 1+ 𝑘2=1; The available resource value is calculated using the following formula: R=(𝜔1×𝑈 𝑔𝑝𝑢 +𝜔2×𝑈 gpuddr +𝜔3×U band𝑤𝑖d𝑡ℎ )×(1−P); Where R is the available resource value, 𝑈 𝑔𝑝𝑢 This represents the number of cores currently available for parallel graphics processing in the graphics processing unit (GPU). gpuddr U represents the current remaining available memory value of the graphics processor. band𝑤𝑖d𝑡ℎ represents the currently available bandwidth of the endoscope host, where 𝜔1, 𝜔2, and 𝜔3 are weighting factors, and P is the penalty factor; The penalty factor P is calculated using the following formula: P=𝛼1×P tmp +𝛼2×P 𝑡𝑖𝑚𝑒 ; Among them, P tmp P is the temperature penalty factor for the endoscope host. 𝑡𝑖𝑚𝑒 To handle the time penalty factor, 𝛼1 and 𝛼2 are the weighting coefficients, respectively; The estimated image processing computing power required is calculated using the following formula: R image =𝜆1×S×F×(𝐴1+𝐴2+⋯+𝐴 𝑛 )+𝜆2×𝑈 𝑔𝑝𝑢ddr +𝜆3×U band𝑤𝑖d𝑡ℎ ; Where S is the size of the image to be processed, F is the frame rate of the image to be processed, and n is the number of algorithms for the image to be processed, from n1 to n. 𝑛 Here are the estimated number of graphics processor cores required for each of the n images to be processed, and n is the number of images to be processed. 𝑔𝑝𝑢ddr To estimate the required graphics processor memory, U band𝑤𝑖d𝑡ℎ To estimate the required network bandwidth, R1, R2, and R3 are weighting coefficients.

3. The distributed cluster endoscope network aggregation method according to claim 2, characterized in that, P tmp P 𝑡𝑖𝑚𝑒 Calculate according to the following formulas respectively: ； ； in, t The operating temperature of the endoscope unit. T This represents the current image processing time of the endoscope host. T farme This is the maximum allowed processing time determined based on the image frame rate.

4. The distributed cluster endoscope network aggregation method according to claim 1, characterized in that, The scheduler selects sharers from the network using the following method: Calculate the absolute value of the difference between the corrected available resource value of each endoscope host in the network and the estimated value of the required image processing computing power, and select the endoscope host with the smallest absolute value of the difference as the sharer; If the number of endoscope hosts with the smallest absolute difference is greater than 1, the endoscope host with the largest available resource value before the available resource value correction is selected as the sharer.

5. The distributed cluster endoscope network aggregation method according to claim 1, characterized in that, The specific implementation method for resource allocation and data transmission is as follows: After the applicant sends a resource request to the scheduler, the scheduler identifies a unique sharer from the network. The applicant maintains a data connection with the sharer throughout the entire image processing task until the task is completed. After the image processing task is completed, the applicant actively disconnects from the sharer and notifies the scheduler to release the sharer's available resources. In the applicant's image processing task based on the sharer, the applicant transmits the first frame of image data to the sharer along with complete image processing parameters. When the image processing parameters do not change, only image data is transmitted subsequently, and the sharer performs image processing according to the current image processing parameters. When the image processing parameters change, the applicant transmits the updated image processing parameters to the sharer, and the sharer uses the most recently received image processing parameters for image processing. Image transfer between the applicant and the sharer is performed using RDMA or GPU Direct RDMA technology.

6. The distributed cluster endoscope network aggregation method according to claim 1, characterized in that, In step S2, after the scheduler periodically obtains the available resource value of the image processing computing power of each endoscope host in the network, it generates a resource information table. The resource information table records the information of each endoscope host, including: The identification and network address of the endoscope host; Available resource values ​​before and after the correction; Network access time and network disconnection time; The current operating mode, which characterizes the role of the endoscope host in the current system, includes: The scheduler pattern as a scheduler, the applicant pattern as an applicant, the sharer pattern as a sharer, and the idler pattern as neither an applicant nor a sharer. Endoscope host association information, which is used to record the correspondence between the current applicant and the sharer; A shared password is used for secure authentication when establishing a data connection between the applicant and the sharer. The offline time is configured as follows: when the endoscope host is normally shut down, the endoscope host actively sends an offline notification to the scheduler before shutting down. After receiving the notification, the scheduler sets the offline time of the endoscope host to the current time. For endoscope hosts that do not receive an offline notification, after the scheduler sends a set number of query requests to the endoscope host and does not receive a response to the query requests, the offline time of the endoscope host is set to the moment when no query request response is received for the first time. In step S3, the scheduler responds to the applicant's resource request according to the resource information table and excludes hosts with offline time greater than 0 when selecting a sharer.

7. The distributed cluster endoscope network aggregation method according to claim 6, characterized in that, For endoscope hosts that the scheduler detects as having an offline time greater than 0, the scheduler checks the association information of the offline endoscope host to determine whether the offline endoscope host is currently participating in the applicant's image processing task as a sharer. If it is determined that the offline endoscope host is currently acting as the sharer, the scheduler notifies the applicants associated with the offline endoscope host and re-triggers the resource allocation step to reselect a sharer for the associated applicants; If it is determined that the offline endoscope host is not currently acting as a sharer, the scheduler marks the available resource value of the endoscope host as unavailable and stops sending periodic available resource value query requests to it.

8. The distributed cluster endoscope network aggregation method according to claim 6, characterized in that, When an applicant sends a resource request to the scheduler, and the scheduler is unable to find a sharer for the applicant on the network according to the resource information table, the scheduler returns a response to the applicant that no sharer is available; after receiving the response, the applicant uses its own image processing computing power to complete the image processing task. During the process of the endoscope host acting as a sharer to execute the applicant's image processing task, if the sharer starts a new image processing task locally, and it is determined that the sharer's current available resources are insufficient to simultaneously satisfy the local task and the shared task being executed, the sharer sends a notification to the scheduler to cancel the sharing of image processing computing power and switches to idle mode to prioritize the processing of the new local task. Upon receiving the notification, the scheduler will reselect a sharer for the associated applicant based on the resource information table.

9. A distributed cluster endoscope system, characterized in that, Includes multiple endoscope units, each endoscope unit comprising: An image acquisition unit is used to connect to the endoscope camera and acquire raw image data; Local display unit, used to connect to a monitor and output processed images; The transmission module is used for data transmission with other endoscope hosts; The task proxy module is used to execute the method according to any one of claims 1 to 8; A high-speed network switch is used to enable network connections between multiple endoscope hosts.

10. A distributed cluster endoscope system according to claim 9, characterized in that, Among multiple endoscope hosts, some endoscope hosts are not configured with a graphics processor as an image processing unit. Endoscope hosts without a graphics processor access the system in requester mode and complete image processing tasks by calling the graphics processors of other endoscope hosts. Some endoscope hosts are configured with a graphics processor as an image processing unit. Endoscope hosts with a graphics processor are configured to act as schedulers, requesters, and sharers in the system.

Images