Image Processing Method and Device for Binocular Surgical Navigator

Abstract

The present application provides an image processing method and device for a binocular surgical navigator, belonging to the technical field of image processing. The method includes: when a to-be-executed image processing task occurs, starting an image processing thread in a preset thread pool to execute the to-be-executed image processing task, and monitoring the real-time occupancy rate of the CPU; the preset thread pool includes a plurality of image processing threads and a plurality of non-image processing threads; based on the real-time occupancy rate of the CPU, the surgical stage being in, and the CPU occupancy of the threads that are working, adjusting the image processing thread corresponding to the to-be-executed image processing task, being able to execute the image processing task based on the preset multiple threads, and at the same time flexibly adjusting the number of image processing threads based on the real-time occupancy rate of the CPU, the surgical stage being in, and the CPU occupancy of the threads that are working, thereby maximizing the image processing efficiency within the range allowed by the CPU performance and ensuring the real-time nature of surgical instrument tracking and navigation guidance.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to an image processing method and device for a binocular surgical navigator. Background Art

[0002] Existing surgical navigators usually perform positioning based on binocular stereo vision. When in use, the position of the surgical instrument is determined through the images collected by the binocular cameras, and then real-time tracking and navigation guidance of the surgical instrument are realized. The real-time performance of the surgical instrument tracking and navigation guidance will directly affect the accuracy of the surgical instrument tracking and navigation guidance, and further affect the quality of the surgery. And the efficiency of image processing will directly affect the real-time performance of the surgical instrument tracking and navigation guidance.

[0003] With the improvement of the performance of binocular cameras, the resolution of the images they collect is getting higher and higher, which improves the accuracy of surgical instrument tracking. However, the improvement of image resolution also leads to a decrease in image processing efficiency, and further leads to a deterioration in the real-time performance of surgical instrument tracking and navigation guidance. Summary of the Invention

[0004] This application provides an image processing method and device for a binocular surgical navigator to maximize the image processing efficiency and thus ensure the real-time performance of surgical instrument tracking and navigation guidance.

[0005] This application provides an image processing method for a binocular surgical navigator, and the method includes:

[0006] When a to-be-executed image processing task appears, start an image processing thread in a preset thread pool to execute the to-be-executed image processing task, and monitor the real-time occupancy rate of the CPU; the preset thread pool includes multiple image processing threads and multiple non-image processing threads;

[0007] Based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the threads that are working, adjust the image processing thread corresponding to the to-be-executed image processing task.

[0008] According to an image processing method for a binocular surgical navigator provided by this application, the to-be-executed image processing task includes a first to-be-executed image processing subtask and a second to-be-executed image processing subtask. The first to-be-executed image processing subtask is used to process the image collected by the left camera of the binocular camera, and the second to-be-executed image processing subtask is used to process the image collected by the right camera of the binocular camera. Correspondingly, the step of starting an image processing thread in a preset thread pool to execute the to-be-executed image processing task specifically includes:

[0009] Start the first subset of image processing threads in the preset thread pool to execute the first image processing subtask to be executed, and start the second subset of image processing threads in the preset thread pool to execute the second image processing subtask to be executed;

[0010] The number of image processing threads in the first subset of image processing threads and the second subset of image processing threads is the same.

[0011] According to an image processing method of a binocular surgical navigator provided by the present application, the number of image processing threads in the preset thread pool is an even number, and the non-image processing threads include an image acquisition thread, a navigation guidance thread, and a data storage thread, and the number of each type of non-image processing thread is at least one.

[0012] According to an image processing method of a binocular surgical navigator provided by the present application, when the number of image processing threads in the first subset of image processing threads and the second subset of image processing threads is greater than one, the step of starting the first subset of image processing threads in the preset thread pool to execute the first image processing subtask to be executed and starting the second subset of image processing threads in the preset thread pool to execute the second image processing subtask to be executed specifically includes:

[0013] Use multiple image processing threads in the first subset of image processing threads to respectively execute the processing tasks of multiple sub-images obtained by equally dividing the images collected by the left camera in the first image processing subtask to be executed, and use multiple image processing threads in the second subset of image processing threads to respectively execute the processing tasks of multiple sub-images obtained by equally dividing the images collected by the right camera in the second image processing subtask to be executed;

[0014] Among them, the number of sub-images obtained by equally dividing the images collected by the left camera is the same as the number of image processing threads in the first subset of image processing threads.

[0015] According to an image processing method of a binocular surgical navigator provided by the present application, the step of adjusting the image processing threads corresponding to the image processing tasks to be executed based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the threads that are working specifically includes:

[0016] Based on the current surgical stage, determine the corresponding task priority ranking, and based on the task priority ranking, determine the priority of the image processing tasks to be executed;

[0017] Based on the priority of the image processing tasks to be executed, the real-time occupancy rate of the CPU, and the CPU occupancy of the threads that are working, adjust the image processing threads corresponding to the image processing tasks to be executed.

[0018] An image processing method for a binocular surgical navigator provided by the present application. The surgical stage is divided into the stage before entering the lesion, the stage during entering the lesion, and the stage after entering the lesion. Correspondingly, for the stage before entering the lesion, the corresponding task priority order is: image acquisition task > image processing task = navigation guidance task > data storage task;

[0019] For the stage during entering the lesion, the corresponding task priority order is: image acquisition task = image processing task = navigation guidance task > data storage task;

[0020] For the stage after entering the lesion, the corresponding task priority order is: navigation guidance task > image acquisition task = image processing task > data storage task.

[0021] An image processing method for a binocular surgical navigator provided by the present application. Based on the priority of the to-be-executed image processing task, the real-time occupancy rate of the CPU, and the CPU occupancy of the working threads, adjust the image processing threads corresponding to the to-be-executed image processing task, specifically including:

[0022] When the real-time occupancy rate of the CPU does not exceed the first preset threshold, if the CPU occupancy of the working image processing threads is lower than the second preset threshold, reduce the number of image processing threads corresponding to the to-be-executed image processing task. If the CPU occupancy of the working image processing threads is higher than the third preset threshold, increase the number of image processing threads corresponding to the to-be-executed image processing task;

[0023] When the real-time occupancy rate of the CPU exceeds the first preset threshold, if the CPU occupancy of the working image processing threads is lower than the second preset threshold, reduce the number of image processing threads corresponding to the to-be-executed image processing task. If the CPU occupancy of the working image processing threads is higher than the third preset threshold, modify the non-image processing threads with priorities lower than the image processing threads into image processing threads to execute the to-be-executed image processing task;

[0024] Among them, the first to third preset thresholds are in ascending order: the second preset threshold, the third preset threshold, and the first preset threshold.

[0025] The present application also provides an image processing device for a binocular surgical navigator. The device includes:

[0026] A first processing module, used to start the image processing threads in the preset thread pool to execute the to-be-executed image processing task when a to-be-executed image processing task appears, and monitor the real-time occupancy rate of the CPU; the preset thread pool includes multiple image processing threads and multiple non-image processing threads;

[0027] A second processing module, configured to adjust an image processing thread corresponding to a to-be-executed image processing task based on the real-time occupancy rate of the CPU, the surgical stage, and the CPU occupancy of the working threads.

[0028] The present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method of the binocular surgical navigator as described in any one of the above are implemented.

[0029] The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the image processing method of the binocular surgical navigator as described in any one of the above are implemented.

[0030] The image processing method and device of the binocular surgical navigator provided by the present application, in the case of a to-be-executed image processing task, start an image processing thread in a preset thread pool to execute the to-be-executed image processing task, and monitor the real-time occupancy rate of the CPU; the preset thread pool includes a plurality of image processing threads and a plurality of non-image processing threads; based on the real-time occupancy rate of the CPU, the surgical stage, and the CPU occupancy of the working threads, adjust the image processing thread corresponding to the to-be-executed image processing task, can execute the image processing task based on a plurality of preset threads, and at the same time flexibly adjust the number of image processing threads based on the real-time occupancy rate of the CPU, the surgical stage, and the CPU occupancy of the working threads, and can maximize the image processing efficiency within the range allowed by the CPU performance, thereby ensuring the real-time performance of surgical instrument tracking and navigation guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0032] Figure 1 is a schematic flowchart of the image processing method of the binocular surgical navigator provided by the present application;

[0033] Figure 2 is a schematic flowchart of the adjustment of the image processing thread corresponding to the to-be-executed image processing task provided by the present application;

[0034] Figure 3 is a schematic structural diagram of the image processing device of the binocular surgical navigator provided by the present application;

[0035] Figure 4 is a schematic structural diagram of the electronic device provided by the present application. Detailed implementation manners

[0036] To make the objectives, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below with reference to the accompanying drawings in this application. Apparently, the described embodiments are some but not all of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without making creative efforts shall fall within the protection scope of this application.

[0037] Figure 1 It is a schematic flowchart of an image processing method for a binocular surgical navigator provided by this application, as Figure 1 shown. This method includes:

[0038] Step 101, when a to-be-executed image processing task occurs, start an image processing thread in a preset thread pool to execute the to-be-executed image processing task, and monitor the real-time occupancy rate of the CPU; the preset thread pool includes a plurality of image processing threads and a plurality of non-image processing threads.

[0039] Specifically, the working process of the binocular surgical navigator is as follows: image acquisition (the binocular surgical navigator will acquire image data of the patient's body part to obtain more depth and three-dimensional information), image processing (including image preprocessing and feature extraction steps. The preprocessing steps may include image filtering, distortion correction, image registration, etc. After preprocessing, features such as key points, edges, and corner points can be extracted from the image), navigation guidance (including disparity calculation, three-dimensional reconstruction, and visualization navigation. Disparity calculation is to match the image features of the left and right cameras through a feature matching algorithm to calculate the disparity information; three-dimensional reconstruction is to convert the disparity into real three-dimensional coordinates according to the disparity value and camera parameters to obtain a three-dimensional model of the patient's body part; visualization navigation is to display the surgical target and tool positions on the navigation interface in real-time or static mode based on the three-dimensional model and the patient's anatomical structure to guide the doctor in performing surgical operations). At the same time, to facilitate subsequent review and analysis, the binocular surgical navigator will also save the image data, navigation guidance information, etc. during the operation to a storage device. Correspondingly, as the operation progresses, the control system of the binocular surgical navigator will continuously issue image acquisition tasks, image processing tasks, navigation guidance tasks, and data storage tasks, and assist in the operation process by executing the above tasks.

[0040] Based on the above content, it can be known that with the improvement of the performance of binocular cameras, the resolution of the images they collect is getting higher and higher, but the improvement of image resolution also leads to a decrease in image processing efficiency, which in turn leads to the deterioration of the real-time performance of surgical instrument tracking and navigation guidance. At the same time, existing binocular surgical navigators usually only use a single thread to continuously execute various tasks, which not only has low execution efficiency, but may also cause task blocking, further reducing the real-time performance of surgical instrument tracking and navigation guidance, and seriously affecting surgical accuracy. In response to the above problems, an embodiment of the present application proposes an image processing method for a binocular surgical navigator, which executes various tasks in parallel and flexibly adjusts the number of threads corresponding to each task based on the CPU occupancy, thereby maximizing the image processing efficiency within the range allowed by the CPU performance.

[0041] More specifically, the embodiment of the present application creates a thread pool in advance according to the number of CPU cores available in the binocular surgical navigator when the binocular surgical navigator is started, and the preset thread pool includes multiple image processing threads and multiple non-image processing threads. It can be understood that the image processing thread is used to perform image processing tasks, and the non-image processing thread is used to perform non-image processing tasks (i.e., the aforementioned image acquisition tasks, navigation guidance tasks, and data storage tasks). The embodiment of the present application preferably uses a number of threads close to the number of CPU cores to make full use of system resources and avoid excessive competition. For example, if the system has 8 CPU cores, the initial number of threads can be selected as 6, and of course it can also be set to other numbers, only to ensure that sufficient CPU resources are reserved to avoid the situation where the CPU resources are tight due to the increase in the amount of tasks during the surgical stage. Through the setting of the preset thread pool, the embodiment of the present application can realize the parallel execution of various tasks, avoid the occurrence of task blocking as much as possible, and thus ensure the real-time tracking and navigation guidance of surgical instruments. Considering that the image processing task consumes the most resources, and its processing efficiency directly affects the real-time tracking and navigation guidance of surgical instruments, the embodiment of the present application further sets multiple image processing threads in the preset thread pool for executing the image processing tasks to be executed. Specifically, the image processing task to be executed includes a first image processing subtask to be executed and a second image processing subtask to be executed, the first image processing subtask to be executed is used to process the image captured by the left camera of the binocular camera, and the second image processing subtask to be executed is used to process the image captured by the right camera of the binocular camera. Accordingly, the image processing thread in the preset thread pool is started to execute the image processing task to be executed, specifically including:

[0042] Starting a first subset of image processing threads in a preset thread pool to execute a first to-be-executed image processing subtask, and starting a second subset of image processing threads in a preset thread pool to execute a second to-be-executed image processing subtask;

[0043] The number of image processing threads in the first subset of image processing threads and the second subset of image processing threads is the same.

[0044] It can be understood that in the embodiments of the present application, first, the first subset of image processing threads and the second subset of image processing threads are used to perform parallel processing on the images collected by the left camera and the right camera respectively. Based on this, the execution efficiency of the image processing task can be at least doubled compared to single-thread processing. Therefore, the number of image processing threads in the preset thread pool is an even number. Based on the foregoing, it can be known that the non-image processing threads include image acquisition threads, navigation guidance threads, and data storage threads. In the embodiments of the present application, considering that the resources required for each task may vary in different application scenarios, the number of each type of non-image processing thread is at least one. Based on this, it is possible to avoid blocking of non-image processing tasks while ensuring parallel execution of each task.

[0045] It should be noted that when the number of image processing threads in the first subset of image processing threads and the second subset of image processing threads is greater than one, starting the first subset of image processing threads in the preset thread pool to execute the first pending image processing sub-task and starting the second subset of image processing threads in the preset thread pool to execute the second pending image processing sub-task specifically includes:

[0046] Using multiple image processing threads in the first subset of image processing threads to respectively execute the processing tasks of multiple sub-images obtained by evenly dividing the images collected by the left camera in the first pending image processing sub-task, and using multiple image processing threads in the second subset of image processing threads to respectively execute the processing tasks of multiple sub-images obtained by evenly dividing the images collected by the right camera in the second pending image processing sub-task;

[0047] Among them, the number of sub-images obtained by evenly dividing the images collected by the left camera is the same as the number of image processing threads in the first subset of image processing threads.

[0048] It can be understood that when the number of image processing threads in the first subset of image processing threads and the second subset of image processing threads is greater than one, the image acquisition task will pre-divide the acquired images into multiple sub-images based on the number of image processing threads in the first subset of image processing threads and the second subset of image processing threads. Based on this, multiple image processing threads in the first subset of image processing threads can be used to separately execute the processing tasks of the multiple sub-images obtained by evenly dividing the images acquired by the left camera in the first to-be-executed image processing sub-task, and multiple image processing threads in the second subset of image processing threads can be used to separately execute the processing tasks of the multiple sub-images obtained by evenly dividing the images acquired by the right camera in the second to-be-executed image processing sub-task, so as to further improve the efficiency of image processing, and at the same time ensure that the processing processes of each image processing thread are consistent, eliminate the waiting time caused by inconsistent processing processes of multiple sub-images corresponding to the same image, and thus maximize the image processing efficiency.

[0049] Furthermore, considering that too high CPU occupancy rate will cause the system response speed to slow down, and in severe cases, it may even cause problems such as system crash or deadlock. In the working process of the binocular surgical navigator in the embodiment of the present application, the real-time occupancy rate of the CPU will be continuously monitored, and the execution threads of the image processing task and other tasks will be adjusted based on the real-time occupancy rate of the CPU.

[0050] Step 102: Adjust the image processing threads corresponding to the to-be-executed image processing task based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the working threads.

[0051] Specifically, considering that the priorities of tasks in different surgical stages are different, the embodiment of the present application will adjust the image processing threads corresponding to the to-be-executed image processing task based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the working threads, so as to maximize the image processing efficiency within the range allowed by the CPU performance. More specifically, Figure 2 is a schematic diagram of the adjustment process of the image processing threads corresponding to the to-be-executed image processing task provided by the present application. As Figure 2 shown, the adjustment of the image processing threads corresponding to the to-be-executed image processing task based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the working threads specifically includes:

[0052] Step 1021: Determine the corresponding task priority sorting based on the current surgical stage, and determine the priority of the to-be-executed image processing task based on the task priority sorting;

[0053] Step 1022: Adjust the image processing thread corresponding to the image processing task to be executed based on the priority of the image processing task to be executed, the real-time occupancy rate of the CPU, and the CPU occupancy of the working threads.

[0054] Among them, the surgical stage is divided into the pre-lesion entry stage, the in-lesion entry stage, and the post-lesion entry stage. Correspondingly, for the pre-lesion entry stage, the corresponding task priority order is: image acquisition task > image processing task = navigation guidance task > data storage task;

[0055] For the in-lesion entry stage, the corresponding task priority order is: image acquisition task = image processing task = navigation guidance task > data storage task;

[0056] For the post-lesion entry stage, the corresponding task priority order is: navigation guidance task > image acquisition task = image processing task > data storage task.

[0057] Through research, it is found in this application that in the pre-lesion entry stage, it is necessary to collect binocular images of the surgical scene in real time to obtain the current surgical environment information. Therefore, the image acquisition task has the highest priority. However, before entering the lesion, the real-time and accuracy requirements for navigation guidance are relatively low. Therefore, the requirements for image processing and navigation guidance tasks are relatively low. Therefore, the priorities of the image processing and navigation guidance tasks are the same and lower than the image acquisition task. Further, the data at this time has very low reference value for subsequent review and analysis. Therefore, the data storage task has the lowest priority.

[0058] In the in-lesion entry stage, since it is necessary to ensure surgical precision, the real-time and accuracy requirements for navigation guidance are relatively high. At this time, the priorities of the image acquisition task, the image processing task, and the navigation guidance task are equal. However, since no surgical operation has been performed at this stage, the data at this time still has relatively low reference value for subsequent review and analysis. Therefore, the data storage task has the lowest priority.

[0059] In the post-lesion entry stage, since it involves fine assistance for surgical operations, the requirements for the real-time and accuracy of navigation guidance reach the highest level. Correspondingly, the navigation guidance task has the highest priority. The image acquisition task and the image processing task are also equally important. Therefore, the priorities of the image acquisition task and the image processing task are the same and lower than the navigation guidance task. At the same time, although the data storage task is also relatively important at this time, its priority is still the lowest compared to the other several tasks.

[0060] Based on this, the embodiments of the present application can quickly determine the corresponding task priority ranking based on the current surgical stage, and determine the priority of the image processing task to be executed based on the task priority ranking. After determining the priority of the image processing task to be executed, the image processing thread corresponding to the image processing task to be executed can be adjusted based on the priority of the image processing task to be executed, the real-time occupancy rate of the CPU, and the CPU occupancy of the working threads. Specifically, the adjustment of the image processing thread corresponding to the image processing task to be executed based on the priority of the image processing task to be executed, the real-time occupancy rate of the CPU, and the CPU occupancy of the working threads specifically includes:

[0061] When the real-time occupancy rate of the CPU does not exceed the first preset threshold, if the CPU occupancy rate of the working image processing thread is lower than the second preset threshold, the number of image processing threads corresponding to the image processing task to be executed is reduced; if the CPU occupancy rate of the working image processing thread is higher than the third preset threshold, the number of image processing threads corresponding to the image processing task to be executed is increased;

[0062] When the real-time occupancy rate of the CPU exceeds the first preset threshold, if the CPU occupancy rate of the working image processing thread is lower than the second preset threshold, the number of image processing threads corresponding to the image processing task to be executed is reduced; if the CPU occupancy rate of the working image processing thread is higher than the third preset threshold, the non-image processing thread with a priority lower than that of the image processing thread is modified to an image processing thread for executing the image processing task to be executed;

[0063] Among them, the first to third preset thresholds are in ascending order: the second preset threshold, the third preset threshold, and the first preset threshold.

[0064] It can be understood that the first preset threshold is the CPU occupancy critical value that causes the system response speed to slow down. In order to ensure the real-time performance of surgical instrument tracking and navigation guidance, the embodiments of the present application need to avoid this situation. It should be noted that since the requirements for the real-time performance and accuracy of surgical instrument tracking and navigation guidance are different in different surgical stages (increasing in turn according to the pre-lesion entry stage, in-lesion entry stage, and post-lesion entry stage), therefore, in order to meet the requirements of real-time performance and accuracy, the binocular surgical navigator of the embodiments of the present application will gradually increase the image sampling rate according to the order of the three surgical stages. Based on this, during the surgical process, the CPU resources required for image processing tasks will gradually increase. And the number of image processing threads set in the preset thread pool may be excessive in the pre-lesion entry stage and insufficient in the subsequent stages. Based on this, the embodiments of the present application flexibly adjust the number of image processing threads to maximize the image processing efficiency within the range allowed by the CPU performance, thereby ensuring the real-time performance of surgical instrument tracking and navigation guidance.

[0065] Specifically, when the real-time occupancy rate of the CPU does not exceed the first preset threshold, it is necessary to further consider the CPU occupancy rate of the working image processing threads. If the CPU occupancy rate of the image processing threads is too low, it will lead to resource waste and unbalanced load. If the CPU occupancy rate of the image processing threads is too high, it will cause the image processing tasks to be blocked. Based on this, in the embodiments of the present application, when the CPU occupancy rate of the working image processing threads is lower than the second preset threshold, the number of image processing threads corresponding to the to-be-executed image processing tasks is reduced. If the CPU occupancy rate of the working image processing threads is higher than the third preset threshold, the number of image processing threads corresponding to the to-be-executed image processing tasks is increased. It can be understood that the first to third preset thresholds are in ascending order: the second preset threshold, the third preset threshold, and the first preset threshold. The first to third preset thresholds can be determined in advance through experiments, and the embodiments of the present application do not make specific limitations here.

[0066] When the real-time occupancy rate of the CPU exceeds the first preset threshold, it is necessary to release the CPU resources in a timely manner. Therefore, at this time, if the CPU occupancy rate of the working image processing threads is lower than the second preset threshold, the number of image processing threads corresponding to the to-be-executed image processing tasks is reduced. If the CPU occupancy rate of the working image processing threads is higher than the third preset threshold, the non-image processing threads with a lower priority than the image processing threads are modified into image processing threads to execute the to-be-executed image processing tasks. Based on this, it can not only release the CPU resources to prevent the real-time occupancy rate of the CPU from exceeding the first preset threshold, but also avoid the problem of reduced image processing efficiency caused by releasing the CPU resources.

[0067] The method provided by the embodiments of the present application starts the image processing threads in the preset thread pool to execute the to-be-executed image processing tasks when there are to-be-executed image processing tasks, and monitors the real-time occupancy rate of the CPU; the preset thread pool includes multiple image processing threads and multiple non-image processing threads; based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the working threads, the image processing threads corresponding to the to-be-executed image processing tasks are adjusted. It can execute the image processing tasks based on the preset multiple threads, and at the same time flexibly adjust the number of image processing threads based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the working threads, and can maximize the image processing efficiency within the range allowed by the CPU performance, thereby ensuring the real-time performance of surgical instrument tracking and navigation guidance.

[0068] Next, the image processing device of the binocular surgical navigator provided by the present application will be described. The image processing device of the binocular surgical navigator described below can be correspondingly referred to the image processing method of the binocular surgical navigator described above.

[0069] Based on any of the above embodiments, Figure 3 is a schematic structural diagram of an image processing device of a binocular surgical navigator provided by the present application. As Figure 3 shown, the device includes:

[0070] A first processing module 201, configured to, when a to-be-executed image processing task occurs, start an image processing thread in a preset thread pool to execute the to-be-executed image processing task, and monitor the real-time occupancy rate of the CPU; the preset thread pool includes a plurality of image processing threads and a plurality of non-image processing threads;

[0071] A second processing module 202, configured to adjust the image processing thread corresponding to the to-be-executed image processing task based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the threads that are working.

[0072] In the device provided by the embodiments of the present application, when a to-be-executed image processing task occurs, the first processing module 201 starts an image processing thread in a preset thread pool to execute the to-be-executed image processing task, and monitors the real-time occupancy rate of the CPU; the preset thread pool includes a plurality of image processing threads and a plurality of non-image processing threads; the second processing module 202 adjusts the image processing thread corresponding to the to-be-executed image processing task based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the threads that are working, so that the image processing task can be executed based on a plurality of preset threads, and at the same time, the number of image processing threads can be flexibly adjusted based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the threads that are working, and the image processing efficiency can be maximally improved within the range allowed by the CPU performance, thereby ensuring the real-time performance of surgical instrument tracking and navigation guidance.

[0073] Based on the above embodiments, the to-be-executed image processing task includes a first to-be-executed image processing subtask and a second to-be-executed image processing subtask. The first to-be-executed image processing subtask is used to process the image collected by the left camera of the binocular camera, and the second to-be-executed image processing subtask is used to process the image collected by the right camera of the binocular camera. Correspondingly, starting an image processing thread in the preset thread pool to execute the to-be-executed image processing task specifically includes:

[0074] Starting a first subset of image processing threads in the preset thread pool to execute the first to-be-executed image processing subtask, and starting a second subset of image processing threads in the preset thread pool to execute the second to-be-executed image processing subtask;

[0075] The number of image processing threads in the first subset of image processing threads and the second subset of image processing threads is the same.

[0076] Based on any of the above embodiments, the number of image processing threads in the preset thread pool is an even number. The non-image processing threads include an image acquisition thread, a navigation guidance thread, and a data storage thread, and the number of each type of non-image processing thread is at least one.

[0077] Based on any of the above embodiments, when the number of image processing threads in the first image processing thread subset and the second image processing thread subset is greater than one, starting the first image processing thread subset in the preset thread pool to execute the first to-be-executed image processing sub-task, and starting the second image processing thread subset in the preset thread pool to execute the second to-be-executed image processing sub-task, specifically including:

[0078] Using multiple image processing threads in the first image processing thread subset to respectively execute the processing tasks of multiple sub-images obtained by equally dividing the images collected by the left camera in the first to-be-executed image processing sub-task, and using multiple image processing threads in the second image processing thread subset to respectively execute the processing tasks of multiple sub-images obtained by equally dividing the images collected by the right camera in the second to-be-executed image processing sub-task;

[0079] Wherein, the number of sub-images obtained by equally dividing the images collected by the left camera is the same as the number of image processing threads in the first image processing thread subset.

[0080] Based on any of the above embodiments, adjusting the image processing thread corresponding to the to-be-executed image processing task based on the real-time occupancy rate of the CPU, the current surgical stage, and the CPU occupancy of the threads that are currently working, specifically including:

[0081] Based on the current surgical stage, determining the corresponding task priority ranking, and determining the priority of the to-be-executed image processing task based on the task priority ranking;

[0082] Based on the priority of the to-be-executed image processing task, the real-time occupancy rate of the CPU, and the CPU occupancy of the threads that are currently working, adjusting the image processing thread corresponding to the to-be-executed image processing task.

[0083] Based on any of the above embodiments, the surgical stage is divided into the stage before entering the lesion, the stage during entering the lesion, and the stage after entering the lesion. Correspondingly, for the stage before entering the lesion, the corresponding task priority ranking is: image acquisition task > image processing task = navigation guidance task > data storage task;

[0084] For the stage during entering the lesion, the corresponding task priority ranking is: image acquisition task = image processing task = navigation guidance task > data storage task;

[0085] For the post-lesion stage, the corresponding task priority order is: navigation guidance task > image acquisition task = image processing task > data storage task.

[0086] Based on any of the above embodiments, adjusting the image processing thread corresponding to the image processing task to be executed according to the priority of the image processing task to be executed, the real-time occupancy rate of the CPU, and the CPU occupancy of the working threads specifically includes:

[0087] When the real-time occupancy rate of the CPU does not exceed the first preset threshold, if the CPU occupancy of the working image processing thread is lower than the second preset threshold, reduce the number of image processing threads corresponding to the image processing task to be executed; if the CPU occupancy of the working image processing thread is higher than the third preset threshold, increase the number of image processing threads corresponding to the image processing task to be executed;

[0088] When the real-time occupancy rate of the CPU exceeds the first preset threshold, if the CPU occupancy of the working image processing thread is lower than the second preset threshold, reduce the number of image processing threads corresponding to the image processing task to be executed; if the CPU occupancy of the working image processing thread is higher than the third preset threshold, modify the non-image processing thread with a priority lower than that of the image processing thread into an image processing thread to execute the image processing task to be executed;

[0089] Wherein, the first to third preset thresholds are in ascending order: the second preset threshold, the third preset threshold, and the first preset threshold.

[0090] Figure 4 An example of the physical structure diagram of an electronic device is shown as Figure 4 shown. The electronic device may include: a processor 301, a communication interface 302, a memory 303, and a communication bus 304. Among them, the processor 301, the communication interface 302, and the memory 303 complete communication with each other through the communication bus 304. The processor 301 can call the logical instructions in the memory 303 to execute the image processing method of the binocular surgical navigator provided by the above methods. The method includes: when an image processing task to be executed appears, start the image processing threads in the preset thread pool to execute the image processing task to be executed, and monitor the real-time occupancy rate of the CPU; the preset thread pool includes multiple image processing threads and multiple non-image processing threads; adjust the image processing thread corresponding to the image processing task to be executed based on the real-time occupancy rate of the CPU, the surgical stage, and the CPU occupancy of the working threads.

[0091] In addition, when the logical instructions in the above-mentioned memory 303 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), magnetic disks, or optical discs.

[0092] On the other hand, this application also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image processing method of the binocular surgical navigator provided by the above-mentioned various methods. The method includes: when there is an image processing task to be executed, starting an image processing thread in a preset thread pool to execute the image processing task to be executed, and monitoring the real-time occupancy rate of the CPU; the preset thread pool includes multiple image processing threads and multiple non-image processing threads; based on the real-time occupancy rate of the CPU, the surgical stage, and the CPU occupancy of the threads that are working, adjusting the image processing thread corresponding to the image processing task to be executed.

[0093] On another aspect, this application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is used to execute the image processing method of the binocular surgical navigator provided by the above-mentioned various methods. The method includes: when there is an image processing task to be executed, starting an image processing thread in a preset thread pool to execute the image processing task to be executed, and monitoring the real-time occupancy rate of the CPU; the preset thread pool includes multiple image processing threads and multiple non-image processing threads; based on the real-time occupancy rate of the CPU, the surgical stage, and the CPU occupancy of the threads that are working, adjusting the image processing thread corresponding to the image processing task to be executed.

[0094] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing device for a binocular surgical navigator, characterized in that, The device includes: A first processing module, configured to, when a to-be-executed image processing task occurs, start an image processing thread in a preset thread pool to execute the to-be-executed image processing task, and monitor the real-time occupancy rate of the CPU; the preset thread pool includes multiple image processing threads and multiple non-image processing threads; A second processing module, configured to adjust the image processing thread corresponding to the to-be-executed image processing task based on the real-time occupancy rate of the CPU, the surgical stage, and the CPU occupancy of the threads that are working; The second processing module is further configured to determine a corresponding task priority sorting based on the current surgical stage, and determine the priority of the to-be-executed image processing task based on the task priority sorting; adjust the image processing thread corresponding to the to-be-executed image processing task based on the priority of the to-be-executed image processing task, the real-time occupancy rate of the CPU, and the CPU occupancy of the threads that are working; The second processing module is further configured to, when the real-time occupancy rate of the CPU does not exceed a first preset threshold, if the CPU occupancy of the working image processing threads is lower than a second preset threshold, reduce the number of image processing threads corresponding to the to-be-executed image processing task, and if the CPU occupancy of the working image processing threads is higher than a third preset threshold, increase the number of image processing threads corresponding to the to-be-executed image processing task; when the real-time occupancy rate of the CPU exceeds the first preset threshold, if the CPU occupancy of the working image processing threads is lower than the second preset threshold, reduce the number of image processing threads corresponding to the to-be-executed image processing task, and if the CPU occupancy of the working image processing threads is higher than the third preset threshold, modify the non-image processing threads with priorities lower than those of the image processing threads into image processing threads to be used for executing the to-be-executed image processing task; wherein, the first to third preset thresholds are in ascending order: the second preset threshold, the third preset threshold, the first preset threshold; The surgical stage is divided into a stage before entering the lesion, a stage during entering the lesion, and a stage after entering the lesion. Correspondingly, for the stage before entering the lesion, the corresponding task priority sorting is: image acquisition task > image processing task = navigation guidance task > data storage task; For the stage during entering the lesion, the corresponding task priority sorting is: image acquisition task = image processing task = navigation guidance task > data storage task; For the stage after entering the lesion, the corresponding task priority sorting is: navigation guidance task > image acquisition task = image processing task > data storage task.

2. The image processing device of the binocular surgical navigator according to claim 1, characterized in that, The to-be-executed image processing task includes a first to-be-executed image processing subtask and a second to-be-executed image processing subtask. The first to-be-executed image processing subtask is used to process the image collected by the left camera of the binocular camera, and the second to-be-executed image processing subtask is used to process the image collected by the right camera of the binocular camera; The first processing module is further configured to start a first subset of image processing threads in the preset thread pool to execute the first to-be-executed image processing subtask, and start a second subset of image processing threads in the preset thread pool to execute the second to-be-executed image processing subtask; The number of image processing threads in the first subset of image processing threads and the second subset of image processing threads is the same.

3. The image processing device of the binocular surgical navigator according to claim 2, characterized in that, The number of image processing threads in the preset thread pool is an even number. The non-image processing threads include an image acquisition thread, a navigation guidance thread, and a data storage thread, and the number of each type of non-image processing thread is at least one.

4. The image processing device of the binocular surgical navigator according to claim 3, characterized in that, When the number of image processing threads in the first subset of image processing threads and the second subset of image processing threads is greater than one, the first processing module is further configured to use multiple image processing threads in the first subset of image processing threads to respectively execute the processing tasks of multiple sub-images obtained by evenly dividing the images acquired by the left camera in the first to-be-executed image processing sub-tasks, and use multiple image processing threads in the second subset of image processing threads to respectively execute the processing tasks of multiple sub-images obtained by evenly dividing the images acquired by the right camera in the second to-be-executed image processing sub-tasks; wherein, the number of sub-images obtained by evenly dividing the images acquired by the left camera is the same as the number of image processing threads in the first subset of image processing threads.

Images