Image processing method and device and storage medium

Abstract

The invention provides an image processing method and device, and a storage medium, relates to the technical field of image processing, and can solve the problems of complex flow and low image processing efficiency caused by the need of skipping in a plurality of different threads in the current image processing technology. The method comprises the steps that collected images are input into image processing threads in an idle state in M image processing threads, the number M of the image processing threads is determined according to an image collection time interval and the processing time of a plurality of image processing algorithms, and M is a positive integer; and controlling the image processing thread in the idle state to sequentially run a plurality of image processing algorithms, and processing the image to obtain an image processing result. The image processing flow can be simplified, and the image processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and storage medium. Background Art

[0002] During the cone computed tomography process, the gantry rotates in a circle around the target object to be inspected, emitting X-rays to the part to be imaged of the target object during the rotation process, and capturing the projection data generated after the X-rays pass through the patient's imaging part. The computer then pre-processes the projection data and combines the pre-processed images through a reconstruction algorithm to construct a three-dimensional image, which is then post-processed to complete high-quality three-dimensional imaging.

[0003] Taking the pre-processing process as an example, the image pre-processing requires multiple algorithms to complete. In the current pre-processing process, a set of independent threads is set for each algorithm. In this way, it is necessary to jump between different threads during the pre-processing process, which makes the pre-processing process extremely complicated. Summary of the invention

[0004] The present application provides an image processing method, device and storage medium, which can solve the problem that the current image processing technology needs to jump among multiple different threads, resulting in complex processes and low image processing efficiency, simplify the image processing process, and improve the efficiency of image processing.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides an image processing method, the method comprising: inputting a captured image into an idle image processing thread among M image processing threads, the number M of image processing threads being determined according to an image capture time interval and a processing time of multiple image processing algorithms, and M being a positive integer; controlling the idle image processing thread to sequentially run multiple image processing algorithms to process the image and obtain an image processing result.

[0007] In combination with the first aspect above, in a possible implementation manner, the number M of image processing threads is a ratio of the sum of processing times of multiple image processing algorithms to an acquisition time interval.

[0008] In combination with the above-mentioned first aspect, in a possible implementation method, when there are O image processing threads in an idle state among M image processing threads, O is a positive integer less than or equal to M, and the above-mentioned step of inputting the captured images into the image processing threads in the idle state among the M image processing threads includes: corresponding to the number of images N being less than or equal to the number of image processing threads O in the idle state, the N images are respectively input into the N image processing threads among the O image processing threads; N is a positive integer; corresponding to the number of images P being greater than the number of image processing threads O in the idle state, O images among the P images are respectively input into the O image processing threads; when there is an image processing thread among the M image processing threads that changes from a processing state to an idle state, P0 images are sequentially input into the image processing thread in the idle state; P is a positive integer.

[0009] In combination with the first aspect above, in a possible implementation, before the captured image is input into an image processing thread in an idle state among the M image processing threads, the method also includes the step of caching the image into an image cache space, and the step of caching the image into the image cache space includes: determining whether there is free space in the current image cache space, and if there is free space, storing the captured image in the free space; otherwise, waiting until there is free space in the image cache space, and storing the captured image in the free space.

[0010] In combination with the first aspect above, in a possible implementation method, the above-mentioned inputting the captured image into an idle image processing thread among M image processing threads includes: inputting the image stored in the image cache space that meets the preset processing conditions into the idle image processing thread; wherein the preset processing conditions include the image capture order and / or the image capture angle.

[0011] In combination with the first aspect above, in a possible implementation method, the above-mentioned image processing thread in an idle state is controlled to run multiple image processing algorithms in sequence to process the image and obtain an image processing result, including: when the image processing thread in an idle state has a competitor for competing for processor resources, processor resources are obtained for the image processing thread in an idle state through locking processing and / or resource pool processing; the processor resources are used to provide the image processing thread with the ability to run multiple image processing algorithms; the competitor is any image processing thread other than the image processing thread; based on the processor resources, the image processing thread in an idle state is controlled to run multiple image processing algorithms in sequence to process the image and obtain an image processing result.

[0012] In combination with the above-mentioned first aspect, in a possible implementation manner, the method further includes: performing a first processing on the image processing result obtained by the image processing thread; the first processing includes at least one of the following: image storage processing and image visualization processing.

[0013] In combination with the above-mentioned first aspect, in a possible implementation method, the above-mentioned first processing of the image processing results obtained by the image processing thread includes: storing the image processing results in a shared cache space; controlling the visualization thread to obtain the image processing results from the shared cache space, and controlling the visualization thread to perform visualization processing on the image processing results to display the image processing results.

[0014] In a second aspect, the present application provides an image processing device, comprising: a processing unit; a processing unit, used to input the captured image into an idle image processing thread among M image processing threads, the number M of image processing threads is determined according to the image capture time interval and the processing time of multiple image processing algorithms, and M is a positive integer; the processing unit is also used to control the idle image processing thread to run multiple image processing algorithms in sequence, process the image, and obtain an image processing result.

[0015] In combination with the second aspect above, in a possible implementation manner, the number M of image processing threads is a ratio of the sum of processing times of multiple image processing algorithms to an acquisition time interval.

[0016] In combination with the above-mentioned second aspect, in a possible implementation method, when there are O image processing threads in an idle state among M image processing threads, O is a positive integer less than or equal to M, and the processing unit is specifically used to: corresponding to the number of images N being less than or equal to the number of image processing threads O in the idle state, input the N images into the N image processing threads among the O image processing threads respectively; N is a positive integer; corresponding to the number of images P being greater than the number of image processing threads O in the idle state, input O images among the P images into the O image processing threads respectively; when there is an image processing thread among the M image processing threads that changes from a processing state to an idle state, input P0 images into the image processing thread in the idle state in sequence; P is a positive integer.

[0017] In combination with the above-mentioned second aspect, in a possible implementation method, before the captured image is input into an image processing thread in an idle state among the M image processing threads, the processing unit is also used to: determine whether there is free space in the current image cache space, and if there is free space, store the captured image in the free space; otherwise, wait until there is free space in the image cache space, and store the captured image in the free space.

[0018] In combination with the above-mentioned second aspect, in a possible implementation method, the processing unit is specifically used to: input an image stored in an image cache space that meets preset processing conditions into an idle image processing thread; wherein the preset processing conditions include the image acquisition order and / or the image acquisition angle.

[0019] In combination with the above-mentioned second aspect, in a possible implementation method, the processing unit is specifically used to: when an image processing thread in an idle state has a competitor for competing for processor resources, obtain processor resources for the image processing thread in an idle state through locking processing and / or resource pool processing; the processor resources are used to provide the image processing thread with the ability to run multiple image processing algorithms; the competitor is any image processing thread other than the image processing thread; based on the processor resources, control the image processing thread in an idle state to run multiple image processing algorithms in sequence, process the image, and obtain an image processing result.

[0020] In combination with the above second aspect, in a possible implementation, the processing unit is further used to: perform a first processing on the image processing result obtained by the image processing thread; the first processing includes at least one of the following: image storage processing and image visualization processing.

[0021] In combination with the above-mentioned second aspect, in a possible implementation method, the image processing device also includes a communication unit; the processing unit is also used to: store the image processing results in a shared cache space through the communication unit; control the visualization thread to obtain the image processing results from the shared cache space through the communication unit, and control the visualization thread to perform visualization processing on the image processing results to display the image processing results.

[0022] In a third aspect, the present application provides an electronic device comprising: a processor; and a memory configured to store instructions executable by the processor, wherein the processor is configured to execute instructions to implement the image processing method described in the first aspect and any possible implementation manner of the first aspect.

[0023] In combination with the third aspect above, in a possible implementation method, when the internal processing logic of the processor is to support processing data from multiple image processing threads, and / or the processing logic of multiple image processing threads is the same, the processor is one processor; one processor is specifically used to run multiple image processing threads; when the processing logic corresponding to multiple image processing threads is different, the processor is multiple processors, and each processor is specifically used to run one image processing thread.

[0024] Optionally, the electronic device also includes a communication interface; the communication interface is coupled to the processor.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal, the terminal executes the image processing method described in the first aspect and any possible implementation of the first aspect.

[0026] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when the computer program product is run on an image processing device, enables the image processing device to perform the image processing method described in the first aspect and any possible implementation of the first aspect.

[0027] In a sixth aspect, the present application provides a chip, the chip comprising a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run a computer program or instructions to implement the image processing method described in the first aspect and any possible implementation method of the first aspect.

[0028] Specifically, the chip provided in the present application also includes a memory for storing computer programs or instructions.

[0029] It should be noted that the above computer instructions may be stored in whole or in part on a computer-readable storage medium, wherein the computer-readable storage medium may be packaged together with the processor of the device, or may be packaged separately from the processor of the device, which is not limited in this application.

[0030] In a seventh aspect, the present application provides an image processing system, comprising: a data processing server and a data storage server, wherein the data processing server is used to execute the image processing method described in the first aspect and any possible implementation of the first aspect.

[0031] The descriptions of the second to seventh aspects of the present application can refer to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second to seventh aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0032] In this application, the name of the above-mentioned image processing device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalent technologies.

[0033] These and other aspects of the present application will become more apparent from the following description.

[0034] Based on the above technical solution, the image processing method provided by the present application inputs the collected image into an image processing thread in an idle state among M image processing threads, and controls the image processing thread in an idle state to run multiple image processing algorithms in sequence to process the image and obtain an image processing result. In other words, in the above technical solution, image processing can be completed in one thread. Thus, the problem that the current image processing technology needs to jump among multiple different threads, resulting in a complicated process and low image processing efficiency can be solved, the image processing process is simplified, and the efficiency of image processing is improved.

[0035] In addition, the number M of image processing threads is determined according to the image acquisition time interval and the processing time of multiple image processing algorithms, and M is a positive integer. In other words, a graphics thread can complete image processing while the next image is being acquired, and can then continue to process the next image. This avoids the problem of image processing backlogs caused by unreasonable design of the number of threads, ensures the smoothness of image processing, avoids time backlogs, and improves image processing efficiency. Furthermore, according to the above rules, the number of image processing threads can be reasonably planned to avoid the phenomenon of a large number of image processing threads being idle during the processing process, so that the threads can be utilized to the greatest extent, thereby reducing the cost of developing threads. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of a pre-processing thread provided in the related art;

[0037] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of an image processing thread provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of another image processing thread provided in an embodiment of the present application;

[0040] Figure 5 A flowchart of another image processing method provided in an embodiment of the present application;

[0041] Figure 6 A flowchart of an image processing method provided in an embodiment of the present application;

[0042] Figure 7 A flowchart of another image processing method provided in an embodiment of the present application;

[0043] Figure 8 A flowchart of another image processing method provided in an embodiment of the present application;

[0044] Fig. 9 A schematic diagram of the architecture of an image processing system provided in an embodiment of the present application;

[0045] Fig.10 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0046] Fig.11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0049] The terms "first" and "second" and the like in the specification and drawings of this application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0050] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0051] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0052] In the description of the present application, unless otherwise specified, “plurality” means two or more.

[0053] During the cone computed tomography process, the gantry rotates in a circle around the irradiator of the target object to be inspected, and during the rotation, it emits X-rays to the part to be photographed of the target object and captures the projection data generated after the X-rays pass through the patient's photographing part. The computer then pre-processes the projection data and combines the pre-processed images through a reconstruction algorithm to construct a three-dimensional image, which is then post-processed to complete high-quality three-dimensional imaging.

[0054] Specifically, cone computed tomography consists of four parts: gantry rotation and image acquisition, image pre-processing, image reconstruction, and image post-processing.

[0055] 1. Frame rotation and image acquisition.

[0056] During the cone computed tomography scan, the gantry (i.e., the device equipped with the X-ray source and detector) rotates in a circle around the object to be examined (e.g., the patient's head, oral cavity, etc.). Every time the gantry rotates a certain angle, the X-ray source emits X-rays, and the detector captures the attenuation information of these X-rays after they penetrate the object to obtain projection data.

[0057] 2. Image preprocessing.

[0058] After the computer receives the projection data, it performs a series of pre-processing operations, such as correction, filtering and denoising, to eliminate or reduce adverse factors such as noise and distortion in the projection data, improve the quality of the projection data, and reduce artifacts, providing high-quality input data for subsequent image reconstruction.

[0059] 3. Image reconstruction.

[0060] The pre-processed projection data will be input into the reconstruction algorithm, and the reconstruction algorithm will combine the pre-processed projection data to construct a three-dimensional image.

[0061] Among them, the reconstruction algorithm includes the Feldkamp-Davis-Kress (FDK) algorithm, etc. The principle of this algorithm is to convert two-dimensional projection data into a three-dimensional image through methods such as filtered back projection.

[0062] 4. Image post-processing.

[0063] The reconstructed images are post-processed, such as contrast adjustment, smoothing, segmentation and annotation, to meet specific diagnostic and treatment needs. Image post-processing can enhance the readability of the image to assist medical personnel in accurately identifying lesions or anatomical structures.

[0064] Taking the pre-processing process as an example, the pre-processing of an image requires multiple algorithms to be completed together. In the current pre-processing process, a set of independent threads is set for each algorithm, or a set of independent threads is set for some algorithms. In this way, during the pre-processing process, the image needs to jump in different threads so that each thread runs its corresponding algorithm to process the image, which makes the pre-processing process extremely complicated. For example, take setting up two sets of independent threads as an example. The image needs to enter the first set of threads for processing first, and then enter the second set of threads for processing after the processing is completed, so that the processing of the image is completed.

[0065] Furthermore, in the current pre-processing process, in order to ensure that image processing does not generate time accumulation, multiple threads are set for the target algorithm, that is, a set of threads running the target algorithm includes multiple threads and the multiple threads are all used to run the target algorithm. Figure 1 FIG. 1 is a schematic diagram of a pre-processing thread provided in the related art. Figure 1 As shown in the figure, taking the image pre-processing that needs to be processed by Algorithm 1 and Algorithm 2 as an example, three threads are set for Algorithm 1 ( Figure 1 There are 3 rows corresponding to Algorithm 1, each row represents a thread, including thread 1, thread 2, and thread 3). There are also 3 threads for Algorithm 2 ( Figure 1 There are 3 rows corresponding to Algorithm 2, each row represents a thread, including thread 4, thread 5, and thread 6). Although this guarantees the image processing requirements, there are too many threads and the cost is high. In addition, during the image processing process, some threads will be idle and cannot be fully utilized.

[0066] For example, in thread 1, algorithm 1 is run to process image 1 (corresponding to Figure 1 After 1-1), since the second image has been processed by thread 2 and the third image has been processed by thread 3, thread 1 can only wait for the fourth image to be processed, but the fourth image has not been acquired yet. Therefore, after thread 1 completes processing of image 1, it will be idle until the fourth image is acquired, and then thread 1 can process the fourth image (corresponding to Figure 1 (4-1).

[0067] Similarly, algorithm 1 is run in thread 2 to process image 2 (corresponding to Figure 1 After 2-1), since the fifth image has not been acquired, thread 2 will be idle after processing image 2 until the fifth image is acquired, and then thread 2 can process the fifth image (corresponding to Figure 1 (5-1).

[0068] Similarly, we can see that in thread 4, algorithm 2 is run to process image 1 (corresponding to Figure 1In 1-2, since the fourth image has not been processed by algorithm 1, thread 4 will be idle after processing image 1 until the fourth image is processed by algorithm 1, and then thread 4 can process the first image (corresponding to Figure 1 (4-2).

[0069] In view of this, the image processing method provided by the present application inputs the collected image into an image processing thread in an idle state among M image processing threads, and controls the image processing thread in an idle state to run multiple image processing algorithms in sequence to process the image and obtain an image processing result. In other words, in the above technical solution, image processing can be completed in one thread. Thus, the problem that the current image processing technology needs to jump between multiple different threads, resulting in a complicated process and low image processing efficiency can be solved, the image processing process is simplified, and the efficiency of image processing is improved.

[0070] In addition, the number M of image processing threads is determined according to the image acquisition time interval and the processing time of multiple image processing algorithms, and M is a positive integer. In other words, a graphics thread can complete image processing while the next image is being acquired, and can then continue to process the next image. This avoids the problem of image processing backlogs caused by an unreasonable number of threads, ensures the smoothness of image processing, avoids time backlogs, and improves image processing efficiency. Furthermore, according to the above rules, the number of image processing threads can be reasonably planned to avoid the phenomenon of a large number of image processing threads being idle during the processing process, so that the threads can be utilized to the greatest extent, thereby reducing the cost of developing threads.

[0071] The implementation of the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

[0072] It should be pointed out that the various embodiments of the present application can learn from or refer to each other. For example, the same or similar steps, method embodiments, system embodiments and device embodiments can all refer to each other without limitation.

[0073] Figure 2 This is a flow chart of an image processing method provided in an embodiment of the present application. Figure 2 As shown, the method includes the following S201-S202.

[0074] S201 . Input a captured image into an idle image processing thread among M image processing threads.

[0075] In some embodiments, the number M of image processing threads is determined according to the image acquisition time interval and the processing time of multiple image processing algorithms, and M is a positive integer.

[0076] Specifically, the implementation process of determining the number M of image processing threads is referred to the following embodiment 1, which will not be described in detail here.

[0077] In a possible implementation, when there are O image processing threads in the M image processing threads in an idle state, the implementation process of S201 may include: corresponding to the number of images N being less than or equal to the number of image processing threads in the idle state O, inputting the N images into N image processing threads in the O image processing threads respectively, where O is a positive integer less than or equal to M, and N is a positive integer.

[0078] In one example, taking the number of images N as 2 and the number of image processing threads O in an idle state as 3, two images are respectively input into two image processing threads in an idle state. For example, the images include image 1 and image 2, and the image processing threads in an idle state include image processing thread 1, image processing thread 2, and image processing thread 3. Image 1 is input into image processing thread 1, and image 2 is input into image processing thread 2.

[0079] In another possible implementation, when there are O image processing threads in the M image processing threads in an idle state, the implementation process of S201 may further include: corresponding to the number of images P being greater than the number of image processing threads O in the idle state, inputting O images in the P images into the O image processing threads respectively; when there is an image processing thread in the M image processing threads that changes from a processing state to an idle state, inputting P0 images in sequence into the image processing threads in the idle state. Wherein, P is a positive integer, and O is a positive integer less than or equal to M.

[0080] In one example, taking the number of images P as 4 and the number of image processing threads in an idle state O as 3, three images are respectively input into three image processing threads in an idle state; and waiting is performed until there is an image processing thread among the M image processing threads that changes from a processing state to an idle state, and then images 4-3 are input into the image processing thread in the idle state.

[0081] In some other embodiments, before S201, the image processing method further includes a step of caching the image into an image cache space.

[0082] The specific implementation process of caching the image into the image cache space refers to S501-S503 in Example 2; after S503, the specific implementation process of S201 refers to the embodiment shown in S504, which will not be repeated here.

[0083] S202 , controlling the image processing thread in an idle state to sequentially run multiple image processing algorithms to process the image and obtain an image processing result.

[0084] The image processing thread is configured with multiple image processing algorithms. The image processing thread can run multiple image processing algorithms in sequence to process an image. After the image processing thread runs multiple image processing algorithms in sequence to process the image and obtains the image processing result, it can run multiple image processing algorithms again to process the next image.

[0085] For example, the image processing thread is configured with two image processing algorithms, including image processing algorithm 1 and image processing algorithm 2. The image processing thread can run image processing algorithm 1 and image processing algorithm 2 in sequence to process image 1. After the image processing thread completes processing of image 1 and obtains the image processing result, it can run image processing algorithm 1 and image processing algorithm 2 again to process image 2.

[0086] In some embodiments, the specific implementation process of S202 may refer to the embodiments shown in S701-S702, and will not be repeated here.

[0087] Optionally, in some embodiments, after S202, the image processing method further includes: further processing the image processing result, and the specific process refers to the embodiment shown in S801, which will not be repeated here.

[0088] Based on the above technical solution, the image processing method provided by the present application inputs the collected image into an image processing thread in an idle state among M image processing threads, and controls the image processing thread in an idle state to run multiple image processing algorithms in sequence to process the image and obtain an image processing result. In other words, in the above technical solution, image processing can be completed in one thread. Thus, the problem that the current image processing technology needs to jump among multiple different threads, resulting in a complicated process and low image processing efficiency can be solved, the image processing process is simplified, and the efficiency of image processing is improved.

[0089] In addition, the number M of image processing threads is determined according to the image acquisition time interval and the processing time of multiple image processing algorithms, and M is a positive integer. In other words, one image processing thread can complete image processing while the next image is being acquired, and then can continue to process the next image. This avoids the problem of image processing backlog caused by unreasonable design of the number of threads, ensures the smoothness of image processing, and avoids time backlog.

[0090] In the current thread determination method, the number of threads is determined by artificial setting. Figure 3As shown, it is determined that the number of image processing threads is 3. In this case, it is taken as an example that the image acquisition time interval is 100 milliseconds, the image processing time is 400 milliseconds, the multiple image processing algorithms are two image processing algorithms, the processing time of algorithm 1 is 100 milliseconds, and the processing time of algorithm 2 is 300 milliseconds.

[0091] like Figure 3 As shown, in the process of image processing thread 1 running algorithm 1 and algorithm 2 to process the first image (corresponding to Figure 3 1-1 and 1-2), when the processing time reaches 100 milliseconds, the second image acquisition is completed, and image processing thread 2 can process the second image (corresponding to Figure 3 2-1 and 2-2); when the processing time reaches 200 milliseconds, the third image is acquired and the image processing thread 3 can process the third image (corresponding to Figure 3 3-1 and 3-2); when the processing time reaches 400 milliseconds, the fourth image is acquired. However, at this time, image processing threads 1 to 3 are all occupied and cannot process the fourth image. Therefore, the fourth image can only wait until an image processing thread completes image processing before processing the fourth image (corresponding to Figure 3 4-1 and 4-2) can be executed only after 1-2 is completed, which will cause image processing time to accumulate and affect the efficiency of image processing.

[0092] In view of this, the embodiment of the present application provides a method for determining the number of image processing threads to avoid image processing time accumulation and improve image processing efficiency. Furthermore, according to the above rules, the number of image processing threads can be reasonably planned to avoid the phenomenon of a large number of image processing threads being idle during the processing process, so that the threads can be used to the greatest extent, thereby reducing the cost of developing threads.

[0093] The following specifically introduces the implementation process of determining the number M of image processing threads through Example 1.

[0094] Embodiment 1:

[0095] In some embodiments, the number M of image processing threads is the ratio of the sum of processing times of multiple image processing algorithms to the acquisition time interval.

[0096] In one example, the image acquisition time interval is 100 milliseconds, the image processing time is 400 milliseconds, the multiple image processing algorithms are two image processing algorithms, the processing time of algorithm 1 is 100 milliseconds, and the processing time of algorithm 2 is 300 milliseconds. Figure 4As shown, the number of image processing threads M is determined based on the ratio of the sum of the processing time of algorithm 1 (100 milliseconds) and the processing time of algorithm 2 (300 milliseconds) to the image acquisition time interval of 100 milliseconds. That is, (100 milliseconds + 300 milliseconds) / 100 milliseconds = the number of image processing threads M, and M is 4.

[0097] like Figure 4 As shown, in the process of image processing thread 1 running algorithm 1 and algorithm 2 to process the first image (corresponding to Figure 4 1-1 and 1-2), when the processing time reaches 100 milliseconds, the second image acquisition is completed, and image processing thread 2 can process the second image (corresponding to Figure 4 2-1 and 2-2); when the processing time reaches 200 milliseconds, the third image is acquired and the image processing thread 3 can process the third image (corresponding to Figure 4 3-1 and 3-2); when the processing time reaches 400 milliseconds, the fourth image is acquired and the image processing thread 4 can process the fourth image (corresponding to Figure 4 4-1 and 4-2); when the processing time reaches 500 milliseconds (that is, while image processing thread 1 runs algorithm 1 and algorithm 2 to complete the image processing), the fifth image acquisition is completed, and the image processing thread 1 can continue to be used to process the fifth image (corresponding to Figure 4 5-1 and 5-2), which can not only meet the needs of image processing and avoid the accumulation of image processing time, but also reduce the number of image processing threads.

[0098] Optionally, after determining the number of image processing threads based on the ratio of the sum of processing times of multiple image processing algorithms to the acquisition time interval, the number of image processing threads is determined according to the maximum number of threads of the processor, wherein the number of image processing threads is less than or equal to the maximum number of threads of the processor.

[0099] For example, taking the example of determining the number of image processing threads to be 5 based on the ratio of the sum of the processing time of multiple image processing algorithms to the acquisition time interval, when the maximum number of threads of the processor is 4, the number of image processing threads is determined to be 4.

[0100] In the above, the implementation process of determining the number M of image processing threads is specifically introduced through Example 1.

[0101] The following specifically introduces the step of caching the acquired image into the image cache space through Example 2.

[0102] Embodiment 2:

[0103] As a possible embodiment of the present application, Figure 2 ,like Figure 5 As shown, before the above S201, the image processing method further includes a step of caching the acquired image into an image cache space.

[0104] The step of caching the collected image into the image cache space specifically includes the following S501-S503.

[0105] S501: Determine whether there is free space in the current image cache space.

[0106] Optionally, the image cache space may be a central processing unit (CPU) cache, such as an L1 cache, an L2 cache, or an L3 cache, located between the CPU and the memory, and used to temporarily store images that the CPU is about to access. Of course, the above is only an exemplary description of the image cache space, and the image cache space may also be a front-end cache, such as a browser cache, an application cache, etc., and this application does not limit this.

[0107] Optionally, further, when the size of the free space in the current image cache space is smaller than the size of one image, it is determined that there is no free space in the current image cache space.

[0108] For example, if the size of the free space in the current image cache space is 10 kilobytes (KB) and the size of an image is 20 KB, in this case, it is determined that the size of the free space in the current image cache space is less than the size of an image, and thus it is determined that there is no free space in the current image cache space.

[0109] S502: If there is free space, store the acquired image in the free space.

[0110] In a possible implementation, when the size of the free space in the current image cache space is greater than or equal to the size of an image, it is determined that there is free space in the current image cache space, and the captured image is stored in the free space.

[0111] Furthermore, in the multi-threaded image acquisition pre-processing process, such as Figure 6 As shown, after the image is captured, it is determined whether there is free space in the current image cache space. If there is free space, the captured image is stored in the free space. After the captured image is stored in the free space, the image processing thread is notified so that the image processing thread can determine that there is an image in the image cache space, obtain the image from the image cache space, and then perform image processing (such as Figure 6 The image processing is performed based on the pre-processing thread as shown in . Among them, the pre-processing thread is a type of image processing thread. ).

[0112] S503: Otherwise, wait until there is free space in the image cache space, and then store the acquired image in the free space.

[0113] In a possible implementation, after waiting until there is free space in the image cache space and the size of the free space is greater than or equal to the size of an image, the captured image is stored in the free space.

[0114] Based on the above technical solution, it is determined whether there is free space in the current image cache space. If there is free space, the captured image is stored in the free space. Otherwise, wait until there is free space in the image cache space, and then store the captured image in the free space. The image acquisition process of the above technical solution is responsible for storing the captured image in the image cache space, but is not responsible for inputting the captured image into the image processing thread, so as to achieve the effect that image acquisition and image processing do not affect each other.

[0115] As a possible embodiment of the present application, Figure 5 As shown, in the above S201, the process of inputting the acquired image into the image processing thread in the idle state among the M image processing threads can be specifically implemented by the following S504.

[0116] S504: Input the image stored in the image cache space and meeting the preset processing conditions into the image processing thread in an idle state.

[0117] The preset processing conditions include the image acquisition sequence and / or the image acquisition angle.

[0118] In one possible implementation, Figure 6 As shown, the image processing thread (eg, the pre-processing thread) determines whether there is an image in the current image cache space; if so, S504 is executed. If not, it waits until there is an image in the image cache space, and then S504 is executed.

[0119] Optionally, the implementation process of the image processing thread determining whether there is an image in the current image cache space may be as follows: after the acquired image is stored in the free space, the image processing thread can receive a notification, and the image processing thread determines whether there is an image in the current image cache space based on the information in the notification.

[0120] Optional, such as Figure 6 As shown, after an image that meets the preset processing conditions is input into an idle image processing thread, the idle image processing thread is controlled to sequentially run multiple image processing algorithms to process the image to obtain an image processing result. Then, the image processing result is further processed through other processing flows (such as the first processing).

[0121] Based on the above technical solution, the image stored in the image cache space that meets the preset processing conditions is input into the image processing thread in an idle state, and the image can be flexibly obtained from the image cache space to avoid thread blocking.

[0122] As a possible embodiment of the present application, Figure 2 ,like Figure 7 As shown, in the above S202, the process of controlling the image processing thread in an idle state to sequentially run multiple image processing algorithms to process the image and obtain the image processing result can be specifically implemented through the following S701-S702.

[0123] S701: When an image processing thread in an idle state has a competitor for processor resources, processor resources are acquired for the image processing thread in an idle state through locking processing and / or resource pool processing.

[0124] The processor resources are used to provide the image processing thread with the ability to run multiple image processing algorithms; the competing object is any image processing thread other than the image processing thread. Locking can ensure that only one image processing thread can access and use the processor resources at the same time. Resource pool processing can optimize the performance of processor resources and the utilization of resource pools by managing the recycling and use of processor resources. It can also control the number of processor resources in the resource pool to avoid system crashes caused by resource exhaustion or overuse.

[0125] In a possible implementation, when the maximum amount of resources that can be provided by processor resources (e.g., CPU resources, graphics processing unit (GPU) resources) is less than the amount of resources required for multiple image processing threads to run simultaneously, there will be competition for processor resources between multiple image processing threads, which makes it impossible to effectively allocate and use processor resources, and the resource utilization rate is low. In this case, in order to ensure the correct allocation and access of processor resources, the embodiment of the present application proposes that when an image processing thread in an idle state has a competitor for processor resources, the processor resources are obtained for the image processing thread in an idle state through locking processing and / or resource pool processing, so as to optimize the utilization rate of processor resources.

[0126] Optionally, the locking process may be to manage the processor resources through a preset lock mechanism (eg, a mutex lock, a read-write lock) so that the processor resources can be accessed and used by only one image processing thread at a time.

[0127] Optionally, the resource pool includes a thread pool, a database connection pool, a memory pool, etc. Resource pool processing can manage processor resources, optimize performance and improve resource utilization by pre-allocating and recycling processor resources. For example, when the resource pool is initialized, a certain number of processor resources are pre-allocated and placed in the resource pool. When the image processing thread needs to access the processor resource, an idle processor resource is obtained from the resource pool based on the resource pool processing method. If there is no idle processor resource in the resource pool, wait until there is an idle processor resource in the resource pool or perform other operations according to the preset policy.

[0128] Among them, the resource pool can also be called a resource operation pool, which is not limited in this application.

[0129] S702 : Based on processor resources, control the idle image processing thread to sequentially run multiple image processing algorithms to process the image and obtain an image processing result.

[0130] Optionally, the image processing thread processes the image based on the acquired processor resources, and after obtaining the image processing result, returns the processor resources to the resource pool, and then the resource pool can check and recycle the processor resources for subsequent use.

[0131] Based on the above technical solution, when an image processing thread in an idle state has a competitor for processor resources, processor resources are obtained for the image processing thread in an idle state (that is, the right to use the processor resources is obtained for the image processing thread in an idle state) through locking processing and / or resource pool processing, and it can also be ensured that the processor resources can only be used by the image processing thread in an idle state at the same time, and cannot be used by other threads, thereby ensuring that the image processing thread in an idle state can use processor resources to run multiple image processing algorithms to process the image and obtain image processing results.

[0132] As a possible embodiment of the present application, Figure 2 ,like Figure 8 As shown, after the above S202, the process of further processing the image processing result can be specifically implemented through the following S801.

[0133] S801: Perform a first process on an image processing result obtained through an image processing thread.

[0134] The first processing includes at least one of the following: image storage processing and image visualization processing. Of course, the above is only an exemplary description of the first processing, and the first processing may also include other image processing, such as image reconstruction processing and / or image post-processing, which is not limited in this application.

[0135] In a possible implementation, taking the first processing including image storage processing and image visualization processing as an example, the implementation process of S801 includes: storing the image processing result in a shared cache space; controlling the visualization thread to obtain the image processing result from the shared cache space, and controlling the visualization thread to perform visualization processing on the image processing result to display the image processing result.

[0136] The shared cache space may be a cache space determined according to the size and type of the image processing result. For example, for graphic data with a small amount of data, the shared cache space may be a thread-safe queue (e.g., Python's first-in-first-out queue (queue.Queue), last-in-first-out queue (queue.LifoQueue), priority queue (queue.PriorityQueue), double-stack queue (queue.deque)) or list. For image processing results with a large amount of data, the shared cache space may be a temporary file on a shared memory or disk, etc.

[0137] Exemplarily, the implementation process of controlling the visualization thread to obtain the image processing result from the shared cache space may include: after obtaining the image processing result, the image processing thread stores the image processing result in the shared cache space. The visualization thread may periodically obtain new image processing results from the shared cache space. For example, obtaining new image processing results from the shared cache space is achieved by polling.

[0138] Furthermore, the implementation process of controlling the visualization thread to perform visualization processing on the image processing results to display the image processing results may include: after obtaining the image processing results, controlling the visualization thread to pre-process the image (for example, decoding the image processing results, applying color mapping, or adjusting the display size), and then updating the visualization interface to display the image processing results.

[0139] In another possible implementation, taking the first processing including image storage processing, image reconstruction processing, image post-processing and image visualization processing as an example, the implementation process of S801 includes: storing the image processing result in the shared cache space; controlling the image reconstruction thread to obtain multiple image processing results from the shared cache space, and controlling the image reconstruction thread to reconstruct the multiple image processing results to obtain a three-dimensional image after reconstruction processing, and storing the three-dimensional image in the shared cache space.

[0140] Furthermore, the post-processing thread is controlled to obtain the three-dimensional image from the shared cache space, and the post-processing thread is controlled to perform post-processing (for example, contrast adjustment, smoothing, segmentation and annotation) on the three-dimensional image to obtain the three-dimensional image after post-processing; and then the three-dimensional image after post-processing is stored in the shared cache space.

[0141] Furthermore, the visualization thread is controlled to obtain the three-dimensional image after post-processing from the shared cache space, and the visualization thread is controlled to perform visualization processing on the three-dimensional image after post-processing to display the three-dimensional image after post-processing.

[0142] Compared with if all tasks (such as image processing tasks and visualization tasks) are performed in one thread, that is, after processing the image and obtaining the image processing results, the thread is also responsible for further visualization of the image. If the image processing takes a long time to complete, the thread will be completely occupied by the task and cannot handle other events (for example, it cannot handle user input, interface refresh, etc.), and the visualization interface is prone to problems such as interface blockage, unresponsiveness, and blockage.

[0143] In order to avoid the above problems, in the embodiment of the present application, the image processing thread and the visualization thread are separated by adding a shared cache space to the image processing thread and the visualization thread. After the image processing thread completes the processing, it stores the image processing result in the shared cache space. The visualization thread obtains the image processing result from the shared cache space and performs visualization processing. In this way, even if the image processing thread takes a long time to process the image, the visualization thread can respond to user input and other events, thereby maintaining the smoothness of the visualization interface.

[0144] The embodiment of the present application can divide the image processing device into functional modules or functional units according to the above method example. For example, each functional module or functional unit can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules or functional units. Among them, the division of modules or units in the embodiment of the present application is schematic, which is only a logical function division. There may be other division methods in actual implementation.

[0145] Optionally, the method shown in the embodiment of the present application can be applied to an image processing system. Fig. 9 This is a schematic diagram of the architecture of an image processing system provided in an embodiment of the present application. Fig. 9 As shown, the image processing system includes: a data processing server 901 and a data storage server 902.

[0146] The data processing server 901 and the data storage server 902 are connected via a communication link. The communication link may be a wired communication link or a wireless communication link, which is not limited in the present application.

[0147] The following is a detailed introduction to the data processing server 901.

[0148] In a possible implementation, the data processing server 901 is used to store the collected image to be processed in the image cache space in the data storage server 902. For example, the data processing server 901 is specifically used to determine whether there is free space in the image cache space in the data storage server 902. If there is, the data processing server 901 stores the image in the free space; otherwise, wait until there is free space in the image cache space in the data storage server 902, and then store the image in the free space.

[0149] In a possible implementation, the data processing server 901 includes an image processing thread and a visualization thread. The image processing thread determines whether there is an image to be processed in the image cache space in the data storage server 902. If so, the image processing thread obtains the image from the image cache space and processes it. After the processing is completed, the data processing server 901 is also used to control the image processing thread to store the image processing result in the shared cache space in the data storage server 902. Then, the data processing server 901 can continue to control the image processing thread to continue to obtain the next image from the image cache space and perform image processing.

[0150] Optionally, the data processing server 901 is further used to control the visualization thread to obtain the image processing result from the shared cache space in the data storage server 902 and perform visualization processing on the image.

[0151] It should be noted that the image processing thread and the visualization thread are two independent threads, and the two threads do not affect each other. For example, when the visualization thread visualizes the image processing results, it obtains the image processing results from the shared cache space and has no direct contact with the image processing thread. In other words, the image processing thread will not affect the visualization thread, for example, the image processing thread will not cause the visualization thread to be blocked.

[0152] Optionally, the data processing server 901 also includes an image reconstruction thread and an image post-processing thread. The data processing server 901 is also used to control the image reconstruction thread to obtain the image processing result from the image cache space and reconstruct the image processing result. The data processing server 901 is also used to control the image reconstruction thread to store the image processing result after the reconstruction process in the shared cache space.

[0153] The data processing server 901 is also used to control the image post-processing thread to obtain the image processing results after reconstruction processing from the shared cache space, and post-process the image processing results to obtain the image processing results after post-processing, and store the image processing results after post-processing in the shared cache space, so that the visualization thread can obtain the image processing results after post-processing from the shared cache space in the data storage server 902, and perform visualization on the image processing results after post-processing.

[0154] The data storage server 902 is described in detail below.

[0155] In a possible implementation, an image cache space and a shared cache space are provided in the data storage server 902. The image cache space is used to store acquired images, for example, images acquired from a rack. The shared cache space can be used to store image processing results processed by an image processing thread, image processing results processed by an image reconstruction thread, and / or image processing results processed by an image post-processing thread. The shared cache space is also used to provide image processing results for a visualization thread.

[0156] Optionally, the data storage server 902 may also be integrated into the data processing server 901, that is, the two are integrated into one server.

[0157] Among them, the server is an independent physical server, or a server cluster or distributed file system composed of multiple physical servers, or at least one of the cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. Of course, the above is only an exemplary introduction to the server. The server can also be a relational database management system (for example, MySQL, Oracle Database or Microsoft SQL Server) and / or a database based on distributed file storage (for example, MongoDB database, HBase database or Cassandra database), and this application does not impose any restrictions on this.

[0158] Optionally, the number of the above servers can be more or less, which is not limited in the embodiments of the present disclosure. Of course, the server can also include other functional servers to provide more comprehensive and diverse services.

[0159] like Fig.10 , which is a schematic diagram of the structure of an image processing device 100 provided in an embodiment of the present application, the image processing device 100 includes: a communication unit 1001 and a processing unit 1002 .

[0160] The processing unit 1002 is used to input the captured image into an idle image processing thread among M image processing threads, where the number M of image processing threads is determined according to the image capture time interval and the processing time of multiple image processing algorithms, and M is a positive integer; the processing unit 1002 is also used to control the idle image processing thread to sequentially run multiple image processing algorithms to process the image and obtain an image processing result.

[0161] In a possible implementation, the number M of image processing threads is the ratio of the sum of processing times of multiple image processing algorithms to the acquisition time interval.

[0162] In a possible implementation, when there are O image processing threads in an idle state among M image processing threads, O is a positive integer less than or equal to M, and the processing unit 1002 is specifically used to: corresponding to the number of images N being less than or equal to the number of image processing threads O in the idle state, input the N images into the N image processing threads among the O image processing threads respectively; N is a positive integer; corresponding to the number of images P being greater than the number of image processing threads O in the idle state, input O images out of the P images into the O image processing threads respectively; when there is an image processing thread among the M image processing threads that changes from a processing state to an idle state, input P0 images in sequence into the image processing thread in the idle state; P is a positive integer.

[0163] In a possible implementation, before inputting the captured image into an idle image processing thread among the M image processing threads, the processing unit 1002 is further used to: determine whether there is free space in the current image cache space, and if so, store the captured image in the free space; otherwise, wait until there is free space in the image cache space, and then store the captured image in the free space.

[0164] In one possible implementation, the processing unit 1002 is specifically used to: input an image stored in an image cache space that meets a preset processing condition into an idle image processing thread; wherein the preset processing condition includes an image acquisition order and / or an image acquisition angle.

[0165] In one possible implementation, the processing unit 1002 is specifically used to: when an image processing thread in an idle state has a competitor for processor resources, obtain processor resources for the image processing thread in an idle state through locking processing and / or resource pool processing; the processor resources are used to provide the image processing thread with the ability to run multiple image processing algorithms; the competitor is any image processing thread other than the image processing thread; based on the processor resources, control the image processing thread in an idle state to run multiple image processing algorithms in sequence, process the image, and obtain an image processing result.

[0166] In a possible implementation, the processing unit 1002 is further used to: perform a first process on the image processing result obtained by the image processing thread; the first process includes at least one of the following: image storage processing and image visualization processing.

[0167] In one possible implementation, the processing unit 1002 is further used to: store the image processing results in a shared cache space through the communication unit 1001; control the visualization thread to obtain the image processing results from the shared cache space through the communication unit 1001, and control the visualization thread to perform visualization processing on the image processing results to display the image processing results.

[0168] In a possible implementation, the image processing apparatus 100 may further include a storage unit 1003 ( Fig.10 The storage unit 1003 stores a program or instruction. When the processing unit 1002 executes the program or instruction, the image processing device 100 can execute the image processing method described in the above method embodiment.

[0169] When implemented by hardware, the embodiment of the present application further provides an electronic device, which integrates an image processing device 100 to execute the image processing method described in the above method embodiment.

[0170] Specifically, Fig.11 The hardware structure diagram of an electronic device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the electronic device includes at least one processor 1101, a communication line 1102, and at least one communication interface 1104, and may also include a memory 1103 configured to store instructions executable by the processor. The processor 1101, the memory 1103, and the communication interface 1104 may be connected via the communication line 1102, and the processor is configured to execute instructions.

[0171] Processor 1101 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as: one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA).

[0172] The communication link 1102 may include a path for transmitting information between the above-mentioned components.

[0173] The communication interface 1104 is used to communicate with other devices or communication networks, and can use any transceiver-like device, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0174] The memory 1103 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to include or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0175] In one possible design, the memory 1103 can exist independently of the processor 1101, that is, the memory 1103 can be a memory outside the processor 1101. In this case, the memory 1103 can be connected to the processor 1101 through the communication line 1102 to store execution instructions or application code, and the processor 1101 controls the execution to implement the image processing method provided in the above-mentioned embodiment of the present application. In another possible design, the memory 1103 can also be integrated with the processor 1101, that is, the memory 1103 can be the internal memory of the processor 1101. For example, the memory 1103 is a high-speed cache that can be used to temporarily store some data and instruction information.

[0176] As a possible implementation, the processor 1101 may include one or more CPUs, for example Fig.11 As another possible implementation, the electronic device may include multiple processors, such as Fig.11 The processor 1101 and the processor 1107 in the electronic device. As another possible implementation, the electronic device may further include: an output device 1105, such as various types of displays, speakers, etc.; an input device 1106, such as a keyboard, a mouse, etc.

[0177] In some embodiments, the internal processing logic of the processor supports processing data from multiple image processing threads, and / or the processing logic of multiple image processing threads is the same, and the processor is one processor; one processor is specifically used to run multiple image processing threads.

[0178] It should be noted that if the processor can support being used by multiple threads without causing data competition inconsistencies, and / or the processing logic of multiple image processing threads is the same, then in an embodiment of the present application, one processor can be used to run multiple image processing threads to achieve the purpose of reducing resource consumption (because there is no need to assign an independent processor to each thread).

[0179] In other embodiments, when the processing logics corresponding to the multiple image processing threads are different, the processor is a plurality of processors, and each processor is specifically used to run one image processing thread.

[0180] It should be noted that if the processing logic of each image is different, an independent processor can be assigned to each image processing thread in the embodiment of the present application. In this way, although the resource consumption is high, the uniqueness of the processing logic and parameters can be ensured.

[0181] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0182] An embodiment of the present application provides a computer program product including instructions. When the computer program product is run on a computer, the computer is enabled to execute the image processing method in the above method embodiment.

[0183] An embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the image processing method in the method flow shown in the above method embodiment.

[0184] Among them, the computer readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a register, a hard disk, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, or any other form of computer readable storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application-specific integrated circuit (ASIC). In the embodiments of the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0185] Since the image processing device, electronic device, computer-readable storage medium, and computer program product in the embodiments of the present application can be applied to the above-mentioned method, the technical effects that can be obtained can also refer to the above-mentioned method embodiments, and the embodiments of the present application will not be repeated here.

[0186] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0187] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of this disclosure can be achieved, and this document is not limited here.

[0188] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

[0189] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0190] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0191] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Input the acquired image into an image processing thread in an idle state among M image processing threads, where the number M of the image processing threads is determined according to the image acquisition time interval and the processing time of multiple image processing algorithms, and M is a positive integer; The image processing thread in the idle state is controlled to run the multiple image processing algorithms in sequence to process the image and obtain an image processing result.

2. The image processing method according to claim 1, characterized in that: The number M of the image processing threads is the ratio of the sum of the processing times of the multiple image processing algorithms to the acquisition time interval.

3. The image processing method according to claim 1, characterized in that: In the case where there are O image processing threads in the idle state among the M image processing threads, O is a positive integer less than or equal to M, the step of inputting the acquired image into the image processing thread in the idle state among the M image processing threads comprises: Corresponding to the number N of images being less than or equal to the number O of image processing threads in an idle state, the N images are respectively input into N image processing threads among the O image processing threads; N is a positive integer; Corresponding to the fact that the number P of the images is greater than the number O of the image processing threads in the idle state, O images among the P images are respectively input into the O image processing threads; when there is an image processing thread that changes from a processing state to an idle state among the M image processing threads, P0 images are sequentially input into the image processing threads in the idle state; P is a positive integer.

4. The image processing method according to claim 1, characterized in that: Before inputting the acquired image into an image processing thread in an idle state among the M image processing threads, the method further includes a step of caching the image into an image cache space, wherein the step of caching the image into the image cache space includes: Determine whether there is free space in the current image cache space. If so, store the captured image in the free space; otherwise, wait until there is free space in the image cache space and store the captured image in the free space.

5. The image processing method according to claim 4, characterized in that: The step of inputting the acquired image into an idle image processing thread among the M image processing threads comprises: Inputting the image stored in the image cache space and meeting the preset processing condition into the image processing thread in the idle state; The preset processing conditions include the image acquisition sequence and / or the image acquisition angle.

6. The image processing method according to claim 1, characterized in that: The controlling the idle image processing thread to sequentially execute the plurality of image processing algorithms to process the image to obtain an image processing result includes: In the case that the image processing thread in the idle state has a competitor for processor resources, the processor resources are acquired for the image processing thread in the idle state through locking processing and / or resource pool processing; the processor resources are used to provide the image processing thread with the ability to run the multiple image processing algorithms; the competitor is any image processing thread other than the image processing thread; Based on the processor resources, the image processing thread in an idle state is controlled to sequentially run the multiple image processing algorithms to process the image and obtain the image processing result.

7. The image processing method according to claim 1, characterized in that: The method further comprises: A first process is performed on the image processing result obtained by the image processing thread; the first process includes at least one of the following: image storage process and image visualization process.

8. The method according to claim 7, characterized in that The performing a first processing on the image processing result obtained by the image processing thread includes: Storing the image processing result in a shared cache space; The visualization thread is controlled to obtain the image processing result from the shared cache space, and the visualization thread is controlled to perform visualization processing on the image processing result to display the image processing result.

9. An electronic device, characterized in that: include: processor; a memory configured to store instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 8.

10. The electronic device according to claim 9, characterized in that: The internal processing logic of the processor supports processing data from multiple image processing threads, and / or the processing logic of the multiple image processing threads is the same, and the processor is one processor; the one processor is specifically used to run the multiple image processing threads; When the processing logics corresponding to the multiple image processing threads are different, the processor includes multiple processors, and each processor is specifically used to run one image processing thread.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions. When a computer executes the instructions, the computer executes the image processing method according to any one of claims 1 to 8.

Images