Medical image processing method and device, equipment, medium and program product

Abstract

The invention relates to the technical field of image processing, and discloses a medical image processing method, device and equipment, a medium and a program product, and the method comprises the steps: obtaining first image data for a to-be-detected object through first image collection equipment; acquiring second image data for the to-be-detected object through second image acquisition equipment, wherein the resolution of the second image data is smaller than that of the first image data; performing identification processing on the second image data to obtain tissue identification results used for representing a plurality of medical tissues of the to-be-detected object; determining structure parameters and color parameters of different medical tissues of the to-be-detected object for the set medical event according to the tissue recognition result; and performing image processing on the first image data according to the structure parameter and the color parameter to obtain a medical detection image of the to-be-detected object. And on the basis of effectively reducing the occupation of storage resources and computing resources, the image self-adaptive enhancement processing is realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a medical image processing method, device, equipment, medium and program product. Background Art

[0002] In the related art, the endoscope system mostly uses the image video data stream collected by a single image sensor, transmits the collected video source to the FPGA (Field Programmable Gate Array) logic chip through the image sensor, and uses the FPGA to process the image video data stream. Furthermore, the processed image video data stream is stored in DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory) through the pipeline image processing module, and then the image is displayed. The ISP (Image Signal Processing) module of the conventional endoscope system can manually set general parameters, or different parameters can be set in advance by professionals according to different medical application scenarios.

[0003] However, different structural enhancement and color enhancement processes are required for different target tissues. For example, laparoscopy requires different structural enhancement and color enhancement processes for different structures such as spleen, stomach and kidney. Therefore, conventional endoscope systems cannot autonomously set specific parameters for different application scenarios to optimize the displayed image, and there is a problem of non-adaptability. Summary of the invention

[0004] In view of this, the present invention provides a medical image processing method, apparatus, device, medium and program product to solve the problem that conventional endoscope systems cannot autonomously set specific parameters for different application scenarios to optimize the displayed image and are unable to adapt themselves.

[0005] In a first aspect, the present invention provides a medical image processing method, which comprises: acquiring first image data for an object to be detected by a first image acquisition device; acquiring second image data for the object to be detected by a second image acquisition device, wherein the resolution of the second image data is smaller than that of the first image data; performing recognition processing on the second image data to obtain tissue recognition results for characterizing multiple medical tissues of the object to be detected; determining structural parameters and color parameters of different medical tissues of the object to be detected for a set medical event based on the tissue recognition results; and performing image processing on the first image data based on the structural parameters and color parameters to obtain a medical detection image of the object to be detected.

[0006] The medical image processing method provided by the present invention obtains first image data and second image data with different resolutions through two image acquisition devices, obtains tissue recognition results for multiple medical tissues of the object to be detected based on the second image data with lower resolution, and determines structural parameters and color parameters according to the recognition results. Therefore, according to the determined parameters, the first image data with higher resolution is processed to obtain a medical detection image of the object to be detected. Therefore, it is only necessary to configure an image acquisition device capable of acquiring an image with lower resolution requirements, so as to adaptively determine the structural parameters and color parameters for enhancing the image data of the object to be detected, thereby effectively avoiding image processing such as identification of multiple medical tissues on the image data with higher resolution, and realizing image adaptive enhancement processing on the basis of effectively reducing the storage resources and computing resources occupied.

[0007] In an optional embodiment, the first image data is processed according to the structural parameters and the color parameters to obtain a medical detection image of the object to be detected, including: preprocessing the first image data to obtain the image to be enhanced; performing structural enhancement using corresponding structural parameters for different medical tissue structures in the image to be enhanced; and performing color enhancement using corresponding color parameters for different medical tissue structures in the image to be enhanced.

[0008] The medical image processing method provided by the present invention can perform color enhancement and structural enhancement on the image to be enhanced obtained by preprocessing after simple preprocessing of the first image data, using the structural parameters and color parameters adaptively determined according to the second image data. This effectively avoids image processing such as identification of multiple medical tissues on image data with a high resolution, and directly increases the color distinction based on the structural parameters and color parameters, making the contours of different medical tissues clear, and quickly and effectively realizing the enhancement processing of the image.

[0009] In an optional embodiment, the second image data is preprocessed to obtain the image to be enhanced, including: decoding the second image data to obtain original image data; de-mosaicing the original image data to obtain three-color image data; and gamma converting the three-color image data to obtain the image to be enhanced.

[0010] The medical image processing method provided by the present invention performs decoding processing, de-mosaic processing and gamma conversion on the second image data to obtain the image to be enhanced. Therefore, the image data collected from the image acquisition device is firstly decoded and de-mosaiced to obtain image data with a format that meets the image enhancement processing requirements, and then the image is gamma converted to amplify the dark details in the image data, effectively avoiding the problem that the image data collected by the image acquisition device does not meet the contrast requirements of the human eye due to the low sensitivity of the human eye to the brightness changes of the dark image.

[0011] In an optional embodiment, the second image data is a low-voltage differential signal; decoding the second image data includes: decoding the low-voltage differential signal to obtain original image data; de-mosaicing the original image data includes: converting the original image data into three-color image data.

[0012] The medical image processing method provided by the present invention decodes the low-voltage differential signal directly acquired by the image acquisition device, and converts the decoded original image data into three-color image data, so as to facilitate subsequent operations such as image recognition on the image data.

[0013] In an optional embodiment, the second image data is processed for identification to obtain tissue identification results for multiple medical tissues of the object to be detected, including: obtaining the number of medical tissues to be identified; performing convolution processing on the second image data according to the number of medical tissues to be identified to obtain convolution image data; performing pooling compression processing on the convolution image data to obtain compressed data; and performing full connection and activation processing on the compressed data to obtain tissue identification results.

[0014] The medical image processing method provided by the present invention performs neural network algorithm operations such as convolution, compression, full connection and activation processing based on the number of medical tissues to be identified, effectively identifies the second image data, and quickly obtains accurate tissue identification results.

[0015] In a second aspect, the present invention provides a medical image processing device, comprising: a first image acquisition device, configured to acquire first image data of an object to be detected; a second image acquisition device, configured to acquire second image data of the object to be detected, the resolution of the second image data being smaller than that of the first image data; a processing system, comprising: a tissue identification system, configured to perform identification processing on the second image data to obtain tissue identification results for characterizing multiple medical tissues of the object to be detected; a parameter determination unit, configured to determine structural parameters and color parameters of different medical tissues of the object to be detected for a set medical event based on the tissue identification results; and a processing unit, configured to perform image processing on the first image data based on the structural parameters and color parameters to obtain a medical detection image of the object to be detected.

[0016] In an optional embodiment, the processing system also includes: a first sending unit, configured to send the medical detection image to a storage unit; and a second sending unit, configured to send the medical detection image to a display device; the device also includes: a storage module, configured to receive and store the medical detection image sent by the processing system; and a display module, configured to receive and display the medical detection image sent by the processing system.

[0017] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the medical image processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the medical image processing method of the first aspect or any corresponding embodiment thereof.

[0019] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the medical image processing method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a schematic flow chart of a medical image processing method according to an embodiment of the present invention;

[0022] Figure 2 is a flowchart of another medical image processing method according to an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of the working principle of a medical image processing device with a dual image acquisition device provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of an implementation flow of a specific application example of the medical image processing method provided by an embodiment of the present invention is shown;

[0025] Figure 5 A schematic diagram of the implementation process of medical tissue recognition by a tissue recognition system in a specific application example of the medical image processing method provided by an embodiment of the present invention is shown;

[0026] Figure 6 A schematic diagram of the implementation flow of image processing by an image processing system in a specific application example of the medical image processing method provided by an embodiment of the present invention is shown;

[0027] Figure 7is a structural block diagram of a medical image processing device according to an embodiment of the present invention;

[0028] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0030] In order to better illustrate the medical image processing method of the present invention, the application scenario of the present invention is first briefly described here.

[0031] First, the medical image processing method of the present invention can be applied to medical instruments such as endoscopes. The present invention improves the medical instruments used. Specifically, an image acquisition device such as a camera can be added. For example, a first image sensor, a second image sensor, and a light source are configured on the insertion portion of the endoscope to collect image data. Taking into account factors such as the cost of the endoscope, the storage resources occupied by the collected images, and the computing resources occupied by processing the collected image data, the first image sensor and the second image sensor can use two image sensors with different parameters. For example, the first image sensor is a camera with a resolution of 1920×1080, and the second image sensor is a camera with a resolution of 64×64.

[0032] Based on the above application scenario, an embodiment of the present invention provides an embodiment of a medical image processing method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here. In addition, the above application scenario is only an exemplary description and does not limit the technical solution of the present invention.

[0033] In this embodiment, a medical image processing method is provided, which can be used for the above-mentioned medical instruments such as endoscopes. Figure 1 is a flowchart of a medical image processing method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0034] Step S101: acquiring first image data of an object to be detected by a first image acquisition device.

[0035] In the embodiment of the present invention, the first image acquisition device may be a camera configured in a medical instrument such as an endoscope, and its resolution may be 1920×1080 or other suitable resolution parameters.

[0036] The first image acquisition device can acquire multiple frames of images of the object to be detected, and the multiple frames of images form the first image data. The object to be detected may be a patient or a person undergoing a medical examination.

[0037] It should be noted that the image acquisition and image processing operations here and in other articles involving the objects to be detected are implemented in compliance with relevant laws and regulations.

[0038] Step S102: acquiring second image data of the object to be detected by a second image acquisition device, wherein the resolution of the second image data is smaller than that of the first image data.

[0039] In the embodiment of the present invention, the second image acquisition device may be a camera configured in a medical instrument such as an endoscope, and its resolution may be a suitable resolution parameter such as 32×32, 64×64 or 128×128.

[0040] Likewise, multiple frames of images of the object to be detected can be acquired through the second image acquisition device, and the multiple frames of images form second image data.

[0041] Step S103: performing recognition processing on the second image data to obtain tissue recognition results for characterizing multiple medical tissues of the object to be detected.

[0042] In the embodiment of the present invention, the resolution of the second image data is smaller than that of the first image data, but the resolution of the second image is sufficient to identify multiple medical tissues of the subject to be detected, such as liver, gallbladder, spleen, lung, kidney, gallbladder, heart and thyroid gland, according to the second image data. The process of identifying and processing the second image data can adopt the image recognition technology commonly used in the art, and the present invention does not specifically limit this.

[0043] It should be noted here that the first image acquisition device and the second image acquisition device can perform image acquisition for the same position of the same object to be detected, so that the first image data of the object to be detected acquired by the first acquisition device and the second image data of the object to be detected acquired by the second acquisition device are for the same detection part of the same object to be detected. In this way, the second image data and the first image data have a certain corresponding relationship. In the process of processing the image data, the corresponding relationship between the two image data is effectively utilized, and the image data is processed quickly and accurately based on the two image data, so as to realize efficient processing of the image data and significantly improve the accuracy of medical tissue recognition.

[0044] In one embodiment of the present invention, based on the number of medical tissues that may be involved in the set detection site of the object to be detected, neural network algorithm processing such as image convolution and pooling can be performed to obtain tissue recognition results that can characterize multiple medical tissues of the object to be detected.

[0045] Step S104, according to the tissue recognition result, determining the structural parameters and color parameters of different medical tissues of the object to be detected for the set medical event.

[0046] In this embodiment of the present invention, different structural enhancement parameters and color parameters can be pre-configured for different medical tissue objects based on different medical events. For example, when an endoscope used for laparoscopy is used to examine different medical events such as suspected ectopic pregnancy, suspected ovarian cyst, suspected appendicitis, suspected cholecystitis, etc., the color enhancement parameters and structural enhancement parameters for the same human tissue can be different. Multiple sets of structural parameters and color parameters can be pre-configured to enhance different medical tissues of the object to be detected for different medical events. Thus, the structure and / or color of the medical tissue that is of greater concern to the set medical event is enhanced, which effectively realizes the adaptive adjustment of parameters and fully guarantees the image processing effect.

[0047] Step S105, performing image processing on the first image data according to the structural parameters and the color parameters to obtain a medical detection image of the object to be detected.

[0048] In this embodiment of the present invention, the structural contour of the set medical tissue can be enhanced according to the structural parameters, so as to better distinguish different tissue structures. Furthermore, the color display of the set medical tissue can be enhanced according to the color parameters to more clearly display the set medical tissue, for example, to make medical tissues such as blood vessels more clearly displayed.

[0049] The medical image processing method provided by the present invention obtains first image data and second image data with different resolutions through two image acquisition devices, obtains tissue recognition results for multiple medical tissues of the object to be detected based on the second image data with lower resolution, and determines structural parameters and color parameters according to the recognition results. Therefore, according to the determined parameters, the first image data with higher resolution is processed to obtain a medical detection image of the object to be detected. Therefore, it is only necessary to configure an image acquisition device capable of acquiring an image with lower resolution requirements, so as to adaptively determine the structural parameters and color parameters for enhancing the image data of the object to be detected, thereby effectively avoiding image processing such as identification of multiple medical tissues on the image data with higher resolution, and realizing image adaptive enhancement processing on the basis of effectively reducing the storage resources and computing resources occupied.

[0050] In this embodiment, a medical image processing method is provided, which can be used for medical instruments such as endoscopes. Figure 2 is a flowchart of a medical image processing method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0051] Step S201: Acquire first image data of the object to be detected by a first image acquisition device. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0052] Step S202: Acquire second image data of the object to be detected by a second image acquisition device, wherein the resolution of the second image data is smaller than that of the first image data. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0053] Step S203: performing recognition processing on the second image data to obtain tissue recognition results for characterizing multiple medical tissues of the object to be detected.

[0054] Specifically, the above step S203 may include:

[0055] Step S2031, obtaining the number of medical tissues to be identified.

[0056] The number of medical tissues to be identified can be set according to different medical events. For example, it can be set according to the examination items that need to be performed by medical instruments such as endoscopes. For example, in laparoscopic examination, the correct tissue structure needs to be identified from 10 medical tissues to be identified, including liver, gallbladder, spleen, pancreas, kidney, ovary, etc.

[0057] Step S2032: performing convolution processing on the second image data according to the number of medical tissues to be identified to obtain convolution image data.

[0058] In this embodiment of the present invention, the second image data may be an image input with a resolution of 64×64. Step S203 is an image recognition process, and a set neural network algorithm may be used for image recognition. Here, the input layer of the algorithm is determined to be (64, 64, 3), where the values ​​in (64, 64, 3) represent the resolution x, the resolution y, and the three color channel values, i.e., the three channel inputs of the second image data R, G, and B.

[0059] In this embodiment of the present invention, the convolution layer can perform convolution processing on the input data based on the second image data (64, 64, 3) of the input layer according to the pre-trained convolution kernel to obtain (62, 62, N) image data. The N in the number of channels is the number of medical tissues to be identified, and the number of channels N can be set according to actual needs. For example, if the correct medical tissue needs to be identified from 10 types of medical tissues such as kidneys and gallbladders, the number of channels N is 10. The specific convolution processing process can adopt the convolution processing process commonly used in this field. Here, taking N=10 as an example, the (62, 62, 10) image data can be convolved again to obtain the convolution image data of (60, 60, 10) as the input of the pooling compression in step S2033.

[0060] Step S2033, performing pooling compression processing on the convolution image data to obtain compressed data.

[0061] In this embodiment of the present invention, the convolution image data (60, 60, 10) obtained by the convolution layer can be compressed by data pooling to obtain compressed data (30, 30, 10).

[0062] In this embodiment of the present invention, step S2032 and step S2033 may be repeated multiple times to obtain compressed data of (13, 13, 10).

[0063] Step S2034, perform full connection and activation processing on the compressed data to obtain the tissue identification result.

[0064] Here, the compressed data is fully connected, that is, the compressed data is converted into a one-dimensional array. Specifically, the input compressed data (13, 13, 10) can be converted into a one-dimensional array.

[0065] Furthermore, S2032 to S2034 is a neural network algorithm that can pre-train the weight ratio weight and skew parameter bias of the data, and further process the one-dimensional array obtained by the full connection according to the pre-trained weight ratio weight and skew parameter bias to fully consider the importance of multiple pixels.

[0066] Specifically, calculations are performed for each of the 10 channels, and a one-dimensional array y with a size of 1690 is obtained for each channel.

[0067] y=weight×pixel+bias, where y represents the one-dimensional array of the channel corresponding to a certain medical tissue in the fully connected layer, weight represents the weight ratio of the medical tissue, pixel represents the number of pixels of the medical tissue, and bias represents the bias parameter of the medical tissue. There are 10 channels in total, and here, y1, y2...y10 will be obtained.

[0068] Furthermore, the activation function can be used to find the expected value of a certain channel to obtain the probability that the image data of the channel corresponds to a certain medical tissue of the object to be detected. Specifically, the expectation of each medical tissue can be found according to the selected activation function. For example, if y1 corresponds to gallbladder tissue, then the probability that the image data of the channel corresponds to the gallbladder tissue of the object to be detected is Among them, exp(y1) represents the expected value of y1, It represents the sum of the expected values ​​of gallbladder tissue in each of the 10 channels. i As an output result, a tissue recognition result is obtained, and the tissue output result can characterize the medical tissue corresponding to the image data of a certain channel.

[0069] The medical image processing method provided by the present invention performs neural network algorithm operations such as convolution, compression, full connection and activation processing based on the number of medical tissues to be identified, effectively identifies the second image data, and quickly obtains accurate tissue identification results.

[0070] Step S204: According to the tissue recognition result, the structural parameters and color parameters of different medical tissues of the object to be detected are determined for the set medical event. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0071] Step S205: performing image processing on the first image data according to the structural parameters and the color parameters to obtain a medical detection image of the object to be detected.

[0072] In an optional implementation, the above step S205 may include:

[0073] Step S2051, preprocessing the first image data to obtain an image to be enhanced.

[0074] In an optional implementation, step S2051 may include:

[0075] Step a1, decoding the second image data to obtain original image data.

[0076] Here, the second image data may be image data directly acquired by a second image acquisition device and transmitted via LVDS (Low-Voltage Differential Signaling). The second image data is decoded, i.e., LVDS is decoded on the second image data to obtain original image data. The original image data may be picture data in RAW format.

[0077] Step a2, de-mosaicing the original image data to obtain three-color image data. In this embodiment of the present invention, the decoded original image data can be picture data in RAW format, which needs to be de-mosaiced and converted into image data in RGB format (three-color image data).

[0078] The medical image processing method provided by the present invention decodes the low-voltage differential signal directly acquired by the image acquisition device, and converts the decoded original image data into three-color image data, so as to facilitate subsequent operations such as image recognition on the image data.

[0079] Step a3, performing gamma conversion on the three-color image data to obtain an image to be enhanced.

[0080] Here, the image data in RGB format is gamma-converted to obtain the image to be enhanced, in order to amplify the dark details and increase the contrast of the image.

[0081] The medical image processing method provided by the present invention performs decoding processing, de-mosaic processing and gamma conversion on the second image data to obtain the image to be enhanced. Therefore, the image data collected from the image acquisition device is firstly decoded and de-mosaiced to obtain image data with a format that meets the image enhancement processing requirements, and then the image is gamma converted to amplify the dark details in the image data, effectively avoiding the problem that the image data collected by the image acquisition device does not meet the contrast requirements of the human eye due to the low sensitivity of the human eye to the brightness changes of the dark image.

[0082] S2052: For different medical tissue structures in the image to be enhanced, corresponding structural parameters are used to perform structural enhancement.

[0083] S2053, performing color enhancement using corresponding color parameters for different medical tissue structures in the image to be enhanced.

[0084] For details of the above steps S2051 and S2053, please refer to Figure 1 Step S205 of the illustrated embodiment will not be described in detail here.

[0085] The medical image processing method provided by the present invention can perform color enhancement and structural enhancement on the image to be enhanced obtained by preprocessing after simple preprocessing of the first image data, using the structural parameters and color parameters adaptively determined according to the second image data. This effectively avoids image processing such as identification of multiple medical tissues on image data with a high resolution, and directly increases the color distinction based on the structural parameters and color parameters, making the contours of different medical tissues clear, and quickly and effectively realizing the enhancement processing of the image.

[0086] Step S206: sending the medical detection image to a storage unit for storage.

[0087] Step S207: sending the medical detection image to a display device for display.

[0088] The medical image processing method provided by the present invention sends the medical detection image to a storage unit for storage, and sends it to a display device for display, thereby effectively outputting the final medical detection image and storing it in time, making the subsequent use of the medical detection image more convenient.

[0089] In this embodiment, a specific application example of a medical image processing method is provided, which can be applied to an endoscope with a dual image sensor. Figure 3 1 is a schematic diagram of the working principle of a medical image processing device with dual image acquisition devices provided by an embodiment of the present invention. The insertion portion 100 is the part of the endoscope that is inserted into the object to be detected, and has a light source and two image acquisition devices, such as an image sensor ( Figure 3 In the figure, the first image acquisition device is shown as the first image sensor 101, and the second image acquisition device is shown as the second image sensor 102. The first image sensor 101 and the second image sensor 102 are integrated into the handheld part of the dual-image sensor endoscope). The first image acquisition device is an image sensor with a resolution of 1920×1080, and the second image acquisition device is an image sensor with a resolution of 64×64, such as a camera. The image data acquired by the first image acquisition device is sent to the image processing system 200, and the image data acquired by the second image acquisition device is sent to the tissue recognition system 300. The tissue recognition system 300 inputs its image recognition result into the image processing system 200. The image processing system 200 performs image processing based on the image data acquired by the first image acquisition device and the recognition result of the tissue recognition system 300, and outputs the image processing result to the display device 400 for display. The display device 400 may be a medical display. Here, the image processing system 200 and the tissue recognition system 300 can jointly implement the following Figure 7 The functions of the processing system in the medical image processing device shown.

[0090] Figure 4 A schematic diagram of an implementation flow of a specific application example of the medical image processing method provided by an embodiment of the present invention is shown.

[0091] refer to Figure 4 The specific application example of the medical image processing method provided by the embodiment of the present invention can be based on Figure 3 The working principle of the medical image processing device with a dual image acquisition device is shown.

[0092] Specifically, a specific application example of the medical image processing method provided by the embodiment of the present invention may include the following process:

[0093] 1. The medical image processing device with dual image acquisition devices can be a dual-sensor endoscope, which performs image acquisition operations through the first image sensor integrated in the handheld part of the endoscope to obtain first image data with a resolution of 1920×1080. The image acquisition operation is performed through the second image sensor integrated in the handheld part of the endoscope to obtain second image data with a resolution of 64×64. The first image sensor and the second image sensor both acquire multi-frame images, so the first image data and the second image data here can be regarded as video stream data, which are transmitted to the tissue recognition system and the image processing system respectively through the video stream data link.

[0094] 2. The tissue recognition system determines the structural parameters and color parameters required for image enhancement of the first image data acquired by the first acquisition device through the output layer, the convolution layer, the planning layer, the fully connected layer, the tissue recognition and the parameter selection operations, and sends the determined structural parameters and color parameters to the image processing system.

[0095] Specifically, Figure 5 FIG. 2 is a schematic diagram showing a process flow of implementing medical tissue recognition by a tissue recognition system in a specific application example of the medical image processing method provided by an embodiment of the present invention. Figure 5 As shown, in the specific application example of the medical image processing method provided by the embodiment of the present invention, the tissue recognition system performs medical tissue recognition, which may include the following operation steps:

[0096] Step b1, receiving image data with a resolution of 64×64 transmitted by a second video data link.

[0097] Image data with a resolution of 64×64 transmitted by the second video data link is received as input layer data (64,64,3), where the respective values ​​of (64,64,3) represent (resolution x, resolution y, number of channels), respectively, where the number of channels refers to the values ​​of three channels R, G and B in the image data in RGB format obtained after preprocessing the data collected by the second image sensor.

[0098] Step b2: the image convolution layer performs image convolution.

[0099] The convolution layer can perform convolution processing on the input data based on the second image data (64,64,3) of the input layer according to the pre-trained convolution kernel to obtain (62,62,N) image data. The N in the number of channels is the number of medical tissues to be identified, and the number of channels N can be set according to actual needs. For example, if the correct medical tissue needs to be identified from 10 types of medical tissues such as kidneys and gallbladders, the number of channels N is 10. The specific convolution processing process can adopt the convolution processing process commonly used in this field. Here, taking N=10 as an example, the (62,62,10) image data can be convolved again to obtain the (60,60,10) convolution image data as the input of the pooling layer.

[0100] Step b3, the pooling layer compresses the data.

[0101] Specifically, the pooling layer performs data pooling compression on the (60, 60, 10) convolution image data obtained by the convolution layer to obtain (30, 30, 10) compressed data.

[0102] Repeat steps b2 and b3 multiple times to obtain the compressed data of (13, 13, 10).

[0103] Step b4, convert the array into a one-dimensional array.

[0104] Specifically, the fully connected layer performs full connection on the compressed data, that is, converts the compressed data into a one-dimensional array. Specifically, the input compressed data (13, 13, 10) can be converted into a one-dimensional array.

[0105] Step b5, calculate the expected value according to the activation function and output the tissue recognition result.

[0106] Specifically, in steps b1 to b5, the tissue recognition system executes a neural network algorithm, which can pre-train the weight ratio weight and the skew parameter bias of the data, and further process the one-dimensional array obtained by full connection according to the pre-trained weight ratio weight and the skew parameter bias to fully consider the importance of multiple pixels.

[0107] Specifically, calculations are performed for each of the 10 channels, and a one-dimensional array y with a size of 1690 is obtained for each channel.

[0108] y=weight×pixel+bias, where y represents the one-dimensional array of the channel corresponding to a certain medical tissue in the fully connected layer, weight represents the weight ratio of the medical tissue, pixel represents the number of pixels of the medical tissue, and bias represents the bias parameter of the medical tissue. There are 10 channels in total, and here, y1, y2...y10 will be obtained.

[0109] Furthermore, the activation function can be used to find the expected value of a certain channel to obtain the probability that the image data of the channel corresponds to a certain medical tissue of the object to be detected. Specifically, the expectation of each medical tissue can be found according to the selected activation function. For example, if y1 corresponds to gallbladder tissue, then the probability that the image data of the channel corresponds to the gallbladder tissue of the object to be detected is Among them, exp(y1) represents the expected value of y1, It represents the sum of the expected values ​​of gallbladder tissue in each of the 10 channels. i As an output result, a tissue recognition result is obtained, and the tissue output result can characterize the medical tissue corresponding to the image data of a certain channel.

[0110] Step b6, selecting corresponding algorithm parameters according to the tissue recognition result.

[0111] Specifically, here, according to the tissue recognition result, the structure enhancement parameters (same as structure parameters) and color enhancement parameters (same as color parameters) of different medical tissues of the object to be detected can be determined for the set medical event.

[0112] Step b7: the parameters are transmitted back to the image processing system.

[0113] Specifically, the color enhancement parameters and the structure enhancement parameters may be sent to an image processing system.

[0114] 3. The image processing system decodes the video stream data, performs de-mosaicing and gamma conversion on the first image data acquired by the first image acquisition device, and obtains an image by gamma conversion according to the graph.

[0115] Specifically, Figure 6 FIG. 2 shows a schematic diagram of the implementation flow of image processing by an image processing system in a specific application example of the medical image processing method provided by an embodiment of the present invention. Figure 6 , which may include the following processes:

[0116] Step c1, receiving image data with a resolution of 1920×2080 transmitted by the second video data link, and decoding to obtain image data in RAW format.

[0117] Specifically, image data with a resolution of 1920×2080 collected by a first image sensor and transmitted via LVDS (Low-Voltage Differential Signaling) is received, and video stream time decoding processing is performed on the image data, where the decoded image data may be image data in RAW format.

[0118] Step c2, performing a demosaicing image processing on the image data in RAW format, and converting the image data in RAW format into image data in RGB format. Specifically, the image data obtained by decoding is image data in RAW format, and the demosaicing processing is required to convert the image data in RAW format into image data in RGB format.

[0119] Step c3, gamma conversion is performed on the image data in RGB format to obtain the image to be enhanced. Here, gamma conversion is performed on the image data in RGB format to obtain the image to be enhanced. The purpose is to amplify the dark details and increase the contrast of the image.

[0120] Step c4, receiving structure enhancement and color enhancement parameters from the tissue recognition system for parameter management.

[0121] Specifically, for different medical tissue structures in the image to be enhanced, corresponding structural parameters are used to perform structural enhancement. For different medical tissue structures in the image to be enhanced, corresponding color enhancement parameters are used to perform color enhancement, thereby obtaining a medical detection image with completed image enhancement.

[0122] In this embodiment, a medical image processing device is also provided, and the medical image processing device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0123] This embodiment provides a medical image processing device, such as Figure 7 As shown, including:

[0124] A first image acquisition device 701 is configured to acquire first image data of an object to be detected;

[0125] A second image acquisition device 702 is configured to acquire second image data of the object to be detected, wherein the resolution of the second image data is smaller than the first image data;

[0126] The processing system 503 includes: an identification module 7031, which is configured to perform identification processing on the second image data to obtain tissue identification results for characterizing multiple medical tissues of the object to be detected; a parameter determination module 7032, which is configured to determine the structural parameters and color parameters of different medical tissues of the object to be detected for a set medical event based on the tissue identification results; and a processing module 7033, which is configured to perform image processing on the first image data based on the structural parameters and color parameters to obtain a medical detection image of the object to be detected.

[0127] In some optional implementations, the processing system 703 further includes:

[0128] A first sending module is configured to send the medical detection image to a storage unit;

[0129] and a second sending module, configured to send the medical detection image to a display device;

[0130] The equipment also includes:

[0131] A storage device configured to receive and store the medical detection image sent by the processing system;

[0132] The display device is configured to receive and display the medical detection image sent by the processing system.

[0133] In an optional implementation, the processing module 7033 includes:

[0134] A preprocessing unit, configured to preprocess the first image data to obtain an image to be enhanced;

[0135] A structure enhancement unit is configured to perform structure enhancement using corresponding structure parameters for different medical tissue structures in the image to be enhanced;

[0136] The color enhancement unit is configured to perform color enhancement using corresponding color parameters for different medical tissue structures in the image to be enhanced.

[0137] In an optional embodiment, the preprocessing unit includes:

[0138] A decoding subunit, configured to decode the second image data to obtain original image data;

[0139] A demosaicing subunit is configured to perform a demosaicing process on the original image data to obtain three-color image data;

[0140] The gamma conversion subunit is configured to perform gamma conversion on the three-color image data to obtain an image to be enhanced.

[0141] In an optional implementation, the second image data is a low voltage differential signal;

[0142] The decoding unit performs decoding processing on the second image data, including: decoding the low voltage differential signal to obtain original image data;

[0143] The demosaicing unit performs demosaicing on the original image data, including: converting the original image data into three-color image data.

[0144] In an optional implementation, the identification module 7031 includes:

[0145] An acquisition unit, configured to acquire the number of medical tissues to be identified;

[0146] A convolution unit, configured to perform convolution processing on the second image data according to the number of medical tissues to be identified, to obtain convolution image data;

[0147] A pooling compression unit is configured to perform pooling compression processing on the convolution image data to obtain compressed data;

[0148] The connection activation unit is configured to perform full connection and activation processing on the compressed data to obtain a tissue recognition result.

[0149] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0150] The medical image processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0151] The embodiment of the present invention also provides a computer device having the above Figure 8 The medical image processing device shown.

[0152] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0153] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0154] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 performs the medical image processing shown in the above embodiment.

[0155] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0156] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0157] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0158] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0159] The embodiment of the present invention also provides a computer-readable storage medium. The medical image processing according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be downloaded through a network and originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium, so that the medical image processing described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the medical image processing shown in the above embodiment is implemented.

[0160] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the medical image processing and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the computer program instruction exists in a computer-readable medium in the form of, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0161] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A medical image processing method, characterized in that: The method comprises: Acquire first image data of the object to be detected by a first image acquisition device; Acquire second image data of the object to be detected by a second image acquisition device, wherein the resolution of the second image data is smaller than that of the first image data; Performing recognition processing on the second image data to obtain tissue recognition results for characterizing multiple medical tissues of the object to be detected; According to the tissue recognition result, determining the structural parameters and color parameters of different medical tissues of the object to be detected for a set medical event; The first image data is processed according to the structural parameters and the color parameters to obtain a medical detection image of the object to be detected.

2. The medical image processing according to claim 1, characterized in that: The step of performing image processing on the first image data according to the structural parameter and the color parameter to obtain a medical detection image of the object to be detected includes: Preprocessing the first image data to obtain an image to be enhanced; For different medical tissue structures in the image to be enhanced, corresponding structural parameters are used to perform structural enhancement; According to different medical tissue structures in the image to be enhanced, corresponding color parameters are used to perform color enhancement.

3. The medical image processing according to claim 2, characterized in that: The preprocessing of the second image data to obtain the image to be enhanced includes: Decoding the second image data to obtain original image data; Performing a demosaicing process on the original image data to obtain three-color image data; Gamma conversion is performed on the three-color image data to obtain the image to be enhanced.

4. The medical image processing according to claim 3, characterized in that: The second image data is a low voltage differential signal; The decoding process of the second image data includes: decoding the low voltage differential signal to obtain the original image data; The de-mosaicing of the original image data includes: converting the original image data into three-color image data.

5. The medical image processing according to claim 1, characterized in that: The step of performing recognition processing on the second image data to obtain tissue recognition results for characterizing multiple medical tissues of the object to be detected includes: Obtain the number of medical tissues to be identified; According to the number of the medical tissues to be identified, performing convolution processing on the second image data to obtain convolution image data; Performing pooling compression processing on the convolution image data to obtain compressed data; The compressed data is fully connected and activated to obtain the tissue identification result.

6. A medical image processing device, characterized in that: The medical image processing device comprises: A first image acquisition device is configured to acquire first image data of the object to be detected; A second image acquisition device is configured to acquire second image data of the object to be detected, wherein the resolution of the second image data is smaller than that of the first image data; The processing system includes: an identification module configured to perform identification processing on the second image data to obtain tissue identification results for characterizing multiple medical tissues of the object to be detected; a parameter determination module configured to determine the structural parameters and color parameters of different medical tissues of the object to be detected for a set medical event based on the tissue identification results; and a processing module configured to perform image processing on the first image data based on the structural parameters and the color parameters to obtain a medical detection image of the object to be detected.

7. The medical image processing device according to claim 6, characterized in that: The processing system further includes: a first sending module configured to send the medical detection image to a storage unit; and a second sending module configured to send the medical detection image to a display device; The medical image processing device further comprises: A storage device configured to receive and store the medical detection image sent by the processing system; The display device is configured to receive and display the medical detection image sent by the processing system.

8. A computer device, characterized in that: The computer device comprises: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the medical image processing method according to any one of claims 1 to 5 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the medical image processing method according to any one of claims 1 to 5.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the medical image processing method according to any one of claims 1 to 5.

Images