Image processing method, device, equipment, readable storage medium and program product

Abstract

This application discloses an image processing method, apparatus, device, readable storage medium, and program product, relating to the field of medical data processing. The method includes: acquiring a sample image, which comprises an image obtained by collecting data from a target sample within a preset wavelength band; acquiring a target image in the sample image corresponding to at least one target wavelength in the preset wavelength band, obtaining a pseudo-color image; dividing the sample image into regions based on differences in the types of sample elements in the sample image, obtaining region division results; and determining target regions in the sample image that include the target element type based on the pseudo-color image and the region division results. This approach can improve the accuracy and reduce the difficulty of pathological sample collection. Analyzing target samples using the spectral characteristics corresponding to different wavelengths is simple, cost-effective, and easier to widely apply. This application can be applied to various scenarios such as cloud technology, artificial intelligence, and intelligent transportation.

Description

Technical Field

[0001] This application relates to the field of medical data processing, and in particular to an image processing method, apparatus, device, readable storage medium, and program product. Background Technology

[0002] After surgically removing a pathological specimen containing tumor tissue, the sampled specimen is usually further studied to find and identify the boundaries of the tumor tissue in the pathological specimen, so as to conduct a more accurate analysis of the tumor tissue and obtain more valuable medical analysis results.

[0003] In related techniques, after the surgically removed pathological specimen is fixed with formalin, the boundaries of the tumor tissue are usually determined by the pathologist through visual observation, or by scanning the pathological specimen with X-ray equipment, and the doctor interprets the X-ray images to determine the boundaries of the tumor tissue and perform the tumor tissue sampling.

[0004] However, when using the above methods to determine the boundaries of tumor tissue, some lesions with indistinct tumor beds are difficult to identify with the naked eye, and X-ray equipment is also difficult to achieve widespread use due to its high price. Summary of the Invention

[0005] This application provides an image processing method, apparatus, device, readable storage medium, and program product that can analyze target samples by utilizing the spectral characteristics of the target samples at different wavelengths, thereby improving the accuracy of pathological sampling. The technical solution is as follows.

[0006] On the one hand, an image processing method is provided, the method comprising:

[0007] Acquire sample images, which include images obtained by acquiring target samples within a preset wavelength band;

[0008] Obtain a target image from the sample image that corresponds to at least one target wavelength in the preset band to obtain a pseudo-color image;

[0009] Based on the differences in the sample element types in the sample image, the sample image is divided into regions to obtain the region division results. The sample element types include the target element types to be identified.

[0010] Based on the pseudo-color image and the region segmentation result, a target region including the target element type is determined in the sample image.

[0011] On the other hand, an image processing apparatus is provided, the apparatus comprising:

[0012] The sample acquisition module is used to acquire sample images, which include images obtained by acquiring target samples within a preset band;

[0013] The image acquisition module is used to acquire a target image in the sample image that corresponds to at least one target wavelength in the preset band, and obtain a pseudo-color image;

[0014] The region segmentation module is used to segment the sample image into regions based on the differences in the sample element types in the sample image, and obtain the region segmentation result, wherein the sample element types include the target element types to be identified;

[0015] The region determination module is used to determine, based on the pseudo-color image and the region segmentation result, a target region in the sample image that includes the target element type.

[0016] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement any of the image processing methods described in the embodiments of this application above.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the image processing method as described in any of the embodiments of this application above.

[0018] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the image processing methods described in the above embodiments.

[0019] The beneficial effects of the technical solutions provided in this application include at least the following:

[0020] The target image corresponding to the target wavelength is obtained from the sample image, and a pseudo-color image is obtained. The sample image is then divided into regions to obtain the region division results. The target region is determined by combining the pseudo-color image and the region division results. This method avoids relying solely on the doctor's naked-eye observation and description to judge the size and region of tumor tissue, thus reducing the inaccuracy of judging the patient's tumor tissue region. Based on a pre-determined preset wavelength, target samples are acquired to obtain sample images. At least one target wavelength with good performance is selected from the preset wavelength, and based on at least one target wavelength, the target image corresponding to the target wavelength is determined from the sample image. After processing the target image, a pseudo-color image is obtained, which can more accurately reflect the advantages of the target wavelength. Furthermore, based on the differences in the sample element types in the sample image, the sample image is divided into regions to obtain the region division results. By combining pseudo-color images and region segmentation results, target regions including target element types are identified, thereby determining the location information of the region to be identified (e.g., tumor tissue), improving the accuracy of pathological sampling, reducing the difficulty of pathological sampling, and using the spectral characteristics of the target sample at different wavelengths to analyze the target sample. This method is not only relatively simple to operate, but also relatively low in cost and easier to widely apply. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application;

[0023] Figure 2 This is a flowchart of an image processing method provided in an exemplary embodiment of this application;

[0024] Figure 3 This is a schematic diagram illustrating the acquisition of a sample image provided in an exemplary embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a sample image within a preset band provided in an exemplary embodiment of this application;

[0026] Figure 5 This is a spectral feature curve corresponding to a sample image provided in an exemplary embodiment of this application;

[0027] Figure 6 This is a schematic diagram of image processing of a target sample provided in an exemplary embodiment of this application;

[0028] Figure 7 This is a schematic diagram illustrating the synthesis of a pseudo-color image provided in an exemplary embodiment of this application;

[0029] Figure 8 This is a flowchart of an image processing method provided in another exemplary embodiment of this application;

[0030] Figure 9 This is a schematic diagram of the collection of pathological samples provided in another exemplary embodiment of this application;

[0031] Figure 10 This is a flowchart of region division of a sample image provided in an exemplary embodiment of this application;

[0032] Figure 11 This is a spectral characteristic curve of a hollow organ provided in an exemplary embodiment of this application;

[0033] Figure 12 This is a spectral characteristic curve of the kidney provided in an exemplary embodiment of this application;

[0034] Figure 13 This is a spectral characteristic curve of the breast tissue provided in an exemplary embodiment of this application;

[0035] Figure 14 This is a spectral characteristic curve of the lung provided in an exemplary embodiment of this application;

[0036] Figure 15 This is a schematic diagram of a monochrome fill hint provided in an exemplary embodiment of this application;

[0037] Figure 16 This is a flowchart of obtaining prediction results provided by an exemplary embodiment of this application;

[0038] Figure 17 This is a schematic diagram illustrating the image representation of four tissue classifications provided in an exemplary embodiment of this application;

[0039] Figure 18 This is a schematic diagram illustrating different image representations of renal cell carcinoma provided in an exemplary embodiment of this application;

[0040] Figure 19 This is a schematic diagram illustrating different image representations of the breast provided in an exemplary embodiment of this application;

[0041] Figure 20 This is a schematic diagram illustrating different image representations of the breast provided in another exemplary embodiment of this application;

[0042] Figure 21 This is a structural block diagram of an image processing apparatus provided in an exemplary embodiment of this application;

[0043] Figure 22 This is a structural block diagram of an image processing apparatus provided in another exemplary embodiment of this application;

[0044] Figure 23 This is a structural block diagram of a server provided in an exemplary embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0046] First, a brief introduction to the terms used in the embodiments of this application will be given.

[0047] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0048] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0049] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0050] In related techniques, after the surgically removed pathological specimen is fixed with formalin, the boundaries of the tumor tissue are usually determined by the pathologist through visual inspection, or by scanning the specimen with X-ray equipment, with the doctor interpreting the X-ray images to determine the boundaries of the tumor tissue and perform tumor tissue sampling. However, when using the above methods to determine the boundaries of the tumor tissue, some lesions with indistinct tumor beds are difficult to identify with the naked eye, and X-ray equipment is also difficult to achieve widespread availability due to its high cost.

[0051] This application provides an image processing method that analyzes target samples using their spectral characteristics at different wavelengths to improve the accuracy of pathological sampling. The method trained according to this application can be applied in at least one of the following scenarios.

[0052] I. Applications in the medical field

[0053] In tumor resection surgery, accurately knowing the tumor margins is crucial for complete tumor removal, preventing recurrence and avoiding secondary surgeries. Postoperative histopathological analysis is the gold standard for tumor diagnosis. The selection of pathological tissue blocks is paramount for accurately obtaining lesion information. Missing lesion-containing tissue blocks limits the pathologist's ability to make accurate judgments, while selecting too many blocks significantly increases slide preparation workload and reduces efficiency. Illustratively, using the aforementioned image processing method, lesion-containing tissues (e.g., kidneys, breast tissue) are used as target samples. Images are acquired within a preset wavelength range. A pseudo-color image is obtained by selecting the target wavelength from these sample images. The sample images are then divided into regions based on their element types. By comprehensively analyzing the region division results and the pseudo-color image, the target region corresponding to the tumor tissue can be accurately identified, achieving target region recognition. The above methods can help pathologists locate lesion areas more quickly, reduce the cost of using imaging equipment such as X-rays, and utilize more widely available and economical spectroscopic instruments to acquire sample images. By analyzing sample images with spectral information, the accuracy of tumor tissue diagnosis can be improved while reducing medical costs.

[0054] II. Applications in the field of food testing

[0055] Food safety is a matter of life and death. Food often contains various components, and unhealthy components or incorrect proportions of components can lead to food safety incidents. Illustratively, using the image processing method described above, the food to be tested is used as the target sample. The target sample is acquired within a preset wavelength band, and a pseudo-color image is obtained by selecting the target image corresponding to the target wavelength. The sample image is then divided into regions based on the sample element types, yielding region division results. By comprehensively analyzing the region division results and the pseudo-color image, the regions corresponding to different components in the food to be tested can be accurately determined, and the target regions corresponding to unhealthy components can be identified, thus achieving the identification of target regions. This method can assist food regulatory agencies in better supervising food, and by combining the pseudo-color image determined based on the target wavelength with the region division results, the target regions can be identified more accurately.

[0056] It is worth noting that the above application scenarios are merely illustrative examples, and the image processing method provided in this embodiment can also be applied to other scenarios, which are not limited in this application.

[0057] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0058] Secondly, the implementation environment involved in the embodiments of this application will be described, for illustrative purposes only. Please refer to [the relevant documentation]. Figure 1 The implementation environment involves a terminal 110 and a server 120, which are connected via a communication network 130.

[0059] In some embodiments, the terminal 110 is equipped with an application that has image acquisition capabilities. In some embodiments, the terminal 110 is used to send sample images to the server 120. The server 120 can determine the target region containing the target element type in the sample image based on the spectral information corresponding to the sample image through the image processing model 121, and identify the target region in a special way and feed it back to the terminal 110 for display.

[0060] The image processing model 121 is applied as follows: a target wavelength is selected from a preset band; based on the target wavelength, a target image corresponding to the target wavelength is determined from the sample image; the target image is then processed to obtain a pseudo-color image; furthermore, based on the sample element types in the sample image, the sample image is divided into regions to obtain the corresponding region division results; combining the region division results and the pseudo-color image, the target region in the sample image is determined. The target region can be used to indicate the location information of the target element type. For example, if the target sample is a pathological sample, after analyzing the target sample, the determined target region is the region corresponding to tumor tissue, thus more accurately determining the region information corresponding to tumor tissue. The above process is an example of a non-unique application of the image processing model 121.

[0061] It is worth noting that the aforementioned terminals include, but are not limited to, mobile terminals such as mobile phones, tablets, portable laptops, smart voice interaction devices, smart home appliances, and in-vehicle terminals, as well as desktop computers; the aforementioned servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0062] Cloud technology refers to a hosting technology that unifies hardware, applications, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0063] In some embodiments, the server described above can also be implemented as a node in a blockchain system. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0064] Based on the above introduction to terminology and application scenarios, the image processing method provided in this application will be described, taking the application of this method to a server as an example. Figure 2 As shown, the method includes the following steps 210 to 240.

[0065] Step 210: Obtain the sample image.

[0066] The sample images include images obtained by acquiring target samples within a preset band.

[0067] Wavebands are used to indicate the range of wavelengths. For example, the visible light band indicates the wavelength range between 380nm and 750nm; the near-infrared band indicates the wavelength range between 750nm and 2500nm; and the mid-infrared band indicates the wavelength range between 2500nm and 25000nm.

[0068] Optionally, the lighting source providing the wavelength includes halogen lamps, incandescent lamps, and light-emitting diode (LED) light sources. Illustratively, a preset wavelength band indicates a pre-set wavelength band, and the lighting source providing the wavelength covers the preset wavelength band. For example, the preset wavelength band is 400nm to 1700nm, and the selected lighting source covers the 400nm to 1700nm band; or, the preset wavelength band is 400nm to 1700nm, lighting source A can provide the 400nm to 1200nm band, and lighting source B can provide the 1100nm to 1800nm ​​band, using lighting sources A and B as lighting sources providing the preset wavelength band, etc.

[0069] The target sample is used to indicate the sample to be analyzed. Optionally, the target sample is a surgically removed pathological sample, and analyzing the surgically removed pathological sample can reveal the location information, properties information, etc. of the pathological sample; or, the target sample is a chemical mixture, and analyzing the chemical mixture can reveal the composition information, proportion information, etc. of the mixture; or, the target sample is a gemstone, and analyzing the gemstone can reveal the structural information of the gemstone, etc.

[0070] In an optional embodiment, a push-broom acquisition operation is performed on the target sample to obtain a sample image.

[0071] Pushbroom acquisition is a method of image acquisition that involves scanning point-by-point along a scan line. The pushbroom acquisition operation is based on the acquisition device. (Illustrative example, such as...) Figure 3 The image shows a pushbroom-type short-wave infrared hyperspectral imaging system. This system includes a sample stage 310, a short-wave infrared hyperspectral camera 320, a line light source 330, and a target sample 340. The hyperspectral camera 320 combines imaging and spectral detection technologies to acquire sample images with spectral information. These hyperspectral images are three-dimensional, with the x-axis and y-axis representing two-dimensional coordinates and the z-axis representing wavelength information. Compared to conventional imaging techniques, hyperspectral imaging adds spectral information to the image; that is, the sample image is an image with spectral information.

[0072] Optionally, after determining the preset wavelength band, the target sample is illuminated with a light source, and multiple images of the target sample are taken under different wavelength irradiation conditions within the preset wavelength band, thereby obtaining multiple sample images corresponding to the target sample. For example, the preset wavelength band is 900nm–1700nm (a band selected from the near-infrared band), the target sample is a surgically removed pathological specimen, a halogen lamp is used as the illumination source, and a hyperspectral camera is used as the image acquisition device to acquire images of the surgically removed pathological specimen at different wavelengths. For instance, within the preset wavelength band, a hyperspectral camera is used to acquire one image for each wavelength, thereby obtaining multiple sample images corresponding to different wavelengths.

[0073] like Figure 3 As shown, a line light source 330 illuminates the target sample 340, and a short-wave infrared hyperspectral camera 320 captures images of the target sample 340 under different wavelengths of illumination, thereby achieving the process of acquiring multiple sample images at multiple different wavelengths. Figure 4 As shown, multiple sample images 410 were acquired using a short-wave infrared hyperspectral camera 320. The multiple sample images are arranged from top to bottom according to the wavelength range of 900nm to 1700nm. The multiple sample images 410 are three-dimensional hyperspectral images with spectral information.

[0074] Indicative, such as Figure 5As shown, points M (420) and N (430) are arbitrarily selected in sample image 410. Based on the spectral information corresponding to sample image 410, a specific spectral curve 510 corresponding to point M (420) and a spectral characteristic curve 520 corresponding to point N (430) are obtained. The spectral characteristic curve is a graph showing the relationship between light reflectance and wavelength, with wavelength on the horizontal axis and reflectance on the vertical axis. Reflectance indicates the ratio of the light flux reflected by the target sample to the light flux incident on the target sample. Optionally, Figure 5 The spectral characteristic curves in the figure are obtained after reflectance correction.

[0075] In an optional embodiment, within the target band range, a tunable filter is used to determine at least one wavelength; based on the acquisition device, a pushbroom acquisition operation is performed on the target sample to obtain a sample image corresponding to at least one wavelength.

[0076] Optionally, a liquid crystal tunable filter (LCTF) can be added in front of the hyperspectral camera. This LCTF is used to select wavelengths from illumination sources covering the target wavelength band, enabling rapid and vibration-free selection of wavelengths within the visible or near-infrared band. For example, if the illumination source covers a wavelength band of 900nm–1700nm, the light emitted by the illumination source, after passing through the LCTF, will yield light with a wavelength of 1130nm. In other words, the LCTF filters out light of other wavelengths within the preset band except for 1130nm.

[0077] Optionally, the acquisition device is a hyperspectral camera, specifically a camera with a built-in grating pushbroom structure. Based on the arrangement of the gratings, a pushbroom acquisition operation is performed on the target sample to obtain a sample image. Alternatively, a hyperspectral imaging method using an external pushbroom structure for the hyperspectral camera can be employed, such as... Figure 3 As shown, the moving sample stage 310 performs push-broom imaging, etc. It is worth noting that the above is only an illustrative example, and the embodiments of this application are not limited thereto.

[0078] Step 220: Obtain the target image in the sample image that corresponds to at least one target wavelength in the preset band, and obtain a pseudo-color image.

[0079] In illustrative terms, the sample images are multiple images acquired from the target sample, and the wavelengths corresponding to the sample images are within a preset wavelength band. Within the preset wavelength band, at least one target wavelength is selected from multiple wavelengths, and the sample image corresponding to the target wavelength is used as the target image, ultimately resulting in a pseudo-color image.

[0080] Optionally, for a target wavelength, there exists at least one sample image corresponding to that target wavelength. Illustratively, when multiple sample images exist corresponding to a selected target wavelength, either a sample image can be randomly selected from the multiple sample images as the target image corresponding to the target wavelength, or multiple sample images can be combined and analyzed to determine the target image corresponding to the target wavelength.

[0081] Alternatively, for a target wavelength, there exists a sample image corresponding to that target wavelength, and this sample image is used as the target image. Optionally, taking one target image corresponding to one target wavelength as an example, the processing method for the target image differs depending on the number of target wavelengths selected. Illustratively, the cases of selecting one target wavelength and selecting multiple target wavelengths are analyzed separately.

[0082] (1) Select a target wavelength

[0083] In an optional embodiment, a target image corresponding to a target wavelength in a preset band is color-coded to obtain a pseudo-color image.

[0084] Indicatively, the preset wavelength range is 900nm to 1700nm selected from the near-infrared band. Since the preset wavelength range is invisible, the corresponding image is a grayscale image. A target wavelength is selected from the preset wavelength range; the target image corresponding to this wavelength is a grayscale image. Color processing is then applied to this grayscale image to obtain a pseudo-color image.

[0085] Pseudo-color image processing refers to the technical process of converting a black-and-white grayscale image into a color image, thereby improving the recognizability of the image content. Illustratively, methods such as grayscale segmentation and grayscale transformation are used for pseudo-color image processing.

[0086] Optionally, the grayscale image is a single-channel image, meaning each pixel has only one value representing color, with pixel values ​​ranging from 0 to 255. 0 indicates black, 255 indicates white, and intermediate values ​​represent different levels of gray. Alternatively, when the grayscale image is a three-channel image, the pixel values ​​in all three channels are the same.

[0087] Optionally, images contrasting with single-channel images include three-channel images, where each pixel has three values. Illustratively, an RGB image is a three-channel image, obtained by varying the red (R), green (G), and blue (B) color channels and superimposing them to produce a variety of colors. Each pixel in this image is represented by three values.

[0088] Indicative, the target sample is a surgically removed pathological sample, such as... Figure 6The figures illustrate different image processing techniques applied to the target sample. Figure 610 indicates the target sample (obtained using a conventional camera for clarity); Figure 620 indicates a hyperspectral image at a wavelength of 1300 nm; Figures 631 to 634 indicate tissue manifestations observed using hematoxylin-eosin (HE) staining, with Figure 631 indicating cancerous tissue (point A in Figure 610 or 620), Figure 632 indicating adipose tissue (point B in Figure 610 or 620), Figure 633 indicating normal mucosal tissue (point B in Figure 610 or 620), and Figure 634 indicating muscle tissue (point D in Figure 610 or 620).

[0089] Indicatively, a wavelength of 1300nm is selected as the target wavelength, and the hyperspectral image corresponding to the target wavelength in Figure 620 is used as the target image. The target image is then subjected to the above color processing to obtain a pseudo-color image.

[0090] (2) Select multiple target wavelengths

[0091] This is illustrative of determining at least two target images corresponding to at least two target wavelengths, based on at least two target wavelengths. Here, the i-th target wavelength corresponds to the i-th target image, and i is a positive integer.

[0092] In an optional embodiment, at least two target images corresponding to at least two target wavelengths in a preset target band are synthesized, and a pseudo-color image is obtained by colorizing the synthesized image.

[0093] Indicatively, at least two target wavelengths are selected from a preset target band, with each target wavelength corresponding to a target image. Optionally, the at least two target images are synthesized to obtain candidate images.

[0094] Illustratively, the methods for synthesizing multiple target images include at least one of the following.

[0095] (1) Pixel value processing

[0096] In an optional embodiment, the pixel values ​​of corresponding pixels in at least two target images are averaged to obtain the target pixel values ​​of the corresponding pixels; candidate images are determined based on the target pixel values ​​corresponding to each pixel.

[0097] Schematic illustration: After obtaining at least two target images corresponding to at least two target bands, the pixel values ​​of corresponding pixels in the at least two target images are summed and averaged to obtain the target pixel value of the corresponding pixel. That is, the target pixel value is the average value obtained after comprehensive analysis of the pixel values ​​of corresponding pixels in different target images. Optionally, after determining the target pixel value corresponding to each pixel, candidate images are obtained based on the pixel position information, and the pixel value of each pixel in the candidate image is the corresponding target pixel value.

[0098] (2) Software processing

[0099] In an optional embodiment, after determining the target image corresponding to the target wavelength, at least two target images are synthesized by software to obtain a candidate image.

[0100] To illustrate, at least two target images are input into Photoshop, and operations such as alignment, color adjustment, seam removal, and export are performed to obtain a candidate image after compositing the at least two target images.

[0101] The above are merely illustrative examples, and the embodiments of this application are not intended to limit the scope of the application.

[0102] In an optional embodiment, the candidate image is color-coded to obtain a pseudo-color image.

[0103] In a schematic manner, based on the brightness values ​​of pixels in the candidate image, the brightness of the pixels in the candidate image is graded to determine at least two brightness levels; and colors are assigned to the at least two brightness levels to obtain a pseudo-color image.

[0104] Optionally, the target image is a grayscale image, and the candidate image synthesized based on the target image is also a grayscale image. The target pixel value corresponding to each pixel in the grayscale image is used to indicate the brightness of the candidate image. Illustratively, the target pixel value is between 0 and 255, where 0 indicates black (minimum brightness) and 255 indicates white (maximum brightness). That is, the smaller the target pixel value, the lower the brightness; the larger the target pixel value, the higher the brightness.

[0105] Indicative, such as Figure 7 The diagram shows the process of synthesizing and assigning values ​​to target images corresponding to three target wavelengths (1100nm, 1300nm, and 1450nm) to obtain a pseudo-color image.

[0106] Figure 710 indicates a hyperspectral image with a wavelength of 1100 nm; Figure 720 indicates a hyperspectral image with a wavelength of 1300 nm; and Figure 730 indicates a hyperspectral image with a wavelength of 1450 nm. Optionally, after synthesizing and assigning values ​​to the hyperspectral images corresponding to the above three target wavelengths, the pseudo-color image shown in Figure 740 is obtained.

[0107] Step 230: Based on the differences in the types of sample elements in the sample image, the sample image is divided into regions to obtain the region division results.

[0108] Among them, the sample element types include the target element types to be identified.

[0109] Optionally, the sample element type is used to indicate the differences in sample properties corresponding to different sample regions in the sample image. Illustratively, when the sample image is an image obtained from a pathological sample, the sample element type includes: tumor tissue, adipose tissue, mucosal tissue, muscle tissue, etc., in the pathological sample; when the sample image is an image obtained from a chemical mixture (including compound A, compound B, and impurities), the sample element type includes: compound A, compound B, and impurities.

[0110] For illustration, when the sample image is a pathological image obtained by taking a picture of a pathological sample, the target element type is a pre-determined tumor tissue to be identified (one of the sample element types corresponding to the pathological image); or, the target element type is a pre-determined adipose tissue to be identified, etc. Optionally, when the sample image is a chemical image obtained by taking a picture of a chemical mixture, the target element type is a pre-determined compound B to be identified (one of the sample element types corresponding to the chemical image), etc.

[0111] In illustrative terms, the sample image is an image with spectral information. Due to the differences in the properties of different substances, the spectral information varies. Based on the spectral information corresponding to the sample image, the sample image is divided into regions to obtain the region division results.

[0112] To illustrate, spectral information is displayed differently in the sample image. For example, when the sample image is a grayscale image, the region corresponding to sample element A is the darkest, while the region corresponding to target sample element is the lightest, thus obtaining the region division result of the sample image.

[0113] In one optional embodiment, to facilitate differentiation, different regions can be filled with different colors to obtain a region division result with color; or, a darker outline can be used to divide different regions to obtain a region division result with more obvious separation, etc.

[0114] It is worth noting that the above are merely illustrative examples, and the embodiments of this application are not limited thereto.

[0115] Step 240: Based on the pseudo-color image and region segmentation results, determine the target region in the sample image that includes the target element type.

[0116] In illustrative terms, a pseudo-color image is an image obtained by processing a target image corresponding to a target wavelength; the region segmentation result is the result of dividing regions according to the type of sample elements in the sample image. Optionally, the pseudo-color image is segmented using different colors. For example, the sample image is an image obtained from a pathological sample, in which tumor tissue is presented as orange; adipose tissue as bright yellow; mucosal tissue as a light orange (lighter than tumor tissue); and muscle tissue as a dark orange (darker than tumor tissue), etc.

[0117] In an optional embodiment, an overlapping region is determined between the pseudo-color image and the region segmentation result; in the sample image, the overlapping region is used as a target region including the target element type.

[0118] In a schematic manner, the target sample is a pathological sample. After acquiring the pathological sample, a sample image with spectral information is obtained. To observe the tumor tissue in the sample image, the above processing procedure is performed on the sample image to obtain a pseudo-color image corresponding to the selected target wavelength and the region segmentation result of the sample image. Based on the tumor tissue region identified in the pseudo-color image and the identification result of the target element type (tumor tissue) in the region segmentation result, the overlapping area is taken as the target region including the target element type (tumor tissue), thus realizing the identification process of the tumor tissue region.

[0119] The above are merely illustrative examples, and the embodiments of this application are not intended to limit the scope of the application.

[0120] In summary, this method involves obtaining the target image corresponding to the target wavelength from the sample image, generating a pseudo-color image, dividing the sample image into regions, and combining the pseudo-color image and the region division results to determine the target region. This approach avoids relying solely on the doctor's naked-eye observation and description to judge the size and region of tumor tissue. Based on a pre-determined preset wavelength, the target sample is acquired, resulting in a sample image. At least one target wavelength with good performance is selected from the preset wavelength, and based on this wavelength, the corresponding target image is determined from the sample image. After processing the target image, a pseudo-color image is obtained, which accurately reflects the advantages of the target wavelength. Based on the differences in sample element types within the sample image, the sample image is divided into regions, resulting in region division results. Combining the pseudo-color image with the region division results, the target region, including the target element type, is determined, thereby identifying the location information of the region to be identified (e.g., tumor tissue). This improves the accuracy and reduces the difficulty of pathological sampling. Analyzing the target sample using the spectral characteristics corresponding to different wavelengths is not only simple to operate but also relatively low-cost, making it easier to apply and widely adopted.

[0121] In an optional embodiment, the process of region segmentation of the sample image is determined by different spectral information corresponding to different sample element types. (Illustrative example, such as...) Figure 8 As shown above, Figure 2 Step 230 in the illustrated embodiment can also be implemented as steps 810 to 850.

[0122] Step 810: Obtain the standard image.

[0123] The standard image is a pre-labeled image with spectral information obtained from the acquisition of the target sample.

[0124] Optionally, the sample image and the standard image are images obtained by acquiring the target sample. When acquiring the target sample, a gold standard for the sample image is determined, i.e., a standard image is determined. The gold standard is used to indicate the most reliable method for diagnosing diseases, currently recognized by the clinical medical community.

[0125] In an optional embodiment, the target sample is a surgically removed pathological sample. (Illustrative, such as...) Figure 9The diagram illustrates the process of obtaining pathological samples. First, the doctor removes the patient's pathological sample (which includes tumor tissue) through a tumor resection surgery (910). Then, the pathological sample is cut (920) to obtain a tissue block of appropriate volume. Next, the tissue block is fixed (930) using methods such as formalin immersion. For example, within 30 minutes of the surgically removed pathological sample, the tissue block is placed in a sufficient volume of 3.7% neutral formalin solution for fixation, with a fixation time of 12-48 hours. Subsequently, tissue slices (average approximately 5mm) with a thickness of 5mm ± 1mm ​​are cut from the fixed tissue block, including tumor tissue and surrounding 1-2cm of normal tissue. Optionally, either the fixed tissue block or the sliced ​​tissue block can be used as the target sample.

[0126] In an optional embodiment, after acquiring the target sample, the tissue slide undergoes gross sampling, routine dehydration, embedding, and HE staining to obtain a stained image 940. The stained image 940 is then scanned by a digital scanner 950 to obtain multiple whole-field-of-view (WSI) images 960. The WSI image is an image of the pathological sample acquired under the digital scanner 950 (a type of motorized microscope). Illustratively, if the size of a single WSI image is small, the analyzed WSI image can be composed of multiple pathological slides stitched together. For example, multiple WSI fragments can be stitched together using WSI stitching software to reconstruct a virtual large slide 970.

[0127] Optionally, using the Advanced Systems Analysis Program (ASAP) on a virtual large slide 970 as the gold standard for annotation yields multiple annotated WSI images obtained from scans of the pathological sample. Annotation can be performed on regions, including areas containing one or more lesions, as well as specific indicative areas. For example, in hollow organs, tumor tissue is marked red, normal mucosa green, adipose tissue yellow, and muscle tissue blue; in solid organs, tumor tissue is marked red, normal tissue green, and adipose tissue yellow, etc. Optionally, the above color markings are merely illustrative examples; different colors can be used to mark the selected tissues. For instance, when marking breast tissue in solid organs, tumor tissue in the breast tissue is marked red, adipose tissue yellow, and fibrous connective tissue green, etc. Optionally, if the observed organ does not contain tissue of the corresponding color, it may be left unmarked. For example, when marking solid organs using the above color-coding method, if the observed organ does not contain adipose tissue, it will not be marked yellow. This is illustrated by using the marked WSI image as the standard image to illustrate the process of acquiring the standard image.

[0128] Step 820: Train the candidate segmentation model using standard images.

[0129] Among them, the candidate segmentation model is an untrained model with certain region segmentation capabilities. Illustratively, the candidate segmentation model is trained using standard images as the gold standard. Through training on a large number of standard images, the candidate segmentation model learns and gradually becomes able to automatically identify special regions such as lesion areas, and gradually acquire region segmentation capabilities.

[0130] Step 830: In response to the training effect of the candidate segmentation model, the image segmentation model is obtained.

[0131] The image segmentation model is used to segment regions of the target image. Illustratively, during the training of the candidate segmentation model, the image segmentation model is obtained when the training of the candidate segmentation model reaches the training objective. Optionally, the training effect of the candidate segmentation model is judged by the loss value, and the training objective includes at least one of the following situations.

[0132] 1. In response to the convergence of the loss value, the candidate segmentation model obtained from the most recent iteration of training is used as the image segmentation model.

[0133] Indicatively, the convergence of the loss value indicates that the value of the loss obtained by the loss function no longer changes or the change is less than a preset threshold. For example, if the loss value corresponding to the nth standard image is 0.1 and the loss value corresponding to the (n+1)th standard image is also 0.1, it can be considered that the loss value has reached a convergence state. The candidate segmentation model with the adjusted loss value corresponding to the nth or (n+1)th standard image is used as the image segmentation model to realize the training process of the candidate segmentation model.

[0134] 2. In response to the number of times the loss value is obtained reaching the threshold, the candidate segmentation model obtained from the most recent iteration of training is used as the image segmentation model.

[0135] In a schematic representation, each acquisition yields one loss value. The number of times the loss value is acquired for training the image segmentation model is pre-defined. When one standard image corresponds to one loss value, the number of acquisitions is equal to the number of standard images; conversely, when one standard image corresponds to multiple loss values, the number of acquisitions is equal to the number of loss values. For example, if one acquisition yields one loss value, and the threshold for the number of acquisitions is 10, then when the threshold is reached, the candidate segmentation model with the most recent loss value adjustment is used as the image segmentation model, or the candidate segmentation model with the smallest loss value adjustment during the 10 loss value adjustments is used as the image segmentation model, thus completing the training process for the candidate segmentation model.

[0136] It is worth noting that the above are merely illustrative examples, and the embodiments of this application are not limited thereto.

[0137] Step 840: The sample images are processed by a pre-trained image segmentation model to determine the difference representation of element types.

[0138] In an optional embodiment, the sample images are preprocessed and then input into a pre-trained image segmentation model.

[0139] Indicative, such as Figure 10 As shown, after acquiring multiple sample images 1010, the sample images 1010 are preprocessed 1020. The image preprocessing 1020 process includes at least one of the following: performing geometric transformation operations and image enhancement operations (such as background correction, registration, and denoising) on ​​the sample images to highlight important features in the sample images. Then, the preprocessed sample images 1010 are passed through a pre-trained image segmentation model 1030, which divides the regions in the sample images.

[0140] In an optional embodiment, the sample image is an image with spectral information. Spectral analysis is performed on the target image to obtain spectral analysis results. Based on the spectral analysis results, the difference representation of the element type corresponding to the sample image is determined.

[0141] Based on the differences between the target samples corresponding to the sample images, different spectral analysis results are obtained for different sample images. Illustratively, the spectral analysis results are represented in the form of spectral characteristic curves, with wavelength on the horizontal axis and reflectance on the vertical axis. Different spectral curves indicate the reflectance changes of different target samples at different wavelengths, i.e., the spectral analysis results.

[0142] In an optional embodiment, after analyzing hyperspectral images of 62 different system tissues, the wavelengths for distinguishing tumor tissue from normal tissue in different organs were initially determined to be 1296-1308 nm (this wavelength range has better performance). Illustratively, hollow organs containing tumor tissue (such as the esophagus, stomach, and colon), kidney, breast, and lung were analyzed as target samples. Sample images corresponding to the hollow organs, kidney, breast, and lung were obtained. These sample images were three-dimensional hyperspectral images. Based on the data corresponding to the three-dimensional hyperspectral images, spectral characteristic curves corresponding to the hollow organs, kidney, breast, and lung were obtained.

[0143] like Figure 11 The image shows the spectral characteristic curves corresponding to hollow organs 1110. The wavelength curves corresponding to tumor tissue (cancer tissue) are as follows: the wavelength curve corresponding to adipose tissue is the tumor wavelength curve 1120; the wavelength curve corresponding to adipose tissue is the adipose wavelength curve 1130; the wavelength curve corresponding to normal mucosa is the mucosa wavelength curve 1140; and the wavelength curve corresponding to muscle tissue is the muscle tissue wavelength curve 1150.

[0144] like Figure 12 The image shows the spectral characteristic curves corresponding to kidney 1210. The wavelength curves corresponding to tumor tissue (cancer tissue) are tumor wavelength curves 1220, adipose tissue wavelength curves are adipose wavelength curves 1230, and normal mucosa wavelength curves are mucosa wavelength curves 1240.

[0145] like Figure 13 The image shows the spectral characteristic curves corresponding to breast tissue 1310. The wavelength curves corresponding to tumor tissue (cancer tissue) are tumor wavelength curves 1320; the wavelength curves corresponding to adipose tissue are adipose wavelength curves 1330; and the wavelength curves corresponding to normal mucosa are mucosa wavelength curves 1340.

[0146] like Figure 14The image shows the spectral characteristic curves corresponding to lung 1410, where the wavelength curve corresponding to tumor tissue (cancer tissue) is the tumor wavelength curve 1420; and the wavelength curve corresponding to normal lung is the normal wavelength curve 1430.

[0147] The differences in element types corresponding to the sample images represent the differences between different tissues; for example, tumor tissue and adipose tissue are different. The above is merely an illustrative example, and the embodiments of this application do not limit this scope.

[0148] comprehensive Figures 11 to 14 Analysis at a wavelength of approximately 1300 nm showed good differentiation between different tissues in samples of hollow organs. Tumor tissue in solid organs (such as the breast, kidney, and lung) also showed good differentiation from surrounding normal and adipose tissue.

[0149] To illustrate, taking colon cancer as an example, in a 1300nm hyperspectral image observed with the naked eye, tumor tissue appears gray, normal muscle tissue appears as a darker gray-black than tumor tissue, adipose tissue appears grayish-white, and normal mucosa appears as a dark gray that is lighter than the muscle layer but slightly darker than the tumor tissue. The 1300nm hyperspectral image shows good differentiation between adipose tissue, muscle layer, and tumor tissue.

[0150] In an optional embodiment, three peaks and troughs (1100nm, 1300nm, and 1450nm) from the hyperspectral image are extracted as characteristic bands to synthesize a short-wave infrared color composite image. This provides a pseudo-color image that better matches human visual perception, facilitating the identification of different tissues by doctors. In the short-wave infrared color composite image, cancerous tissue appears orange, muscle tissue appears a darker orange than tumor tissue, normal mucosa appears a lighter orange than cancerous tissue, and adipose tissue appears bright yellow.

[0151] Step 850: Based on the difference representation of element type, the sample image is divided into regions to determine the region division result corresponding to the sample image.

[0152] As an illustration, after obtaining the spectral analysis results, the image segmentation model provides corresponding regional information prompts on the sample image. The regional information prompts include at least one of the following methods.

[0153] (1) Outline hints

[0154] In a schematic way, contour lines are used to divide different regions in a sample image to obtain different regions. The contour lines can be dark curves or colored curves, etc.

[0155] (2) Heat map prompts

[0156] The tumor tissue area is indicated schematically using a special highlighting method.

[0157] (3) Single color fill hint

[0158] Indicative, such as Figure 15 As shown, different areas are distinguished by different fill colors; for example, tumor tissue areas are filled with red (1510), and adipose tissue areas are filled with green (1520). Optionally, areas that cannot be accurately defined are filled with white or left unfilled.

[0159] In an optional embodiment, such as Figure 16 As shown, deep learning is performed based on the short-wave infrared hyperspectral image 1610 (sample image) and the labeled WSI 1620 to finally obtain the prediction result 1630 after predicting the short-wave infrared hyperspectral image 1610. Schematic, the prediction result 1630 is indicated by a single-color fill method. The above is only an illustrative example, and the embodiments of this application are not limited thereto.

[0160] In summary, based on predetermined wavelengths, target samples are acquired to obtain sample images. At least one target wavelength with good performance is selected from the predetermined wavelengths, and a target image corresponding to that wavelength is determined from the sample images. After processing the target image, a pseudo-color image is obtained, which accurately reflects the advantages of the target wavelength. Based on the differences in sample element types within the sample image, the image is divided into regions to obtain region division results. Combining the pseudo-color image with the region division results, target regions including the target element types are determined, thereby identifying the location information of the region to be identified (e.g., tumor tissue). This method avoids relying solely on the doctor's naked-eye observation and description to judge the size and region of tumor tissue, reducing the difficulty of pathological sampling. It is not only simple to operate but also relatively low in cost.

[0161] In this embodiment, the training and application processes of the region segmentation model are described. When training the region segmentation model, a full-view digital image is used as the standard image. The untrained candidate segmentation models are trained using the standard image until convergence is achieved, resulting in an image segmentation model. This model is then used to segment sample images into regions. Based on the differences in element types within the sample images, the corresponding region segmentation result is determined. Through this method, the model learns about the excised tissue lesion region. The model automatically identifies lesion regions and other special regions, and the target regions are specially marked on the model's output image using region information prompts. This allows for better image analysis and improves the accuracy of pathological sampling.

[0162] In an optional embodiment, the above image processing method is applied to the medical field to process pathological images. After obtaining pathological images with infrared hyperspectral information corresponding to different parts, narrowband synthetic pseudo-color images are combined with deep learning to predict the lesion area of ​​the excised tissue, thereby providing a new solution for intraoperative tumor edge determination and postoperative auxiliary pathological sampling. Illustratively, the above image processing method is applied to at least two of the following identification processes: (i) identifying tumor tissue in hollow organs; (ii) identifying tumor tissue in solid organs.

[0163] (I) Identification of tumor tissue in hollow organs

[0164] Hollow organs refer to tubular organs with a large internal space, such as the stomach, intestines, bladder, and gallbladder. Solid organs, in contrast to hollow organs, include the heart, lungs, kidneys, liver, spleen, and so on. The difference is that the former are solid while the latter are hollow. For illustration, solid abdominal organs include the liver, spleen, kidneys, adrenal glands, and pancreas; hollow abdominal organs include the gallbladder, stomach, duodenum, jejunum, ileum, appendix, and colon.

[0165] In an optional embodiment, colon cancer tissue, rectal cancer tissue, gastric cancer tissue, and esophageal cancer tissue in hollow organs were studied. HSI 1300nm showed good discrimination among the four different tumor tissues, and the imaging colors were similar.

[0166] Compared to X-ray images, hyperspectral imaging shows a significant advantage in identifying the muscle layer of hollow organs. Hyperspectral images are significantly clearer than conventional color images when determining tumor boundaries. A color image synthesized from carefully selected 1100nm, 1300nm, and 1450nm HSI images clearly shows the extent of tumor tissue, with different tissues exhibiting colors ranging from yellow to orange in varying intensities.

[0167] Indicative, such as Figure 17 The image shows the classification of four tissues selected from hollow organs. Sample 1 is colon cancer tissue, sample 2 is rectal cancer tissue, sample 3 is gastric cancer tissue, and sample 4 is esophageal cancer tissue.

[0168] Among them, Figures 1710 to 1740 shown in the first row are used to indicate conventional color images taken with an ordinary camera (equivalent to observation by the naked eye);

[0169] Figures 1711 to 1741 shown in the second row are used to indicate X-ray images obtained using X-ray equipment, which can show the general outline of the tumor area, but the effect is not clear and the muscle layer structure cannot be distinguished.

[0170] Figures 1712 to 1742 in the third row are used to indicate hyperspectral images (HSI 1300nm images) with a wavelength of 1300nm acquired using a hyperspectral camera. The hyperspectral images are grayscale images, and different tissues show different shades of color, which can be distinguished by the naked eye.

[0171] Figures 1713 to 1743 shown in the fourth row are used to indicate pseudo-color images synthesized using hyperspectral images with a wavelength of 1100 nm, a hyperspectral image with a wavelength of 1300 nm, and a hyperspectral image with a wavelength of 1450 nm.

[0172] Figures 1714 to 1744 in the fifth row are used to indicate AI-segmented images, such as the output images obtained using the region segmentation model described above. These images can provide more detailed material information. For example, color A represents tumor tissue, color B represents muscle layer tissue, color C represents normal mucosal tissue, and color D represents adipose tissue. Optionally, the darker the color, the higher the confidence level.

[0173] Figures 1715 to 1745 shown in the sixth row indicate WSI images (gold standard) used to show the true extent of tumor tissue.

[0174] (II) Identification of solid organ tumor tissue

[0175] In an alternative embodiment, the kidney, lung, and breast are studied as solid organs with lesions (such as tumor tissue), where visual identification is not difficult in relatively large samples of tumor tissue. (Illustrative, as...) Figure 18 The image shown represents different image formats in renal cell carcinoma.

[0176] Figure 1810 shows a standard color image (visual observation) of kidney tissue. In this image, tumor tissue appears grayish-white, adipose tissue appears yellow, and normal kidney tissue appears light brown. Figure 1811 shows a magnified view of the tumor boundary in the standard color image. In the magnified image, the boundary between the tumor tissue and normal kidney tissue is difficult to distinguish.

[0177] Figure 1820 shows an X-ray image acquired using an X-ray device. While the X-ray image shows the outline of the tumor, the boundaries are not clearly defined, and it is difficult to distinguish between normal kidney tissue and tumor tissue.

[0178] Figure 1830 indicates a hyperspectral image at a wavelength of 1300 nm; Figure 1831 indicates a magnified schematic diagram of the tumor boundary after magnifying the hyperspectral image at a wavelength of 1300 nm. In the 1300 nm hyperspectral image, the tumor tissue appears grayish-white, the normal kidney tissue appears gray, and the adipose tissue appears bright grayish-white. In the magnified image, the tumor tissue has a relatively clear boundary with the surrounding normal tissue.

[0179] Figure 1840 is used to indicate the pseudo-color image; Figure 1832 is used to indicate a schematic diagram of the tumor boundary after magnification of the synthesized pseudo-color image.

[0180] As an illustration, a pseudo-color image can be obtained by colorizing a single hyperspectral image corresponding to a single wavelength, or by synthesizing and colorizing multiple hyperspectral images corresponding to multiple wavelengths. The wavelength selection can be random or predetermined. As an illustration, based on experimental results, at least one wavelength with good pre-selected performance is used to obtain a pseudo-color image from the hyperspectral image corresponding to that at least one wavelength.

[0181] For example: A pseudo-color image is obtained by pre-selecting a wavelength of 1300nm that yields good results and then colorizing the hyperspectral image corresponding to that wavelength; or, a pseudo-color image is obtained by randomly selecting a wavelength of 1250nm and then colorizing the hyperspectral image corresponding to that wavelength. Alternatively, a pseudo-color image is obtained by pre-selecting three wavelengths that yield good results: 1300nm, 1100nm, and 1450nm, and then combining and colorizing the hyperspectral images corresponding to each wavelength.

[0182] In short-infrared composite color images, renal cell carcinoma tissue appears orange-yellow, adipose tissue appears bright yellow, and normal renal tissue areas appear orange-yellow with a blackish tinge. In magnified images, the boundary between tumor tissue and surrounding tissue is clear and easily distinguishable.

[0183] Figure 1850 is used to illustrate AI segmentation images, used to divide sample images into regions. Schematic, different regions are distinguished using different methods, such as red for tumor tissue, green for normal kidney tissue, and yellow for adipose tissue; darker colors indicate higher confidence. In the AI ​​segmentation image, red represents tumor tissue, green for normal kidney tissue, and yellow for adipose tissue; darker colors indicate higher confidence, and the tumor outline more closely matches the tumor boundary in the WSI (Weighted Injection System).

[0184] Figure 1860 is used to indicate the contour of the WSI tumor region.

[0185] In one optional embodiment, the breast, a solid organ, is used as an example. Visually determining the boundaries of a tumor may not be easy; for example, the boundaries of tumor tissue cannot be accurately identified from photographs taken with a regular camera. Figure 19 As shown in Figure 1910, a conventional color image taken by a regular camera is used to indicate the boundary of the tumor tissue. The boundary between the tumor tissue and the surrounding tissue in this part cannot be clearly distinguished in a conventional color image.

[0186] X-ray images are effective in identifying tumor tissue and have long been used as a primary auxiliary tool in pathological sampling to help pathologists locate the tumor bed. Figure 1920 illustrates X-ray images acquired using X-ray equipment. In the case shown, the tumor outline in the X-ray image has spiky edges and is significantly larger than the tumor outline shown in WSI.

[0187] Figure 1930 is used to indicate a hyperspectral image at a wavelength of 1300 nm, in which tumor tissue appears as dark gray (corresponding to the irregular shape), and the surrounding normal breast tissue, which appears as a lighter gray, is circled.

[0188] Figure 1940 is used to indicate a pseudo-color image determined based on a hyperspectral image corresponding to at least one selected wavelength. Schematic, in the pseudo-color image synthesized in shortwave infrared, the area of ​​tumor tissue is a deeper orange compared to the surrounding breast tissue, while adipose tissue appears bright yellow.

[0189] Figure 1950 is used to indicate artificial intelligence segmentation images (the processing results of sample images through a deep learning model), providing a relatively accurate reference for the extent of tumor tissue; Figure 1960 is used to indicate full-view digital slice images (WSI is the gold standard).

[0190] Optionally, in breast cases, X-ray images obtained using X-ray equipment can show punctate calcifications. For example, illustrative... Figure 20As shown, Figure 2010 illustrates a conventional color image captured by a standard camera, where the tumor area appears grayish-white, allowing for the general identification of the tumor tissue's extent. Figure 2020 illustrates an X-ray image acquired using an X-ray device, which roughly shows the edges of the tumor tissue, appearing spiky and containing punctate calcifications (indicated by the arrows in Figure 2020). Figure 2030 illustrates a hyperspectral image at a wavelength of 1300 nm, where the tumor tissue appears dark gray, normal breast tissue is lighter in color compared to the tumor area, and adipose tissue appears grayish-white. Figure 2040 illustrates a short-wave infrared color image obtained by processing the hyperspectral image corresponding to at least one of the selected wavelengths, which displays a clearer tumor outline. Figure 2050 illustrates the result of manually segmented images. Figure 2060 illustrates a full-field digital slice (WSI) as the gold standard. Schematic, combining shortwave infrared color images and manually segmented image results, shows the identification of target regions including tumor tissue, whose outlines show the highest degree of agreement with the gold standard (WSI).

[0191] In summary, based on predetermined wavelengths, target samples are acquired to obtain sample images. At least one target wavelength with good performance is selected from the predetermined wavelengths, and a target image corresponding to that wavelength is determined from the sample images. After processing the target image, a pseudo-color image is obtained, which accurately reflects the advantages of the target wavelength. Based on the differences in sample element types within the sample image, the image is divided into regions to obtain region division results. Combining the pseudo-color image with the region division results, target regions including the target element types are determined, thereby identifying the location information of the region to be identified (e.g., tumor tissue). This method avoids relying solely on the doctor's naked-eye observation and description to judge the size and region of tumor tissue, reducing the difficulty of pathological sampling. It is not only simple to operate but also relatively low in cost.

[0192] In this application embodiment, the above-described image processing method was applied to the medical field to analyze hollow and solid organs, demonstrating its benefits. On one hand, this image processing method is more reliable than methods such as naked-eye observation and tactile examination, ensuring greater image consistency. On the other hand, the hyperspectral imaging system is non-invasive, non-contact, and emits no ionizing radiation, and its hardware cost is lower than that of X-ray equipment.

[0193] Figure 21 This is a structural block diagram of an image processing apparatus provided in an exemplary embodiment of this application, such as... Figure 21 As shown, the device includes the following parts:

[0194] The sample acquisition module 2110 is used to acquire sample images, which include images obtained by acquiring target samples within a preset band;

[0195] Image acquisition module 2120 is used to acquire a target image in the sample image that corresponds to at least one target wavelength in the preset band, and obtain a pseudo-color image;

[0196] The region segmentation module 2130 is used to segment the sample image into regions based on the differences in the sample element types in the sample image, and obtain the region segmentation result, wherein the sample element types include the target element types to be identified;

[0197] The region determination module 2140 is used to determine a target region including the target element type in the sample image based on the pseudo-color image and the region division result.

[0198] In an optional embodiment, the image acquisition module 2120 is further configured to perform color processing on the target image corresponding to the target wavelength in the preset band to obtain the pseudo-color image; or, to perform composite processing on at least two target images corresponding to at least two target wavelengths in the preset band, and to perform color processing on the composite image to obtain the pseudo-color image.

[0199] In an optional embodiment, the image acquisition module 2120 is further configured to determine at least two target images corresponding to the at least two target wavelengths, wherein the i-th target wavelength corresponds to the i-th target image, and i is a positive integer; to perform a synthesis process on the at least two target images to obtain a candidate image; and to perform a color processing on the candidate image to obtain the pseudo-color image.

[0200] In an optional embodiment, the image acquisition module 2120 is further configured to average the pixel values ​​of corresponding pixels in the at least two target images to obtain the target pixel values ​​of the corresponding pixels; and to determine the candidate images based on the target pixel values ​​corresponding to each pixel.

[0201] In an optional embodiment, the image acquisition module 2120 is further configured to classify the pixels in the candidate image based on the brightness values ​​of the pixels in the candidate image, determine at least two brightness levels, and assign colors to the at least two brightness levels respectively to obtain the pseudo-color image.

[0202] like Figure 22 As shown, in an optional embodiment, the region division module 2130 includes:

[0203] The determining unit 2131 is used to determine the difference representation of the element type by using a pre-trained image segmentation model to analyze the sample image.

[0204] The segmentation unit 2132 is used to segment the sample image into regions based on the difference representation of the element type, and determine the region segmentation result corresponding to the sample image.

[0205] In an optional embodiment, the sample image is an image with spectral information;

[0206] The determining unit 2131 is further configured to perform spectral analysis on the target image to obtain spectral analysis results; and based on the spectral analysis results, determine the difference representation of the element type corresponding to the sample image.

[0207] In an optional embodiment, the apparatus is further configured to acquire a standard image, which is a pre-labeled image with spectral information obtained by collecting data for the target sample; train a candidate segmentation model using the standard image; and obtain an image segmentation model in response to the training effect achieved by training the candidate segmentation model, the image segmentation model being used to perform region segmentation on the target image.

[0208] In an optional embodiment, the sample acquisition module 2110 is further configured to perform a push-broom acquisition operation on the target sample to obtain the sample image.

[0209] In an optional embodiment, the pushbroom acquisition operation is performed based on the acquisition device, and the sample acquisition module 2110 is further configured to determine at least one wavelength within the target band range using a tunable filter; and to perform a pushbroom acquisition operation on the target sample based on the acquisition device to acquire a sample image corresponding to the at least one wavelength.

[0210] In an optional embodiment, the region determination module 2140 is further configured to determine the overlapping region between the pseudo-color image and the region segmentation result; in the sample image, the overlapping region is used as the target region including the target element type.

[0211] In summary, based on predetermined preset wavelengths, target samples are acquired to obtain sample images. At least one target wavelength with good performance is selected from the preset wavelengths, and a target image corresponding to that wavelength is determined from the sample images. After processing the target image, a pseudo-color image is obtained, which accurately reflects the advantages of the target wavelength. Based on the differences in sample element types within the sample image, the image is divided into regions to obtain region division results. Combining the pseudo-color image with the region division results, target regions including the target element types are determined, thereby identifying the location information of the region to be identified (e.g., tumor tissue). This device avoids relying solely on doctors' naked-eye observation and description to judge the size and region of tumor tissue, reducing the difficulty of pathological sampling. It is not only simple to operate but also relatively low in cost.

[0212] It should be noted that the image processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus and the image processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0213] Figure 23 A schematic diagram of a server provided in an exemplary embodiment of this application is shown. The server 2300 includes a Central Processing Unit (CPU) 2301, a system memory 2304 including Random Access Memory (RAM) 2302 and Read Only Memory (ROM) 2303, and a system bus 2305 connecting the system memory 2304 and the CPU 2301. The server 2300 also includes a mass storage device 2306 for storing an operating system 2313, application programs 2314, and other program modules 2315.

[0214] Mass storage device 2306 is connected to central processing unit 2301 via a mass storage controller (not shown) connected to system bus 2305. Mass storage device 2306 and its associated computer-readable media provide non-volatile storage for server 2300. That is, mass storage device 2306 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drives.

[0215] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 2304 and mass storage device 2306 described above can be collectively referred to as memory.

[0216] According to various embodiments of this application, server 2300 can also be connected to a remote computer on a network, such as the Internet. That is, server 2300 can be connected to network 2312 via network interface unit 2311 connected to system bus 2305, or it can also use network interface unit 2311 to connect to other types of networks or remote computer systems (not shown).

[0217] The aforementioned memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU.

[0218] Embodiments of this application also provide a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the image processing method provided in the above-described method embodiments.

[0219] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the image processing method provided in the above-described method embodiments.

[0220] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the image processing methods described in the above embodiments.

[0221] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0222] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0223] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: Acquire sample images, which include images obtained by collecting target samples within a preset spectral band, and the sample images are images with spectral information; A pseudo-color image is obtained by acquiring a target image from the sample image that corresponds to at least one target wavelength in the preset band; wherein, when at least two target wavelengths are selected, at least two target images corresponding to the at least two target wavelengths are determined according to the at least two target wavelengths, wherein the i-th target wavelength corresponds to the i-th target image, and i is a positive integer; the at least two target images are synthesized to obtain a candidate image; based on the brightness values ​​of the pixels in the candidate image, the pixels in the candidate image are graded in brightness to determine at least two brightness levels; the at least two brightness levels are respectively colored to obtain the pseudo-color image; Based on the differences in the sample element types in the sample image, the sample image is divided into regions to obtain the region division results. The sample element types include the target element types to be identified. Based on the pseudo-color image and the region segmentation result, a target region including the target element type is determined in the sample image.

2. The method according to claim 1, characterized in that, The step of obtaining a target image from the sample image that corresponds to at least one target wavelength in the preset band, to obtain a pseudo-color image, includes: When a single target wavelength is selected, the target image corresponding to the target wavelength in the preset band is color-coded to obtain the pseudo-color image.

3. The method according to claim 1, characterized in that, The process of synthesizing the at least two target images to obtain candidate images includes: The pixel values ​​of corresponding pixels in the at least two target images are averaged to obtain the target pixel values ​​of the corresponding pixels. The candidate image is determined based on the target pixel value corresponding to each pixel.

4. The method according to any one of claims 1 to 3, characterized in that, The step of dividing the sample image into regions based on the differences in the types of sample elements in the sample image to obtain the region division result includes: The sample images are used to determine the difference representation of the element type through a pre-trained image segmentation model; Based on the difference representation of the element type, the sample image is divided into regions to determine the region division result corresponding to the sample image.

5. The method according to claim 4, characterized in that, The step of determining the difference representation of the element type by using the pre-trained image segmentation model of the sample image includes: Perform spectral analysis on the target image to obtain the spectral analysis results; Based on the spectral analysis results, the difference representation of the element type corresponding to the sample image is determined.

6. The method according to claim 5, characterized in that, The method further includes: Acquire a standard image, which is a pre-labeled image with spectral information obtained by collecting data from the target sample; The candidate segmentation model is trained using the standard image; In response to the training effect achieved by the candidate segmentation model, an image segmentation model is obtained, which is used to perform region segmentation on the target image.

7. The method according to any one of claims 1 to 3, characterized in that, The acquisition of sample images includes: A push-broom acquisition operation is performed on the target sample to obtain the sample image.

8. The method according to claim 7, characterized in that, The push-broom acquisition operation is performed based on the acquisition device; The acquisition of sample images includes: Within the preset band range, a tunable filter is used to determine at least one target wavelength; Based on the acquisition device, a push-broom acquisition operation is performed on the target sample to obtain a sample image corresponding to at least one target wavelength.

9. The method according to any one of claims 1 to 3, characterized in that, The step of determining a target region including the target element type in the sample image based on the pseudo-color image and the region segmentation result includes: Determine the overlapping region between the pseudo-color image and the region segmentation result; In the sample image, the overlapping region is taken as the target region including the target element type.

10. An image processing apparatus, characterized in that, The device includes: The sample acquisition module is used to acquire sample images, which include images obtained by collecting target samples within a preset wavelength band, and the sample images are images with spectral information. An image acquisition module is used to acquire a target image in the sample image that corresponds to at least one target wavelength in the preset band, thereby obtaining a pseudo-color image; wherein, when at least two target wavelengths are selected, at least two target images corresponding to the at least two target wavelengths are determined based on the at least two target wavelengths, wherein the i-th target wavelength corresponds to the i-th target image, and i is a positive integer; the at least two target images are synthesized to obtain a candidate image; based on the brightness values ​​of the pixels in the candidate image, the pixels in the candidate image are graded in brightness to determine at least two brightness levels; and the at least two brightness levels are respectively colored to obtain the pseudo-color image; The region segmentation module is used to segment the sample image into regions based on the differences in the sample element types in the sample image, and obtain the region segmentation result, wherein the sample element types include the target element types to be identified; The region determination module is used to determine, based on the pseudo-color image and the region segmentation result, a target region in the sample image that includes the target element type.

11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the image processing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The storage medium stores at least one program segment, which is loaded and executed by a processor to implement the image processing method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the image processing method as described in any one of claims 1 to 9.

Images