An image processing method and apparatus

Abstract

This invention discloses an image processing method and apparatus, relating to the field of computer technology. A specific embodiment of the method includes: acquiring a first image and inputting it into a detection model; cropping the first image according to the corresponding output coordinates to obtain a second image; inputting the second image into a classification model to obtain a first array of corresponding outputs and determining the target elements therein, recording the corresponding indices in a target set; inputting the second image into a segmentation model to obtain a corresponding output data matrix; retrieving a preset region enhancement model and performing corresponding enhancement processing on the second image according to the target set and data matrix to replace the cropped portion of the first image; inputting the replaced first image into a recognition model to obtain a second array of corresponding outputs and determining the target data indices therein, thereby determining the lesion nature corresponding to the first image. Thus, the embodiment of this invention can solve the technical problem of low accuracy in existing lesion recognition methods.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an image processing method and apparatus. Background Technology

[0002] Currently, lesion identification based on deep learning models is widely used, providing users with efficient and fast lesion localization and identification services.

[0003] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art:

[0004] While users enjoy efficient and fast lesion localization and identification services, current deep learning models generally have low accuracy in identifying lesions in medical images, and misjudging the nature of lesions occurs frequently. In particular, when positive lesions are misjudged as false positives, or when severe lesions are misjudged as mild cases, it may affect the attention that doctors and patients pay to the condition, thus delaying the best treatment time and causing irreversible losses. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an image processing method and apparatus that can solve the technical problem of low accuracy in existing lesion identification.

[0006] To achieve the above objectives, according to one aspect of the present invention, an image processing method is provided, comprising: acquiring a first image and inputting it into a trained detection model to crop the first image according to corresponding output coordinates to obtain a second image; inputting the second image into a trained classification model to obtain a first array of corresponding outputs, determining target elements in the first array, and recording the index of the target elements into a target set; inputting the second image into a trained segmentation model to obtain a corresponding output data matrix, and determining the corresponding region in the second image according to the coordinates of each matrix element; retrieving a preset region enhancement model, determining the corresponding enhancement method for each corresponding region in the second image according to the target set and the value of each matrix element, so as to sequentially enhance the regions included in the second image, and replacing the cropped portion in the first image with the enhanced second image; inputting the replaced first image into a trained recognition model to obtain a second array of corresponding outputs, determining the target data index in the second array, and retrieving a preset lookup table to determine the lesion nature corresponding to the first image according to the target data index.

[0007] Optionally, before acquiring the first image, the process includes: acquiring a preset medical image and cropping the medical image according to preset coordinate information to obtain a first image, wherein the first image includes at least one lesion tissue and part of its boundary.

[0008] Optionally, after cropping the first image according to the corresponding output coordinates, the process includes: obtaining a corresponding screenshot, estimating the area of ​​the screenshot, and calculating the magnification ratio corresponding to the screenshot based on a preset standard area, so as to enlarge the screenshot proportionally.

[0009] Optionally, after sequentially enhancing the regions included in the second image, the process includes: proportionally reducing the enhanced second image to the corresponding size of the screenshot, and proportionally replacing the cropped portion of the first image with the reduced second image.

[0010] Optionally, determining the target element in the first array includes: comparing and obtaining the maximum value in the first array, filtering and obtaining elements in the first array whose difference from the maximum value is less than a preset threshold, and using the maximum value and the element as the target element.

[0011] Optionally, determining the target data sequence number in the second array includes: comparing and obtaining the maximum value in the second array, and using the sequence number corresponding to the maximum value as the target data sequence number.

[0012] Optionally, it further includes: if the target set includes multiple serial numbers, then the second image corresponds to multiple enhancement methods, and then the second image is sequentially enhanced multiple times to obtain multiple enhanced second images, and the multiple enhanced second images are sequentially used to replace the cropped area in the first image in proportion to obtain multiple replaced first images.

[0013] Optionally, after obtaining multiple replaced first images, the method further includes: sequentially inputting the multiple replaced first images into a trained recognition model to obtain multiple corresponding output second arrays; determining the sequence number of each second array in the target set; reading the element corresponding to the sequence number in both the second array and the first array; determining whether the element corresponding to the second array is a large value; if not, deleting the sequence number in the target set to update the target set; reading the sequence numbers included in the updated target set; and retrieving a preset lookup table to determine the lesion nature corresponding to the first image based on the sequence number.

[0014] In addition, the present invention also provides an image processing apparatus, including a processing module for acquiring a first image and inputting it into a trained detection model to crop the first image according to the corresponding output coordinates to obtain a second image; for inputting the second image into a trained classification model to obtain a first array of corresponding outputs, determining the target elements in the first array, and recording the index of the target elements into a target set; for inputting the second image into a trained segmentation model to obtain a corresponding output data matrix, and determining the corresponding region in the second image according to the coordinates of each matrix element; including a region enhancement module for retrieving a preset region enhancement model, determining the corresponding enhancement method for each corresponding region in the second image according to the target set and the value of each matrix element, so as to sequentially enhance the regions included in the second image, and replace the cropped part in the first image with the enhanced second image; and including a recognition module for inputting the replaced first image into a trained recognition model to obtain a second array of corresponding outputs, determining the target data index in the second array, and retrieving a preset lookup table to determine the lesion nature corresponding to the first image according to the target data index.

[0015] One embodiment of the above invention has the following advantages or beneficial effects: The present invention acquires a preset medical image and crops the medical image according to preset coordinate information to obtain a first image, thus filtering the normal tissue region in the medical image and locking the region of interest; furthermore, by inputting the first image into a trained detection model and cropping the first image according to the corresponding output coordinates to obtain a second image, all lesions included in the first image are detected, and a second image corresponding to each lesion is obtained, laying the groundwork for predicting the lesion nature of the multiple lesions one by one in subsequent operations; simultaneously, by inputting the second image into a trained classification model to obtain a corresponding output first array, the target element in the first array is determined, and the index of the target element is recorded into the target set, completing the preliminary prediction of the lesions included in each second image; furthermore, by inputting the second image into a trained segmentation model to obtain corresponding output data... A matrix is ​​used to determine the corresponding region in the second image based on the coordinates of each matrix element, thus achieving the effect of describing the morphological features of the lesion tissue included in the second image through the matrix. In addition, by inputting the second image into the segmentation model, a corresponding output data matrix is ​​obtained. A preset region enhancement model is invoked, and the second image is enhanced accordingly based on the target set and the data matrix. The cropped part in the first image is replaced, completing the differentiated region enhancement processing based on the lesion nature corresponding to each lesion tissue. The processed image is then used to replace the cropped unprocessed part in the first image, achieving the effect of increasing the display difference between the lesion tissue region and the normal tissue region in the first image. Finally, by inputting the replaced first image into the trained recognition model, a corresponding output second array is obtained, and the target data sequence number in the second array is determined. A preset lookup table is invoked to determine the lesion nature corresponding to the first image based on the target data sequence number, thus completing the final prediction of the lesions included in the first image.

[0016] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0017] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0018] Figure 1 This is a schematic diagram of the main flow of the image processing method according to the first embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the main flow of the image processing method according to the second embodiment of the present invention;

[0020] Figure 3This is a schematic diagram of the main flow of the image processing method according to the third embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of the main modules of an image processing apparatus according to a first embodiment of the present invention;

[0022] Figure 5 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0023] Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0024] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] Figure 1 This is a schematic diagram of the main flow of the image processing method according to the first embodiment of the present invention, as shown below. Figure 1 As shown, the image processing method includes:

[0026] Step S101: Obtain the first image and input it into the trained detection model to crop the first image according to the corresponding output coordinates to obtain the second image.

[0027] In some embodiments, to accurately locate the region of interest in a preset medical image, a preset medical image can be acquired before acquiring the first image. This medical image is then cropped according to preset coordinate information to obtain the first image, wherein the first image includes at least one lesion and part of its boundary. This step removes most of the area corresponding to normal tissue in the preset medical image while increasing the proportion of the lesion in the medical image, thereby effectively increasing the accuracy of lesion prediction. Simultaneously, this step crops the medical image with unknown pixel ratios into a first image corresponding to the input ratio of the detection model, making the subsequent operation of inputting the first image into the detection model smoother.

[0028] In one embodiment, the trained detection model can locate multiple lesions in the input first image and output multiple sets of coordinates. Based on each set of coordinates, a corresponding second image can be extracted from the first image, resulting in multiple second images corresponding to the multiple lesions in the first image. This lays the groundwork for subsequent preliminary prediction processing (i.e., inputting the data into a classification model) of the second image corresponding to each lesion. The detection model can be a deep learning model of any structure, such as a CNN, RNN, or GAN model. In a further embodiment, to obtain a trained detection model, a training set can be constructed and input into the detection model. The model is trained with the goal of making the output of the training set approximate the coordinates of the lesions in the training set.

[0029] In some embodiments, to further improve the accuracy of lesion identification in the second image, a screenshot can be obtained after cropping the first image according to the corresponding output coordinates. The area of ​​the screenshot is estimated, and the magnification ratio corresponding to the screenshot is calculated according to a preset standard area to enlarge the screenshot proportionally. This step unifies the pixel ratio of the second image, ensuring that each second image can be successfully input into the trained classification model and obtain the corresponding output.

[0030] Step S102: Input the second image into the trained classification model to obtain the first array of corresponding outputs, determine the target element in the first array, and record the index of the target element into the target set.

[0031] In this embodiment, the ordered array output by the classification model, which consists of multiple preset lesion properties corresponding to probability values, typically selects the lesion property corresponding to the highest probability value (i.e., the target element) as the lesion property corresponding to the input second image. These preset lesion properties are recorded in a preset lookup table, each with a unique corresponding index, which is the same as the index corresponding to the element in the first array. The preset lesion properties can be descriptions of lesion types and their severity, or simply descriptions of lesion types, depending on the specific lesion type. The classification model can be a deep learning model of any structure, such as a CNN model, an RNN model, or a GAN model. In a further embodiment, to obtain a trained classification model, a training set can be constructed and input into the classification model. The goal is to train the classification model so that the output corresponding to the training set approximates the corresponding lesion type of the lesion tissue in the training set.

[0032] In some embodiments, to avoid the classification model misjudging the second image as corresponding to lesion tissue, when determining the target element in the first array, the maximum value in the first array can be compared, and elements in the first array whose difference from the maximum value is less than a preset threshold can be selected. The maximum value and the selected element are then used as the target element. This step preserves the lesion properties corresponding to multiple higher probability values ​​in the classification model output, that is, it preserves multiple lesion properties that each second image may correspond to, thus avoiding the classification model misjudging the input image as containing lesion tissue.

[0033] Step S103: Input the second image into the trained segmentation model to obtain the corresponding output data matrix, and determine the corresponding region in the second image based on the coordinates of each matrix element.

[0034] In an embodiment, the data matrix output by the segmentation model includes matrix elements that correspond to the morphological features of each region after the input second image has been segmented into multiple regions; matrix elements with the same value in the data matrix correspond to the same structure of the lesion tissue included in the second image. For example, matrix elements with a value of "1" in the data matrix correspond to the boundary portion of the lesion tissue included in the second image. Therefore, based on the multiple coordinates corresponding to the matrix elements with a value of "1" in the data matrix, the morphology of the boundary of the lesion tissue included in the second image can be inferred. The segmentation model can be a deep learning model of any structure, such as a CNN model, an RNN model, or a GAN model, etc. In a further embodiment, in order to obtain a trained segmentation model, a training set can be constructed and input into the segmentation model, with the goal of making the output corresponding to the training set approximate the morphological features of the lesion tissue in the training set, and the segmentation model can be trained.

[0035] Step S104: Retrieve the preset region enhancement model, determine the corresponding enhancement method for each corresponding region in the second image based on the target set and the value of each matrix element, so as to sequentially enhance the regions included in the second image, and replace the cropped part in the first image with the enhanced second image.

[0036] In this embodiment, the second image is first segmented based on the data matrix output by the segmentation model. The corresponding region for each matrix element in the second image is determined, and an enhancement method is determined based on the matrix element and the index of the target set to enhance the region. For example, if the matrix element is "1", it indicates that the corresponding region belongs to the boundary of the lesion tissue, and the index of the target set is "2", indicating that the corresponding lesion is mild calcification. Then, according to the preset enhancement rules, the boundary region of mildly calcified lesion tissue should be filled with a preset color number 3, thus determining the corresponding enhancement method. The above processing procedure is applied to each region of the second image to complete the overall enhancement of the second image.

[0037] In some embodiments, to ensure that the normal tissue region in the first image is not affected by the second image, after sequentially enhancing the regions included in the second image, the enhanced second image can be proportionally reduced to the corresponding size of the screenshot, and the reduced second image can be used to proportionally replace the cropped portion in the first image. This step ensures that the difference between the first image after replacement and before replacement lies only in whether or not the corresponding region of the second image has been enhanced, avoiding a decrease in the accuracy of the final lesion identification due to misaligned replacement operations.

[0038] In some embodiments, to preserve multiple highly suspected lesion properties corresponding to each second image, multiple enhancement methods corresponding to the second image can be determined based on multiple indices included in the target set. Then, the second image is sequentially enhanced multiple times to obtain multiple enhanced second images. The regions cropped in the first image are then replaced proportionally with these enhanced second images to obtain multiple replaced first images. For example, if the target set is {1, 3}, it indicates that the input second image is highly suspected of being moderate calcification (corresponding to indice 1) or a solid nodule (corresponding to indice 3). If the enhancement method corresponding to indice 1 of the target set is to fill the second image with color 11, and the enhancement method corresponding to indice 3 of the target set is to fill the second image with color 13, then the second image is filled with color 11 and color 13 respectively to obtain two filled second images. The purpose of this step is to retain multiple highly suspicious lesion characteristics obtained from the initial prediction (i.e., classification model) of each second image, so that they can be enhanced and predicted again to avoid missing any highly suspicious lesion characteristics obtained from the initial prediction (i.e., classification model), thereby avoiding the occurrence of missed screening and missed diagnosis of true positive lesions.

[0039] Step S105: Input the replaced first image into the trained recognition model to obtain the corresponding output second array, determine the target data sequence number in the second array, and retrieve the preset lookup table to determine the lesion nature corresponding to the first image based on the target data sequence number.

[0040] In some embodiments, in order to quickly obtain the lesion properties corresponding to the first image, the maximum value in the second array can be compared and the index corresponding to the maximum value can be used as the target data index.

[0041] In some embodiments, in order to comprehensively analyze the preliminary prediction results (i.e., the output of the classification model) to finally determine the lesion nature corresponding to the first image, multiple replaced first images can be obtained. These multiple replaced first images are then sequentially input into a trained recognition model to obtain multiple second arrays corresponding to the output. The index of each second array in the target set is determined. The element corresponding to the index is read from both the second array and the first array, and it is then determined whether the element corresponding to the second array is a large value. If not, the index is deleted from the target set, thereby updating the target set. The indexes included in the updated target set are read, and a preset lookup table is retrieved to determine the lesion nature corresponding to the first image based on the indexes.

[0042] For example, if the first image is input into the detection model and only one second image is captured, and the second image is input into the classification model, the resulting first array is [0, 0.51, 0, 0.49], and the corresponding target set is {2, 4}. Then, according to the lookup table, the highly suspicious lesion nature corresponding to the first image (including the second image) is determined to be either a solid nodule (corresponding to number 2) or severe calcification (number 4). The second image is enhanced according to number 2, and the first image is replaced before being input into the recognition model, resulting in the corresponding second array [0, 0.4, 0, 0.6]. Comparing this to the first array, 0.4 < 0.51, therefore... This indicates that the likelihood of the lesion tissue in the first image corresponding to a solid nodule is reduced after enhancement processing. Therefore, index 2 is deleted from the target set, and the target set is updated to {4}. The second image is enhanced according to index 4, and the first image is replaced and then input into the recognition model to obtain the corresponding second array [0, 0.45, 0, 0.55]. Comparing it with the first array, 0.55 > 0.49, indicating that the likelihood of the lesion tissue in the first image corresponding to severe calcification is increased after enhancement processing. Therefore, index 4 is retained in the target set. Finally, according to the target set, the lesion nature corresponding to the first image is severe calcification.

[0043] Figure 2This is a schematic diagram of the main flow of an image processing method according to a second embodiment of the present invention, the image processing method including:

[0044] Step S201: Obtain a preset medical image and crop the medical image according to preset coordinate information to obtain a first image.

[0045] Step S202: Input the first image into the trained detection model to crop the first image according to the corresponding output coordinates to obtain a corresponding screenshot. Estimate the area of ​​the screenshot and calculate the magnification ratio corresponding to the screenshot according to the preset standard area to enlarge the screenshot proportionally to obtain the second image.

[0046] Step S203: Input the second image into the trained classification model to obtain the first array of corresponding outputs.

[0047] Step S204: Determine the maximum value in the first array as the target element, and record the index of the target element to the target set.

[0048] Step S205: Input the second image into the trained segmentation model to obtain the corresponding output data matrix, and determine the corresponding region in the second image based on the coordinates of each matrix element.

[0049] Step S206: Retrieve a preset region enhancement model, and determine the corresponding enhancement method for each corresponding region in the second image based on the target set and the value of each matrix element, so as to sequentially enhance the regions included in the second image.

[0050] Step S207: The enhanced second image is scaled down proportionally to the corresponding size of the screenshot, and the scaled-down second image is used to replace the cropped portion of the first image proportionally.

[0051] Step S208: Input the replaced first image into the trained recognition model to obtain the corresponding output second array.

[0052] Step S209: Based on the corresponding index of the maximum value in the second array, retrieve a preset lookup table to determine the nature of the lesion corresponding to the first image.

[0053] Figure 3 This is a schematic diagram of the main flow of an image processing method according to a third embodiment of the present invention, the image processing method comprising:

[0054] Step S301: Obtain a preset medical image and crop the medical image according to preset coordinate information to obtain a first image.

[0055] Step S302: Input the first image into the trained detection model to crop the first image according to the corresponding output coordinates to obtain a corresponding screenshot. Estimate the area of ​​the screenshot and calculate the magnification ratio corresponding to the screenshot according to the preset standard area to enlarge the screenshot proportionally to obtain the second image.

[0056] Step S303: Input the second image into the trained classification model to obtain the first array of corresponding outputs.

[0057] Step S304: Compare and obtain the maximum value in the first array, and filter out the elements in the first array whose difference from the maximum value is less than a preset threshold. Take the maximum value and the element as the target element, and record multiple serial numbers corresponding to the target element to the target set.

[0058] Step S305: Input the second image into the trained segmentation model to obtain the corresponding output data matrix, and determine the corresponding region in the second image based on the coordinates of each matrix element.

[0059] Step S306: Retrieve a preset region enhancement model, determine multiple enhancement methods corresponding to the second image based on the multiple elements included in the target set, and then determine the corresponding region in the second image based on each matrix element in the data matrix to perform enhancement processing on the second image, thereby obtaining multiple enhanced second images.

[0060] Step S307: The regions cropped in the first image are replaced by multiple enhanced second images that are scaled down and proportionally replaced in turn, resulting in multiple replaced first images.

[0061] Step S308: The multiple replaced first images are sequentially input into the trained recognition model to obtain multiple corresponding output second arrays.

[0062] Step S309: Determine the index of each second array in the target set, read the element corresponding to the index in the second array and the first array respectively, and then determine whether the element corresponding to the second array is a larger value. If not, delete the index in the target set, thereby updating the target set.

[0063] Step S310: Read the serial numbers included in the updated target set, and retrieve a preset lookup table to determine the lesion nature corresponding to the first image based on the serial numbers.

[0064] Figure 4 This is a schematic diagram of the main modules of an image processing apparatus according to an embodiment of the present invention, such as... Figure 4As shown, the image processing device 400 includes a processing module 401, a region enhancement module 402, and a recognition module 403. The processing module 401 is used to acquire a first image and input it into a trained detection model to crop the first image according to the corresponding output coordinates to obtain a second image; it is used to input the second image into a trained classification model to obtain a first array of corresponding outputs, determine the target elements in the first array, and record the index of the target elements into a target set; it is used to input the second image into a trained segmentation model to obtain a corresponding output data matrix, and determine the corresponding region in the second image according to the coordinates of each matrix element; the region enhancement module 402 is used to call a preset region enhancement model, determine the corresponding enhancement method for each corresponding region in the second image according to the target set and the value of each matrix element, so as to sequentially enhance the regions included in the second image, and replace the cropped part in the first image with the enhanced second image; the recognition module 403 is used to input the replaced first image into a trained recognition model to obtain a second array of corresponding outputs, determine the target data index in the second array, and call a preset lookup table to determine the lesion nature corresponding to the first image according to the target data index.

[0065] In some embodiments, before the processing module 401 acquires the first image, it is further configured to: acquire a preset medical image and crop the medical image according to preset coordinate information to obtain a first image, wherein the first image includes at least one lesion tissue and part of its boundary.

[0066] In some embodiments, after the processing module 401 crops the first image according to the corresponding output coordinates, it is further configured to: obtain the corresponding screenshot, estimate the area of ​​the screenshot, calculate the magnification ratio corresponding to the screenshot according to the preset standard area, so as to enlarge the screenshot proportionally.

[0067] In some embodiments, after the region enhancement module 402 sequentially enhances the regions included in the second image, it is further configured to: proportionally reduce the enhanced second image to the corresponding size of the screenshot, and proportionally replace the cropped portion in the first image with the reduced second image.

[0068] In some embodiments, the processing module 401 determines the target element in the first array and is further configured to: compare and obtain the maximum value in the first array, and filter and obtain elements in the first array whose difference from the maximum value is less than a preset threshold, and use the maximum value and the element as the target element.

[0069] In some embodiments, the identification module 403 determines the target data sequence number in the second array and is further configured to: compare and obtain the maximum value in the second array, and use the sequence number corresponding to the maximum value as the target data sequence number.

[0070] In some embodiments, the region enhancement module 402 is further configured to: if the target set includes multiple serial numbers, then the second image corresponds to multiple enhancement methods, and then perform multiple enhancement processes on the second image in sequence to obtain multiple enhanced second images, and then use the multiple enhanced second images in sequence to replace the cropped region in the first image in proportion to obtain multiple replaced first images.

[0071] In some embodiments, after the recognition module 403 obtains multiple replaced first images, it is further configured to: input the multiple replaced first images sequentially into a trained recognition model to obtain multiple second arrays corresponding to the output; determine the sequence number of each second array in the target set; read the element corresponding to the sequence number in the second array and the first array respectively; and then determine whether the element corresponding to the second array is a large value. If not, delete the sequence number in the target set to update the target set; read the sequence number included in the updated target set; and retrieve a preset lookup table to determine the lesion nature corresponding to the first image based on the sequence number.

[0072] Figure 5 An exemplary system architecture 500 is shown that can be applied to the image processing method or image processing apparatus of the present invention.

[0073] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, and 503, a network 504, and a server 505. Network 504 serves as the medium for providing communication links between terminal devices 501, 502, and 503 and server 505. Network 504 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0074] Users can use terminal devices 501, 502, and 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, and 503.

[0075] Terminal devices 501, 502, and 503 can be various electronic devices with image processing screens and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0076] Server 505 can be a server that provides various services, such as a backend management server that supports users using terminal devices 501, 502, and 503 (for example only). The backend management server can analyze and process data such as received product information query requests, and feed back the processing results (such as target push information, product information - for example only) to the terminal device.

[0077] It should be noted that the image processing method provided in the embodiments of the present invention is generally executed by server 505, and correspondingly, the computing device is generally located in server 505.

[0078] It should be understood that Figure 5 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0079] The following is for reference. Figure 6 It shows a schematic diagram of the structure of a computer system 600 suitable for implementing a terminal device of the present invention. Figure 6 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0080] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the computer system 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0081] The following components are connected to I / O interface 605: an input section 605 including a keyboard, mouse, etc.; an output section 606 including a cathode ray tube (CRT), liquid crystal image processor (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0082] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined above in the system of this invention.

[0083] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0084] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0085] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor can be described as including a processing module, a region enhancement module, and a recognition module. The names of these modules do not necessarily limit the functionality of the module itself.

[0086] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist alone and not assembled into the device. The aforementioned computer-readable medium carries one or more programs. When one or more of these programs are executed by a device, the device includes: acquiring a first image and inputting it into a trained detection model to crop the first image according to the corresponding output coordinates to obtain a second image; inputting the second image into a trained classification model to obtain a first array of corresponding outputs, determining the target elements in the first array, and recording the index of the target elements into a target set; inputting the second image into a trained segmentation model to obtain a corresponding output data matrix, and determining the corresponding region in the second image according to the coordinates of each matrix element; retrieving a preset region enhancement model, determining the corresponding enhancement method for each corresponding region in the second image according to the target set and the value of each matrix element, sequentially enhancing the regions included in the second image, and replacing the cropped portion of the first image with the enhanced second image; inputting the replaced first image into a trained recognition model to obtain a second array of corresponding outputs, determining the target data index in the second array, and retrieving a preset lookup table to determine the lesion nature corresponding to the first image according to the target data index.

[0087] The technical solution of the present invention can solve the technical problem of low accuracy in lesion identification.

[0088] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image processing method, characterized by, The method comprises the following steps: acquiring a first image to input into a trained detection model to crop the first image according to corresponding output coordinates to obtain a second image; inputting the second image into a trained classification model to obtain a corresponding output first array, determining a target element in the first array, and recording a serial number of the target element to a target set; inputting the second image into a trained segmentation model to obtain a corresponding output data matrix, and determining a corresponding region in the second image according to the coordinates of each matrix element; calling a preset region enhancement model to determine a corresponding enhancement mode of each corresponding region in the second image according to the target set and the numerical value of each matrix element, to sequentially enhance the regions included in the second image, and replace the cropped part in the first image with the second image after enhancement processing; inputting the replaced first image into a trained recognition model to obtain a corresponding output second array, determining a target data serial number in the second array, calling a preset lookup table to determine the lesion property corresponding to the first image according to the target data serial number.

2. The method of claim 1, wherein, Before acquiring the first image, the method comprises the following steps: acquiring a preset medical image and cropping the medical image according to preset coordinate information to obtain a first image, wherein the first image at least includes a lesion tissue and part of the boundary thereof.

3. The method of claim 1, wherein, After cropping the first image according to the corresponding output coordinates, the method comprises the following steps: obtaining a corresponding screenshot, estimating the area of the screenshot, calculating a magnification ratio corresponding to the screenshot according to a preset standard area, and performing equal-ratio magnification on the screenshot.

4. The method of claim 3, wherein, After sequentially enhancing the regions included in the second image, the method comprises the following steps: equally reducing the second image after enhancement processing to a corresponding size of the screenshot, and equally replacing the cropped part in the first image with the second image after reduction.

5. The method of claim 1, wherein, Determining the target element in the first array comprises the following steps: comparing the maximum value in the first array to obtain, and screening the elements in the first array whose difference from the maximum value is less than a preset threshold to obtain the target element.

6. The method of claim 1, wherein, Determining the target data serial number in the second array comprises the following steps: comparing the maximum value in the second array to obtain, and taking the serial number corresponding to the maximum value as the target data serial number.

7. The method of claim 1, wherein, The method comprises the following steps: if the target set includes multiple serial numbers, the second image corresponds to multiple enhancement modes, and then multiple enhancement processing is sequentially performed on the second image to obtain multiple second images after enhancement processing, and multiple first images after replacement are obtained by sequentially using the multiple second images after enhancement processing to equally replace the cropped regions in the first image.

8. The method of claim 7, wherein, After obtaining the multiple first images after replacement, the method comprises the following steps: inputting the multiple first images after replacement into a trained recognition model to obtain multiple second arrays corresponding to the output, determining the serial number corresponding to each second array in the target set, reading the elements corresponding to the serial number in the second array and the first array, and then judging whether the elements corresponding to the second array are greater values, and if not, deleting the serial number in the target set to update the target set. reading a serial number included in the updated target set, and calling a preset lookup table to determine the lesion property corresponding to the first image according to the serial number.

9. An image processing apparatus characterized by comprising: The method comprises the steps of: a processing module configured to obtain a first image, input the first image into a trained detection model, and obtain a second image by intercepting the first image according to corresponding output coordinates; a module configured to input the second image into a trained classification model, obtain a first array corresponding to an output, determine a target element in the first array, and record a serial number of the target element in a target set; a module configured to input the second image into a trained segmentation model, obtain a data matrix corresponding to an output, and determine a corresponding region in the second image according to coordinates of each matrix element; a region enhancement module configured to call a preset region enhancement model, determine a corresponding enhancement mode of each corresponding region in the second image according to the target set and a numerical value of each matrix element, sequentially perform enhancement processing on regions included in the second image, and replace a portion intercepted in the first image with the second image after the enhancement processing; an identification module configured to input the first image after the replacement into a trained identification model, obtain a second array corresponding to an output, determine a target data serial number in the second array, call a preset lookup table to determine a lesion property corresponding to the first image according to the target data serial number.

10. An electronic device, comprising: The method comprises the steps of: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-8.

11. A computer readable medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method according to any one of claims 1-8.

Images