Image recognition method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to the technical field of image processing, and provides an image recognition method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: obtaining a to-be-recognized microscopic image; performing shear wave transformation on the microscopic image, and determining an amplitude image and a phase image of the microscopic image based on the transformed image; inputting the microscopic image, the amplitude image and the phase image into a trained image recognition model to obtain a lesion area in the microscopic image output by the image recognition model and a lesion degree of the lesion area; wherein the image recognition model is obtained by training based on a sample microscopic image, a lesion area label and a lesion degree label. According to the image recognition method and device, the electronic equipment, the storage medium and the program product provided by the invention, the recognition accuracy and reliability of the lesion area and the lesion degree can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image recognition method, device, electronic equipment, storage medium and program product. Background Art

[0002] Cancer has always been one of the diseases that seriously threaten human health worldwide. The gold standard for cancer detection is still through tissue biopsy, which is imaged under a microscope after sampling, staining, freezing, slicing, etc. Doctors judge whether cancer has occurred and judge cancer by the morphological changes of the cells themselves or the morphological changes of multiple cell structures in the image. Doctors' own judgment and efficiency of reading images are limited. The development of deep learning makes it possible to use computers to assist in judgment.

[0003] In related technologies, deep learning models can be used to predict the lesion area and degree of lesions in microscopic images, but the accuracy and reliability of the prediction results need to be improved. Summary of the invention

[0004] The present invention provides an image recognition method, device, electronic device, storage medium and program product, which are used to solve the defects of poor accuracy and reliability of prediction results in the prior art.

[0005] The present invention provides an image recognition method, comprising: Acquiring a microscopic image to be identified; Performing shear wave transformation on the microscopic image, and determining an amplitude image and a phase image of the microscopic image based on the transformed image; Inputting the microscopic image, the amplitude image and the phase image into a trained image recognition model to obtain the lesion area in the microscopic image output by the image recognition model and the lesion degree of the lesion area; The image recognition model is trained based on sample microscopic images, lesion area labels and lesion degree labels.

[0006] According to the image recognition method provided by the present invention, the image recognition model includes a convolutional neural network, a long short-term memory recurrent neural network, a deep convolutional neural network and a prediction layer, the microscopic image, the amplitude image and the phase image are input into the trained image recognition model, and the lesion area and lesion degree output by the image recognition model are obtained, including: Inputting the microscopic image into the convolutional neural network and the deep convolutional neural network respectively, obtaining image features output by the convolutional neural network and depth features output by the deep convolutional neural network; Inputting the amplitude image and the phase image into the long short-term memory recurrent neural network respectively, to obtain the amplitude features and the phase features respectively output by the long short-term memory recurrent neural network; The image features, depth features, amplitude features and phase features are input into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

[0007] According to the image recognition method provided by the present invention, the image recognition model further includes a fully connected layer, and the image features, depth features, amplitude features and phase features are input to the prediction layer to obtain the lesion area and lesion degree output by the prediction layer, including: Inputting the image features, depth features, amplitude features and phase features into the fully connected layer to obtain fusion features output by the fully connected layer; The fused features are input into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

[0008] According to the image recognition method provided by the present invention, the step of obtaining a microscopic image to be recognized comprises: Acquire original microscopic images; The original microscopic image is segmented, and each image block obtained by segmentation is used as the microscopic image to be identified.

[0009] According to the image recognition method provided by the present invention, the sample microscopic image is determined based on the following steps: Obtain original sample microscopic images; The original sample microscopic image is segmented to obtain sample blocks, and each sample block is preprocessed to obtain the sample microscopic image. The preprocessing includes at least one of block value normalization, rotation, flipping, and scaling.

[0010] The present invention also provides an image recognition device, comprising: An image acquisition unit, used for acquiring a microscopic image to be identified; An image transformation unit, configured to perform shear wave transformation on the microscopic image, and determine an amplitude image and a phase image of the microscopic image based on the transformed image; An image recognition unit, used for inputting the microscopic image, the amplitude image and the phase image into a trained image recognition model, and obtaining the lesion area in the microscopic image output by the image recognition model, and the lesion degree of the lesion area; The image recognition model is trained based on sample microscopic images, lesion area labels and lesion degree labels.

[0011] According to the image recognition device provided by the present invention, the image recognition model includes a convolutional neural network, a long short-term memory recurrent neural network, a deep convolutional neural network and a prediction layer, and the image recognition unit includes: A first feature extraction unit, used to input the microscopic image into the convolutional neural network and the deep convolutional neural network respectively, to obtain image features output by the convolutional neural network and depth features output by the deep convolutional neural network; A second feature extraction unit is used to input the amplitude image and the phase image into the long short-term memory recurrent neural network respectively, to obtain the amplitude feature and the phase feature respectively output by the long short-term memory recurrent neural network; The prediction unit is used to input the image features, depth features, amplitude features and phase features into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned image recognition methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements any of the above-mentioned image recognition methods.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image recognition method described above is implemented.

[0015] The image recognition method, device, electronic device, storage medium and program product provided by the present invention perform shear wave transform on the microscopic image, determine the amplitude image and phase image of the microscopic image based on the transformed image, and identify the lesion area and degree of the lesion through a trained image recognition model based on the microscopic image, amplitude image and phase image, thereby improving the accuracy and reliability of recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 It is one of the flow charts of the image recognition method provided by the present invention.

[0018] Figure 2 It is a schematic diagram of the structure of the convolutional neural network in the present invention.

[0019] Figure 3 It is a schematic diagram of the structure of the long short-term memory recurrent neural network in the present invention.

[0020] Figure 4 This is the second flow chart of the image recognition method provided by the present invention.

[0021] Figure 5 It is a structural schematic diagram of the image recognition device provided by the present invention.

[0022] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In related technologies, deep learning models can be used to predict the lesion area and degree of lesions in microscopic images, but the accuracy and reliability of the prediction results need to be improved.

[0025] To address this problem, an embodiment of the present invention proposes an image recognition method, in which a microscopic image to be recognized is subjected to shear wave transformation, and an amplitude image and a phase image of the microscopic image are determined based on the transformed image; the microscopic image, the amplitude image, and the phase image are input into a trained image recognition model to obtain the lesion area in the microscopic image output by the image recognition model, and the degree of lesion in the lesion area. This method can improve the reliability of identifying the lesion area and the degree of lesion in the microscopic image.

[0026] The embodiments of the present invention can be applied to scenarios where it is necessary to identify the lesion area and degree of lesion in a microscopic image. The execution subject of the method can be an electronic device such as a terminal device, a computer, a server, a server cluster, or a specially designed image recognition device, or an image recognition device set in the electronic device, which can be implemented by software, hardware, or a combination of the two.

[0027] Figure 1 It is one of the flow charts of the image recognition method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step 110: Acquire a microscopic image to be identified.

[0028] Specifically, the microscopic image to be identified is the microscopic image that needs to be identified for the lesion area and lesion degree. The tissue biopsy sample is observed using a microscope, and the microscopic image is captured by an image acquisition device (such as a digital camera) to ensure that the image is clear and has no blur or jitter. The collected microscopic image is saved in a computer or server for subsequent processing and analysis.

[0029] Here, the microscopic image to be identified may be an original image directly acquired, or an image obtained by preprocessing the original image. The preprocessing may be operations such as image segmentation, denoising, and standardization.

[0030] Step 120 , performing shear wave transform on the microscopic image, and determining an amplitude image and a phase image of the microscopic image based on the transformed image.

[0031] Specifically, the Shearlet transform is a multi-scale and multi-directional transformation method. The Shearlet transform constructs basis functions through affine transformations such as scaling, shearing, and translation to capture the geometric characteristics of the image. In image processing, the Shearlet transform can more effectively represent directional features such as edges and textures in the image.

[0032] Then, the amplitude image and phase image of the microscopic image are determined based on the transformed image. The amplitude image usually refers to the part that describes the image intensity (or amplitude, brightness). It reflects the energy of the image at each point. The phase image describes the phase information of each point in the image at a specific moment. The phase is a scale that describes the relative position of a point in a periodic signal in the cycle at a specific moment, which is important for understanding the waveform changes of the image.

[0033] In the process of shearlet transform, information related to amplitude and phase can be obtained indirectly. Specifically, shearlet transform decomposes the microscopic image into a series of sub-bands (or coefficients) at different scales and directions. These sub-bands contain the characteristic information of the image at different scales and directions, including information related to amplitude and phase.

[0034] In the shearlet transform, the coefficient of each subband can be regarded as a measure of the image intensity at that scale and direction. The coefficient reflects the energy distribution of the image at different scales and directions and can be regarded as a generalized amplitude information to a certain extent.

[0035] In addition, since the shearlet transform constructs basis functions through affine transformation, these basis functions themselves contain phase information (although implicit). Therefore, when performing image processing through the shearlet transform, the phase information can be indirectly obtained through phase recovery algorithms such as phase recovery.

[0036] In other embodiments, the shear wave transformed image can be converted from the time (or space) domain to the frequency domain by Fourier transform, so that the frequency component (ie, amplitude) and phase information of the microscopic image can be directly observed.

[0037] Step 130, inputting the microscopic image, the amplitude image and the phase image into the trained image recognition model to obtain the lesion area in the microscopic image output by the image recognition model and the lesion degree of the lesion area; Among them, the image recognition model is trained based on sample microscopic images, lesion area labels and lesion degree labels.

[0038] Specifically, on the basis of obtaining the amplitude image and the phase image, the microscopic image, the amplitude image and the phase image can be used as input data and input into the image recognition model separately or combined. The image recognition model processes and analyzes the input data to identify the lesion area in the microscopic image. At the same time, the model will also evaluate the degree of lesion in the lesion area and give corresponding quantitative indicators or classification results. For example, the degree of lesion in the lesion area is divided into grades, such as grade one, grade two, grade three, etc. Furthermore, the lesion area and lesion degree information output by the model can be presented to the user in a visual manner, such as marking on the original microscopic image or generating a report document.

[0039] Since the image obtained after shearlet transform contains the characteristic information of the image at different scales and directions. The amplitude usually corresponds to the brightness or color intensity of the pixel, and the amplitude image can intuitively display the intensity change of the data. The phase usually corresponds to the color or texture information of the pixel, and the phase image can intuitively display the change direction or texture information of the data. The recognition of the lesion area and the degree of lesion based on the microscopic image, amplitude image and phase image can improve the accuracy and reliability of the recognition.

[0040] Before executing step 130, the image recognition model may be trained in advance. Specifically, the model training may be performed by the following method: First, a large number of sample microscopic images are collected, and the sample microscopic images are segmented using a segmentation model, and experts mark the lesion areas. For example, in the case of skin cancer, they can be marked as normal areas, skin abnormalities, melanoma and nevus areas, and the lesion level is marked for the lesion area.

[0041] Then, the sample microscopic image is subjected to shear wave transformation, and the transformed sample amplitude image and sample phase image are calculated. The sample microscopic image, sample amplitude image and sample phase image are input into the initial model for training. During the training process, the initial model can learn the difference characteristics between the lesion area and the non-lesion area in the sample microscopic image, and learn the mapping relationship between the lesion area and the lesion grade. The image recognition model thus trained can better distinguish the image difference between the normal area and the lesion area, as well as the image difference between different lesion grades.

[0042] The method provided by the embodiment of the present invention performs shear wave transform on the microscopic image, determines the amplitude image and phase image of the microscopic image based on the transformed image, and identifies the lesion area and the degree of the lesion based on the microscopic image, the amplitude image and the phase image through a trained image recognition model, thereby improving the accuracy and reliability of recognition.

[0043] Based on any of the above embodiments, the image recognition model includes a convolutional neural network, a long short-term memory recurrent neural network, a deep convolutional neural network and a prediction layer, and the microscopic image, the amplitude image and the phase image are input into the trained image recognition model to obtain the lesion area and the lesion degree output by the image recognition model, that is, step 130 specifically includes: Step 131, inputting the microscopic image into a convolutional neural network and a deep convolutional neural network respectively, obtaining image features output by the convolutional neural network and depth features output by the deep convolutional neural network; Step 132, inputting the amplitude image and the phase image into the long short-term memory recurrent neural network respectively, to obtain the amplitude features and the phase features respectively output by the long short-term memory recurrent neural network; Step 133, input the image features, depth features, amplitude features and phase features into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

[0044] Specifically, the image recognition model includes a convolutional neural network (CNN), a long short-term memory recurrent neural network (LSTM-RNN), a deep convolutional neural network (Visual Geometry Group Network, VGGNet), and a prediction layer (softmax function).

[0045] Figure 2 It is a schematic diagram of the structure of the convolutional neural network in the present invention, such as Figure 2 As shown in Figure 1, this structure is composed of a convolutional layer and a pooling layer. Figure 2Taking 4 groups as an example, it can also include multiple groups, the convolution kernel is 4*4, the maximum pooling layer is 2*2, and the final feature vector is 256 pixels. The microscopic image is input into the convolutional neural network to obtain the image features output by the convolutional neural network, that is, the image features are 256 pixels.

[0046] At the same time, the microscopic image can be input into a deep convolutional neural network to obtain the deep features output by the deep convolutional neural network. The deep convolutional neural network can reach 16 or 19 layers in depth, which can capture the details and high-level features of the image.

[0047] The long short-term memory recurrent neural network can solve the gradient vanishing or exploding problem faced by traditional RNN when processing long sequence data by introducing long short-term memory units (LSTM units) and gating mechanisms, thereby effectively capturing the long-term dependencies in the sequence. Since the lesion area and lesion level usually present different image forms over time, the accuracy of lesion area and lesion level recognition can be further improved by capturing the changes in the image sequence.

[0048] Input the amplitude image and phase image into the long short-term memory recurrent neural network respectively, and obtain the amplitude features and phase features output by the long short-term memory recurrent neural network respectively. Set the input layer for the amplitude image and phase image respectively, and ensure that their input size matches the model design. Since LSTM processes sequence data, it is necessary to ensure that the input data is provided in the form of a sequence. If the amplitude image and phase image have been converted into time series data, they can be directly input; otherwise, appropriate conversion is required.

[0049] Figure 3 Schematic diagram of the structure of the long short-term memory recurrent neural network in the present invention. Figure 3 As shown in the figure, this structure contains two layers of recurrent convolutional layers, each layer is composed of 64 long short-term memory unit LSTM structures, and the final feature vector is 64 pixels. After the amplitude image and phase image pass through LSTM-RNN respectively, the obtained amplitude feature and phase feature are 128 pixels.

[0050] Then, the image features, depth features, amplitude features and phase features are input into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer. Here, the image features, depth features, amplitude features and phase features may be first fused to obtain fused features, and then prediction is performed based on the fused features to obtain the lesion area and lesion degree. It is also possible to predict each feature separately to obtain the prediction results obtained by each feature separately, and then fuse the prediction results obtained by each feature separately to output the final lesion area and lesion degree, which is not specifically limited in the embodiments of the present invention.

[0051] Based on any of the above embodiments, the image recognition model further includes a fully connected layer, and the image features, depth features, amplitude features, and phase features are input into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer, that is, step 133 specifically includes: Step 133-1, inputting the image features, depth features, amplitude features and phase features into the fully connected layer to obtain the fusion features output by the fully connected layer; Step 133 - 2 , input the fused features to the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

[0052] Specifically, in the fully connected layer, image features, depth features, amplitude features, and phase features can be fused into a long feature vector through feature concatenation or feature addition. In the fully connected layer, feature vectors are fused through linear transformation and nonlinear activation functions. Each layer calculates new feature representations based on the output of the previous layer, and these feature representations gradually become more abstract and advanced. After being processed by the fully connected layer, a fused feature vector that combines all input features is finally output. This vector will be used as the input of the prediction layer.

[0053] For the prediction of lesion area and lesion degree, the prediction layer may need to design two sub-parts: one for locating the lesion area (such as outputting a segmentation map of the same size as the original image), and the other for classifying the lesion degree (such as outputting the probability distribution of different lesion degrees).

[0054] For lesion area positioning, the prediction layer may include a convolution layer or an upsampling layer to convert the fused feature vector into a segmentation map of the same size as the original image. Each pixel on the segmentation map may represent the probability that the location belongs to the lesion area.

[0055] For the classification of lesion severity, the prediction layer outputs the probability distribution of different lesion severity through a softmax layer.

[0056] In some embodiments, step 110 specifically includes: Step 111, obtaining an original microscopic image; Step 112, performing image segmentation on the original microscopic image, and using each image block obtained by segmentation as a microscopic image to be identified.

[0057] Specifically, the original microscopic image is captured by an image acquisition device (such as a digital camera). The original microscopic image is then divided into several regions or objects with similar characteristics to achieve image segmentation. Threshold-based segmentation, region-based segmentation, edge-based segmentation, morphology-based segmentation, and deep learning-based segmentation can be used. The deep learning model can be, for example, a deep convolutional generative adversarial network (DCGAN).

[0058] According to the segmentation results, each segmented image block is used as the microscopic image to be identified. These image blocks usually contain specific target areas or features in the original image.

[0059] Based on any of the above embodiments, the sample microscopic image is determined based on the following steps: Obtain original sample microscopic images; The original sample microscopic image is segmented to obtain sample blocks, and each sample block is preprocessed to obtain a sample microscopic image. The preprocessing includes at least one of block value homogenization, rotation, flipping, and scaling.

[0060] Specifically, in order to further improve the recognition accuracy of the image recognition model, a large number of sample microscopic images can be used to train the model. After obtaining the original sample microscopic image, the original sample microscopic image is segmented to obtain each sample block. Then, each sample block is preprocessed, such as cropping each sample block to 128*128 pixels.

[0061] Normalization of image block values ​​refers to adjusting the pixel values ​​of each sample image block to the same range to eliminate the differences between different images caused by factors such as lighting and exposure. Commonly used methods for normalization of values ​​include linear transformation and histogram equalization.

[0062] Rotation refers to rotating the sample block around its center point or other specified point by a certain angle to increase the diversity of the data set and improve the generalization ability of the model. The rotation operation can be implemented through image geometric transformation algorithms, such as affine transformation, perspective transformation, etc.

[0063] Flipping refers to flipping the sample tiles horizontally or vertically, which is also to increase the diversity of the data set and effectively improve the robustness of the model.

[0064] Scaling refers to adjusting the size of the sample tile to meet the requirements of subsequent processing or analysis. The scaling operation can be achieved through interpolation algorithms, such as bilinear interpolation, bicubic interpolation, etc.

[0065] Based on any of the above embodiments, an image recognition method is provided, including: S1, obtaining an original sample microscopic image; performing image segmentation on the original sample microscopic image to obtain sample blocks, and performing preprocessing on each sample block to obtain a sample microscopic image, wherein the preprocessing includes at least one of block value homogenization, rotation, flipping, and scaling.

[0066] S2, an image recognition model is trained based on sample microscopic images, lesion area labels and lesion severity labels.

[0067] S3, obtaining a microscopic image to be identified.

[0068] S4, performing shear wave transformation on the microscopic image, and determining an amplitude image and a phase image of the microscopic image based on the transformed image.

[0069] S5, inputting the microscopic image, the amplitude image and the phase image into a trained image recognition model to obtain the lesion area in the microscopic image output by the image recognition model, and the lesion degree of the lesion area. Figure 4 FIG. 2 is a flow chart of the image recognition method provided by the present invention. Figure 4 As shown, S5 specifically includes: Input the microscopic image into the convolutional neural network and the deep convolutional neural network respectively, and obtain the image features output by the convolutional neural network and the depth features output by the deep convolutional neural network; The amplitude image and the phase image are respectively input into the long short-term memory recurrent neural network to obtain the amplitude features and the phase features respectively output by the long short-term memory recurrent neural network; Input the image features, depth features, amplitude features and phase features into the fully connected layer to obtain the fusion features output by the fully connected layer; The fused features are input into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

[0070] Based on any of the above embodiments, Figure 5 is a schematic diagram of the structure of the image recognition device provided by the present invention, such as Figure 5 As shown, the device comprises: An image acquisition unit 510, used to acquire a microscopic image to be identified; An image transformation unit 520, configured to perform shear wave transformation on the microscopic image, and determine an amplitude image and a phase image of the microscopic image based on the transformed image; An image recognition unit 530 is used to input the microscopic image, the amplitude image and the phase image into a trained image recognition model to obtain the lesion area in the microscopic image output by the image recognition model and the lesion degree of the lesion area; The image recognition model is trained based on sample microscopic images, lesion area labels and lesion degree labels.

[0071] The device provided by the embodiment of the present invention performs shear wave transform on the microscopic image, determines the amplitude image and phase image of the microscopic image based on the transformed image, and identifies the lesion area and the degree of the lesion based on the microscopic image, the amplitude image and the phase image through a trained image recognition model, thereby improving the accuracy and reliability of recognition.

[0072] Based on any of the above embodiments, the image recognition model includes a convolutional neural network, a long short-term memory recurrent neural network, a deep convolutional neural network and a prediction layer, and the image recognition unit includes: A first feature extraction unit, used to input the microscopic image into the convolutional neural network and the deep convolutional neural network respectively, to obtain image features output by the convolutional neural network and depth features output by the deep convolutional neural network; A second feature extraction unit is used to input the amplitude image and the phase image into the long short-term memory recurrent neural network respectively, to obtain the amplitude feature and the phase feature respectively output by the long short-term memory recurrent neural network; The prediction unit is used to input the image features, depth features, amplitude features and phase features into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

[0073] Based on any of the above embodiments, the image recognition unit is specifically used for: Inputting the image features, depth features, amplitude features and phase features into the fully connected layer to obtain fusion features output by the fully connected layer; The fused features are input into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

[0074] Based on any of the above embodiments, the image acquisition unit is specifically used for: Acquire original microscopic images; The original microscopic image is segmented, and each image block obtained by segmentation is used as the microscopic image to be identified.

[0075] Based on any of the above embodiments, it further includes a sample acquisition unit, which is specifically used to: Obtain original sample microscopic images; The original sample microscopic image is segmented to obtain sample blocks, and each sample block is preprocessed to obtain the sample microscopic image. The preprocessing includes at least one of block value normalization, rotation, flipping, and scaling.

[0076] Figure 6An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the image recognition method, which includes: acquiring a microscopic image to be recognized; performing shear wave transformation on the microscopic image, and determining the amplitude image and phase image of the microscopic image based on the transformed image; inputting the microscopic image, the amplitude image and the phase image into a trained image recognition model to obtain the lesion area in the microscopic image output by the image recognition model, and the lesion degree of the lesion area; wherein the image recognition model is trained based on the sample microscopic image, the lesion area label and the lesion degree label.

[0077] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0078] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image recognition method provided by the above-mentioned methods, which includes: acquiring a microscopic image to be identified; performing a shear wave transform on the microscopic image, and determining an amplitude image and a phase image of the microscopic image based on the transformed image; inputting the microscopic image, the amplitude image and the phase image into a trained image recognition model to obtain the lesion area in the microscopic image output by the image recognition model, and the lesion degree of the lesion area; wherein the image recognition model is trained based on sample microscopic images, lesion area labels and lesion degree labels.

[0079] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the image recognition method provided by the above-mentioned methods, the method comprising: acquiring a microscopic image to be identified; performing a shear wave transform on the microscopic image, and determining an amplitude image and a phase image of the microscopic image based on the transformed image; inputting the microscopic image, the amplitude image and the phase image into a trained image recognition model to obtain the lesion area in the microscopic image output by the image recognition model, and the lesion degree of the lesion area; wherein the image recognition model is trained based on sample microscopic images, lesion area labels and lesion degree labels.

[0080] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image recognition method, characterized in that: include: Acquiring a microscopic image to be identified; Performing shear wave transformation on the microscopic image, and determining an amplitude image and a phase image of the microscopic image based on the transformed image; Inputting the microscopic image, the amplitude image and the phase image into a trained image recognition model to obtain the lesion area in the microscopic image output by the image recognition model and the lesion degree of the lesion area; The image recognition model is trained based on sample microscopic images, lesion area labels and lesion degree labels.

2. The image recognition method according to claim 1, characterized in that: The image recognition model includes a convolutional neural network, a long short-term memory recurrent neural network, a deep convolutional neural network and a prediction layer. The microscopic image, the amplitude image and the phase image are input into the trained image recognition model to obtain the lesion area and lesion degree output by the image recognition model, including: Inputting the microscopic image into the convolutional neural network and the deep convolutional neural network respectively, obtaining image features output by the convolutional neural network and depth features output by the deep convolutional neural network; Inputting the amplitude image and the phase image into the long short-term memory recurrent neural network respectively, to obtain the amplitude features and the phase features respectively output by the long short-term memory recurrent neural network; The image features, depth features, amplitude features and phase features are input into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

3. The image recognition method according to claim 2, characterized in that: The image recognition model further includes a fully connected layer, wherein the image features, depth features, amplitude features, and phase features are input to the prediction layer to obtain the lesion area and lesion degree output by the prediction layer, including: Inputting the image features, depth features, amplitude features and phase features into the fully connected layer to obtain fusion features output by the fully connected layer; The fused features are input into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

4. The image recognition method according to claim 1, characterized in that: The step of obtaining a microscopic image to be identified comprises: Acquire original microscopic images; The original microscopic image is segmented, and each image block obtained by segmentation is used as the microscopic image to be identified.

5. The image recognition method according to claim 1, characterized in that: The sample microscopic image is determined based on the following steps: Obtain original sample microscopic images; The original sample microscopic image is segmented to obtain sample blocks, and each sample block is preprocessed to obtain the sample microscopic image. The preprocessing includes at least one of block value normalization, rotation, flipping, and scaling.

6. An image recognition device, characterized in that: include: An image acquisition unit, used for acquiring a microscopic image to be identified; An image transformation unit, configured to perform shear wave transformation on the microscopic image, and determine an amplitude image and a phase image of the microscopic image based on the transformed image; An image recognition unit, used for inputting the microscopic image, the amplitude image and the phase image into a trained image recognition model, and obtaining the lesion area in the microscopic image output by the image recognition model, and the lesion degree of the lesion area; The image recognition model is trained based on sample microscopic images, lesion area labels and lesion degree labels.

7. The image recognition device according to claim 6, characterized in that: The image recognition model includes a convolutional neural network, a long short-term memory recurrent neural network, a deep convolutional neural network and a prediction layer, and the image recognition unit includes: A first feature extraction unit, used to input the microscopic image into the convolutional neural network and the deep convolutional neural network respectively, to obtain image features output by the convolutional neural network and depth features output by the deep convolutional neural network; A second feature extraction unit is used to input the amplitude image and the phase image into the long short-term memory recurrent neural network respectively, to obtain the amplitude feature and the phase feature respectively output by the long short-term memory recurrent neural network; The prediction unit is used to input the image features, depth features, amplitude features and phase features into the prediction layer to obtain the lesion area and lesion degree output by the prediction layer.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the image recognition method according to any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image recognition method according to any one of claims 1 to 5 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image recognition method according to any one of claims 1 to 5 is implemented.

Images