Lesion identification method and device

Abstract

The present invention discloses a lesion identification method and device, relating to the field of image processing technology. A specific embodiment of the method includes: inputting a first frame sample image and a continuous plurality of frames of sample images including the first frame sample image into a neural network for supervised training, thereby training a lesion identification model; wherein the neural network includes a classifier and multiple feature extraction layers in parallel, one feature extraction layer is used to extract features of the first frame sample image, and the other feature extraction layers are used to extract features of the multiple frames of sample images; inputting a first frame test image and a continuous plurality of frames of test images including the first frame test image into the lesion identification model to identify lesions in the test images. This embodiment can solve the technical problem of insufficient lesion identification accuracy.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a lesion recognition method and device. Background Art

[0002] At present, convolutional neural networks have been widely used in lesion recognition, but the existing recognition algorithms are not accurate enough in lesion recognition, and there are a large number of false positives, resulting in a high false positive rate. Summary of the Invention

[0003] In view of this, an embodiment of the present invention provides a lesion identification method and device to solve the technical problem of insufficient accuracy in lesion identification.

[0004] To achieve the above object, according to one aspect of an embodiment of the present invention, a lesion identification method is provided, comprising:

[0005] Inputting a first frame sample image and a plurality of consecutive frames of sample images including the first frame sample image into a neural network for supervised training, thereby training a lesion recognition model; wherein the neural network includes a classifier and a plurality of feature extraction layers connected in parallel, wherein one feature extraction layer is used to extract features of the first frame sample image, and the other feature extraction layers are used to extract features of the plurality of frames sample images;

[0006] A first frame test image and a plurality of consecutive frames of test images including the first frame test image are input into the lesion recognition model to recognize lesions in the test images.

[0007] Optionally, inputting a first test image frame or a plurality of consecutive test images including the first test image frame into the lesion recognition model to identify lesions in the test image includes:

[0008] Inputting a first frame of test image and a plurality of consecutive frames of test images including the first frame of test image into the lesion recognition model to output a recognition result of each abnormal area;

[0009] The lesions in the test image are identified according to the identification results of each abnormal area and preset lesion identification rules.

[0010] Optionally, the recognition result of the abnormal region includes location information of the abnormal region and a lesion probability of the abnormal region, and each abnormal region corresponds to one or more recognition results;

[0011] Identifying the lesion in the test image according to the recognition result of each abnormal area and a preset lesion recognition rule includes:

[0012] If the probability of the abnormal region as a lesion in each recognition result decreases or increases successively, the abnormal region is identified as a lesion.

[0013] Optionally, if the probability of the abnormal region being a lesion in each recognition result decreases or increases successively, identifying the abnormal region as a lesion includes:

[0014] If the probability of the abnormal area as a lesion in each recognition result decays in sequence and the decay amplitude between adjacent recognition results is less than or equal to a preset decay threshold, the abnormal area is identified as a lesion;

[0015] or,

[0016] If the probability of the abnormal region as a lesion in each recognition result increases successively and the enhancement amplitude between adjacent recognition results is less than or equal to a preset enhancement threshold, the abnormal region is identified as a lesion.

[0017] Optionally, the first frame sample image and a plurality of consecutive frames of sample images including the first frame sample image are input into a neural network for supervised training, thereby training a lesion recognition model, including:

[0018] The first frame sample image, the continuous first frame sample images and the second frame sample images, and the continuous first frame sample images, second frame sample images and third frame sample images are input into a neural network for supervised training, so as to train a lesion recognition model; wherein, the neural network includes a classifier and a first feature extraction layer, a second feature extraction layer and a third feature extraction layer in parallel, the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, and the third feature extraction layer is used to extract features of the third frame sample image.

[0019] Optionally, inputting a first test image frame or a plurality of consecutive test images including the first test image frame into the lesion recognition model to identify lesions in the test image includes:

[0020] Inputting a first test image frame, a continuous first test image frame and a second test image frame, and a continuous first test image frame, a second test image frame and a third test image frame into the lesion recognition model to output a recognition result for each abnormal region; wherein each abnormal region corresponds to one or more recognition results;

[0021] The lesions in the test image are identified according to the identification results of each abnormal area and preset lesion identification rules.

[0022] Optionally, the first frame sample image and a plurality of consecutive frames of sample images including the first frame sample image are input into a neural network for supervised training, thereby training a lesion recognition model, including:

[0023] The first frame sample image, the continuous first frame sample images and the second frame sample images, and the continuous first frame sample images, second frame sample images and third frame sample images are input into a neural network for supervised training, so as to train a lesion recognition model; wherein, the neural network includes a classifier, a first feature extraction layer, a second feature extraction layer and a third feature extraction layer in parallel, and a fusion layer connected between the classifier and the multiple parallel feature extraction layers; the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, the third feature extraction layer is used to extract features of the third frame sample image, and the fusion layer fuses the features extracted by each feature extraction layer.

[0024] Optionally, inputting a first test image frame or a plurality of consecutive test images including the first test image frame into the lesion recognition model to identify lesions in the test image includes:

[0025] Inputting a first test image frame, a continuous first test image frame and a second test image frame, and a continuous first test image frame, a second test image frame, and a third test image frame into the lesion recognition model to output a recognition result for each abnormal area, thereby identifying the lesion in the test image;

[0026] The recognition result of the abnormal region includes the location information of the abnormal region and the lesion probability of the abnormal region, and each abnormal region corresponds to one recognition result.

[0027] In addition, according to another aspect of the embodiments of the present invention, a lesion identification device is provided, comprising:

[0028] a training module, configured to input a first frame of sample image and a plurality of consecutive frames of sample images including the first frame of sample image into a neural network for supervised training, thereby training a lesion recognition model; wherein the neural network includes a classifier and a plurality of feature extraction layers connected in parallel, wherein one feature extraction layer is configured to extract features of the first frame of sample image, and the other feature extraction layers are configured to extract features of the plurality of frames of sample images;

[0029] The recognition module is used to input a first frame test image and a plurality of consecutive frames of test images including the first frame test image into the lesion recognition model to recognize the lesions in the test image.

[0030] Optionally, the identification module is further configured to:

[0031] Inputting a first frame of test image and a plurality of consecutive frames of test images including the first frame of test image into the lesion recognition model to output a recognition result of each abnormal area;

[0032] The lesions in the test image are identified according to the identification results of each abnormal area and preset lesion identification rules.

[0033] Optionally, the recognition result of the abnormal region includes location information of the abnormal region and a lesion probability of the abnormal region, and each abnormal region corresponds to one or more recognition results;

[0034] The identification module is further configured to:

[0035] If the probability of the abnormal region as a lesion in each recognition result decreases or increases successively, the abnormal region is identified as a lesion.

[0036] Optionally, the identification module is further configured to:

[0037] If the probability of the abnormal area as a lesion in each recognition result decays in sequence and the decay amplitude between adjacent recognition results is less than or equal to a preset decay threshold, the abnormal area is identified as a lesion;

[0038] or,

[0039] If the probability of the abnormal region as a lesion in each recognition result increases successively and the enhancement amplitude between adjacent recognition results is less than or equal to a preset enhancement threshold, the abnormal region is identified as a lesion.

[0040] Optionally, the training module is further used to:

[0041] The first frame sample image, the continuous first frame sample images and the second frame sample images, and the continuous first frame sample images, second frame sample images and third frame sample images are input into a neural network for supervised training, so as to train a lesion recognition model; wherein, the neural network includes a classifier and a first feature extraction layer, a second feature extraction layer and a third feature extraction layer in parallel, the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, and the third feature extraction layer is used to extract features of the third frame sample image.

[0042] Optionally, the identification module is further configured to:

[0043] Inputting a first test image frame, a continuous first test image frame and a second test image frame, and a continuous first test image frame, a second test image frame and a third test image frame into the lesion recognition model to output a recognition result for each abnormal region; wherein each abnormal region corresponds to one or more recognition results;

[0044] The lesions in the test image are identified according to the identification results of each abnormal area and preset lesion identification rules.

[0045] Optionally, the training module is further used to:

[0046] The first frame sample image, the continuous first frame sample images and the second frame sample images, and the continuous first frame sample images, second frame sample images and third frame sample images are input into a neural network for supervised training, so as to train a lesion recognition model; wherein, the neural network includes a classifier, a first feature extraction layer, a second feature extraction layer and a third feature extraction layer in parallel, and a fusion layer connected between the classifier and the multiple parallel feature extraction layers; the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, the third feature extraction layer is used to extract features of the third frame sample image, and the fusion layer fuses the features extracted by each feature extraction layer.

[0047] Optionally, the identification module is further configured to:

[0048] Inputting a first test image frame, a continuous first test image frame and a second test image frame, and a continuous first test image frame, a second test image frame, and a third test image frame into the lesion recognition model to output a recognition result for each abnormal area, thereby identifying the lesion in the test image;

[0049] The recognition result of the abnormal region includes the location information of the abnormal region and the lesion probability of the abnormal region, and each abnormal region corresponds to one recognition result.

[0050] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, including:

[0051] one or more processors;

[0052] a storage device for storing one or more programs,

[0053] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the above embodiments.

[0054] According to another aspect of the embodiments of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in any one of the above embodiments is implemented.

[0055] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method described in any one of the above embodiments is implemented.

[0056] One embodiment of the above invention has the following advantages or beneficial effects: By employing a technical approach of inputting a first test image frame and a continuous plurality of test image frames including the first test image frame into a lesion recognition model to identify lesions in the test images, the prior art problem of being unable to accurately determine message backlogs is overcome. By testing the first test image frame and the continuous plurality of test image frames including the first test image frame, this embodiment of the present invention can effectively improve the accuracy of lesion recognition, thereby reducing the false positive rate.

[0057] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. Among them:

[0059] Figure 1 is a flow chart of a lesion identification method according to an embodiment of the present invention;

[0060] Figure 2 is a schematic diagram of the structure of a neural network according to one embodiment of the present invention;

[0061] Figure 3 is a schematic diagram of the structure of a neural network according to another embodiment of the present invention;

[0062] Figure 4 is a flow chart of a lesion identification method according to a reference embodiment of the present invention;

[0063] Figure 5 is a schematic diagram of a lesion identification device according to an embodiment of the present invention;

[0064] Figure 6 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0065] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0067] It should be noted that the acquisition, storage, use, and processing of user personal information involved in the technical solution of this application comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0068] Figure 1 Flowchart of the lesion identification method according to an embodiment of the present invention. Figure 1 As shown, the lesion identification method may include:

[0069] Step 101: Input a first frame sample image and a plurality of consecutive frames of sample images including the first frame sample image into a neural network for supervised training, thereby training and obtaining a lesion recognition model.

[0070] First, the neural network needs to be trained. Specifically, the first sample image frame and multiple consecutive sample image frames containing the first sample image frame are input into the neural network for supervised training, thereby training a lesion recognition model. Before step 101, the lesion and its location information need to be marked in each sample image frame. If there is no lesion in the sample image, no marking is required.

[0071] For example, the first frame of sample image is input into the first feature extraction layer, the first frame of sample image and the second frame of sample image are input into the second feature extraction layer, and the parameters of the neural network are optimized based on the gradient descent algorithm, thereby training a lesion recognition model. For another example, the first frame of sample image is input into the first feature extraction layer, the first frame of sample image and the second frame of sample image are input into the second feature extraction layer, and the first frame of sample image, the second frame of sample image, and the third frame of sample image are input into the third feature extraction layer, and the parameters of the neural network are optimized based on the gradient descent algorithm, thereby training a lesion recognition model. For another example, the first frame of sample image is input into the first feature extraction layer, the first frame of sample image and the second frame of sample image are input into the second feature extraction layer, the first frame of sample image, the second frame of sample image, and the third frame of sample image are input into the third feature extraction layer, and the first frame of sample image, the second frame of sample image, the third frame of sample image, and the fourth frame of sample image are input into the fourth feature extraction layer, and the parameters of the neural network are optimized based on the gradient descent algorithm, thereby training a lesion recognition model. And so on, no further details will be given.

[0072] To ensure sufficient sample images, data augmentation can be performed on the images to enrich the sample images. Specifically, by analyzing the changes between two consecutive frames, we extract the change parameters (such as displacement parameters, blurring parameters, changes in lesion area size, etc.), and use these change parameters as variables to randomly generate sample images. For example, for the lesion image that appears in the first frame of the sample image, we randomly apply the change parameters to the lesion and background areas. This way, we can generate a large number of sample images for model training.

[0073] In some embodiments of the present invention, Figure 2 As shown, the neural network includes a classifier and multiple feature extraction layers in parallel, wherein one feature extraction layer is used to extract features of the first frame sample image, and the other feature extraction layers are used to extract features of the multiple frames of sample images. Taking three consecutive frames of images as an example, step 101 may include: inputting the first frame sample image, consecutive first frame sample images and second frame sample images, and consecutive first frame sample images, second frame sample images, and third frame sample images into the neural network for supervised training, thereby training to obtain a lesion recognition model; wherein the neural network includes a classifier and a first feature extraction layer, a second feature extraction layer, and a third feature extraction layer in parallel, wherein the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, and the third feature extraction layer is used to extract features of the third frame sample image.

[0074] In some embodiments of the present invention, Figure 3As shown, the neural network includes a classifier and multiple feature extraction layers connected in parallel, and a fusion layer connected between the classifier and the multiple feature extraction layers connected in parallel. Taking three consecutive frames of images as an example, step 101 may include: inputting the first frame sample image, the consecutive first frame sample images and the second frame sample images, and the consecutive first frame sample images, second frame sample images, and third frame sample images into the neural network for supervised training, thereby training to obtain a lesion recognition model; wherein the neural network includes a classifier, the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer connected in parallel, and a fusion layer connected between the classifier and the multiple feature extraction layers connected in parallel; the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, and the third feature extraction layer is used to extract features of the third frame sample image, and the fusion layer fuses the features extracted by each feature extraction layer.

[0075] Step 102: Input a first frame test image and a plurality of consecutive frames of test images including the first frame test image into the lesion recognition model to identify lesions in the test image.

[0076] Similar to step 101, the first frame test image and the continuous multiple frames of test images including the first frame test image are input into the corresponding feature extraction layer, and the extracted features are identified by the classifier to output the lesions in the test image.

[0077] For example, the first test image is input into the first feature extraction layer, the first test image and the second test image are input into the second feature extraction layer, and the lesion recognition model is used to identify the lesion in the test image. For another example, the first test image is input into the first feature extraction layer, the first test image and the second test image are input into the second feature extraction layer, the first test image, the second test image, and the third test image are input into the third feature extraction layer, and the lesion recognition model is used to identify the lesion in the test image. For another example, the first test image is input into the first feature extraction layer, the first test image and the second test image are input into the second feature extraction layer, the first test image, the second test image, and the third test image are input into the third feature extraction layer, the first test image, the second test image, the third test image, and the fourth test image are input into the fourth feature extraction layer, and the lesion recognition model is used to identify the lesion in the test image. And so on, no further details are given.

[0078] It should be noted that, in order to ensure recognition accuracy, the number of image frames used in step 102 is the same as the number of image frames used in step 101 .

[0079] Taking three consecutive frames of images as an example, step 102 may include: inputting the first frame test image, the consecutive first frame test image and the second frame test image, the consecutive first frame test image, the second frame test image and the third frame test image into the lesion recognition model to output the recognition result of each abnormal area; wherein each abnormal area corresponds to one or more recognition results; and identifying the lesion in the test image according to the recognition result of each abnormal area and the preset lesion recognition rules. Figure 2 As shown, the first test image frame is input into the first feature extraction layer, the consecutive first and second test images are input into the second feature extraction layer, and the consecutive first, second, and third test images are input into the third feature extraction layer. The classifier identifies abnormal regions in the image and outputs an identification result for each abnormal region. Because the neural network has three feature extraction layers, there are three identification results for each abnormal region. The identification results for each abnormal region include the location information of the abnormal region and the probability of the abnormal region being a lesion. Lesions in the test image are then accurately identified based on the identification results for each abnormal region and the preset lesion identification rules.

[0080] Taking three consecutive frames of images as an example, step 102 may include: inputting the first test frame image, the consecutive first test frame image and the second test frame image, the consecutive first test frame image, the second test frame image and the third test frame image into the lesion recognition model to output the recognition result of each abnormal area, thereby identifying the lesion in the test image; wherein the recognition result of the abnormal area includes the position information of the abnormal area and the lesion probability of the abnormal area, and each abnormal area corresponds to a recognition result. Figure 3 As shown, the first test image frame is input into the first feature extraction layer, the consecutive first and second test images are input into the second feature extraction layer, and the consecutive first, second, and third test images are input into the third feature extraction layer. The fusion layer fuses the features extracted by the three feature extraction layers, and then inputs the fused features into the classifier for recognition. The classifier outputs the recognition result for each abnormal region, and then determines whether the abnormal region is a lesion based on the recognition result. Because the neural network sets a fusion layer between the classifier and the three feature extraction layers, the fused features are input into the classifier for recognition, and the classifier outputs the recognition result for each abnormal region, and each abnormal region has only one recognition result.

[0081] Assuming that there are a large number of false positives in the first frame image, when the embodiment of the present invention integrates the features of the second frame image (or even the features of the third frame image, the features of the fourth frame image, or even the features of more frames of images) for joint detection, it is a process of confirming the true positives and excluding the false positives. Therefore, it can effectively improve the recognition accuracy of the lesions and thus reduce the false positive rate.

[0082] When predicting the probability of lesions in combination with the second frame image, the fusion features of the first and second frames are extracted. For example, for the same lesion, it appears clearly in the first frame image and appears displaced and blurred in the second frame image. Then the displacement and blur of the second frame image provide richer identification features for the lesion in the first frame image, thereby effectively improving the recognition accuracy of the lesion and reducing the false positive rate.

[0083] Based on the various embodiments described above, it can be seen that the technical approach of the embodiments of the present invention, which inputs a first test image frame and a continuous plurality of test image frames including the first test image frame into a lesion recognition model to identify lesions in the test images, solves the technical problem of insufficient lesion recognition accuracy in the prior art. By testing the first test image frame and the continuous plurality of test image frames including the first test image frame, the embodiments of the present invention can effectively improve the accuracy of lesion recognition, thereby reducing the false positive rate.

[0084] Figure 4 Flowchart of a lesion identification method according to a reference embodiment of the present invention. As another embodiment of the present invention, Figure 4 As shown, the lesion identification method may include:

[0085] Step 401: Input a first frame sample image and a plurality of consecutive frames of sample images including the first frame sample image into a neural network for supervised training, thereby training a lesion recognition model; wherein the neural network includes a classifier and a plurality of feature extraction layers in parallel, wherein one feature extraction layer is used to extract features of the first frame sample image, and the other feature extraction layers are used to extract features of the plurality of frames of sample images.

[0086] Step 402: Input a first test image frame and a plurality of consecutive test image frames including the first test image frame into the lesion recognition model to output a recognition result for each abnormal region. The recognition result for each abnormal region includes location information of the abnormal region and a lesion probability of the abnormal region, and each abnormal region corresponds to one or more recognition results.

[0087] Step 403: If the probability of the abnormal region being a lesion in each recognition result decreases or increases successively, the abnormal region is recognized as a lesion.

[0088] like Figure 2 As shown in the figure, for the same abnormal area, the classifier outputs three recognition results, namely three lesion probabilities, which are 0.6, 0.75 and 0.9 respectively, then the abnormal area can be identified as a lesion; if the three lesion probabilities of the abnormal area are 0.8, 0.6 and 0.58 respectively, then the abnormal area can be identified as a lesion.

[0089] Optionally, step 403 may include: if the probability of the abnormal area being a lesion in each recognition result decays successively and the attenuation amplitude between adjacent recognition results is less than or equal to a preset attenuation threshold, then the abnormal area is identified as a lesion; or, if the probability of the abnormal area being a lesion in each recognition result increases successively and the enhancement amplitude between adjacent recognition results is less than or equal to a preset enhancement threshold, then the abnormal area is identified as a lesion.

[0090] It should be noted that the attenuation threshold and enhancement threshold can be set according to actual needs. Figure 2 For example, if the probability of the abnormal area being a lesion in the first recognition result, the second recognition result, and the third recognition result decays successively and the attenuation amplitude between adjacent recognition results is less than or equal to a preset attenuation threshold, the abnormal area is identified as a lesion; or, if the probability of the abnormal area being a lesion in the first recognition result, the second recognition result, and the third recognition result increases successively and the enhancement amplitude between adjacent recognition results is less than or equal to a preset enhancement threshold, the abnormal area is identified as a lesion.

[0091] It should also be pointed out that the number of recognition results is determined by the number of consecutive frame images. If there are four consecutive frames of images, the neural network contains four feature extraction layers. Accordingly, the classifier outputs four recognition results, and so on. I will not go into details.

[0092] In addition, the specific implementation content of the lesion identification method in a reference embodiment of the present invention has been described in detail in the lesion identification method described above, so the repeated content will not be described again here.

[0093] Figure 5 FIG is a schematic diagram of a lesion identification device according to an embodiment of the present invention. Figure 5As shown, the lesion identification device 500 includes a training module 501 and a recognition module 502; wherein, the training module 501 is used to input a first frame sample image and a continuous multi-frame sample image including the first frame sample image into a neural network for supervised training, thereby training to obtain a lesion recognition model; wherein, the neural network includes a classifier and a plurality of feature extraction layers in parallel, wherein one feature extraction layer is used to extract features of the first frame sample image, and the other feature extraction layers are used to extract features of the multi-frame sample images; the recognition module 502 is used to input a first frame test image and a continuous multi-frame test image including the first frame test image into the lesion recognition model to identify the lesion in the test image.

[0094] Optionally, the identification module 502 is further configured to:

[0095] Inputting a first frame of test image and a plurality of consecutive frames of test images including the first frame of test image into the lesion recognition model to output a recognition result of each abnormal area;

[0096] The lesions in the test image are identified according to the identification results of each abnormal area and preset lesion identification rules.

[0097] Optionally, the recognition result of the abnormal region includes location information of the abnormal region and a lesion probability of the abnormal region, and each abnormal region corresponds to one or more recognition results;

[0098] The identification module 502 is further configured to:

[0099] If the probability of the abnormal region as a lesion in each recognition result decreases or increases successively, the abnormal region is identified as a lesion.

[0100] Optionally, the identification module 502 is further configured to:

[0101] If the probability of the abnormal area as a lesion in each recognition result decays in sequence and the decay amplitude between adjacent recognition results is less than or equal to a preset decay threshold, the abnormal area is identified as a lesion;

[0102] or,

[0103] If the probability of the abnormal region as a lesion in each recognition result increases successively and the enhancement amplitude between adjacent recognition results is less than or equal to a preset enhancement threshold, the abnormal region is identified as a lesion.

[0104] Optionally, the training module 501 is further configured to:

[0105] The first frame sample image, the continuous first frame sample images and the second frame sample images, and the continuous first frame sample images, second frame sample images and third frame sample images are input into a neural network for supervised training, so as to train a lesion recognition model; wherein, the neural network includes a classifier and a first feature extraction layer, a second feature extraction layer and a third feature extraction layer in parallel, the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, and the third feature extraction layer is used to extract features of the third frame sample image.

[0106] Optionally, the identification module 502 is further configured to:

[0107] Inputting a first test image frame, a continuous first test image frame and a second test image frame, and a continuous first test image frame, a second test image frame and a third test image frame into the lesion recognition model to output a recognition result for each abnormal region; wherein each abnormal region corresponds to one or more recognition results;

[0108] The lesions in the test image are identified according to the identification results of each abnormal area and preset lesion identification rules.

[0109] Optionally, the training module 501 is further configured to:

[0110] The first frame sample image, the continuous first frame sample images and the second frame sample images, and the continuous first frame sample images, second frame sample images and third frame sample images are input into a neural network for supervised training, so as to train a lesion recognition model; wherein, the neural network includes a classifier, a first feature extraction layer, a second feature extraction layer and a third feature extraction layer in parallel, and a fusion layer connected between the classifier and the multiple parallel feature extraction layers; the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, the third feature extraction layer is used to extract features of the third frame sample image, and the fusion layer fuses the features extracted by each feature extraction layer.

[0111] Optionally, the identification module 502 is further configured to:

[0112] Inputting a first test image frame, a continuous first test image frame and a second test image frame, and a continuous first test image frame, a second test image frame, and a third test image frame into the lesion recognition model to output a recognition result for each abnormal area, thereby identifying the lesion in the test image;

[0113] The recognition result of the abnormal region includes the location information of the abnormal region and the lesion probability of the abnormal region, and each abnormal region corresponds to one recognition result.

[0114] It should be noted that the specific implementation content of the lesion identification device of the present invention has been described in detail in the lesion identification method described above, so the repeated content will not be described again here.

[0115] Figure 6 An exemplary system architecture 600 is shown to which the lesion identification method or lesion identification apparatus according to the embodiment of the present invention can be applied.

[0116] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0117] Users can use terminal devices 601, 602, and 603 to interact with server 605 via network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0118] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0119] Server 605 may be a server that provides various services, such as a backend management server (for example only) that supports shopping websites browsed by users using terminal devices 601, 602, and 603. The backend management server may analyze and process received data such as item information query requests, and feed back the processing results to the terminal device.

[0120] It should be noted that the lesion identification method provided in the embodiment of the present invention is generally executed by the server 605, and accordingly, the lesion identification device is generally provided in the server 605. The lesion identification method provided in the embodiment of the present invention can also be executed by the terminal devices 601, 602, and 603, and accordingly, the lesion identification device can be provided in the terminal devices 601, 602, and 603.

[0121] It should be understood that Figure 6The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0122] Reference below Figure 7 , which shows a schematic structural diagram of a computer system 700 of a terminal device suitable for implementing an embodiment of the present invention. Figure 7 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0123] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the system 700 are also stored in the RAM 703. The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0124] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output section 707 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 708 including a hard disk; and a communication section 709 including a network interface card such as a LAN card or a modem. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 710 as needed, so that computer programs read therefrom can be installed into the storage section 708 as needed.

[0125] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from a removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0126] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer programs according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0128] The modules described in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be provided in a processor. For example, a processor may be described as including a training module and a recognition module. The names of these modules do not, in some cases, limit the modules themselves.

[0129] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The computer-readable medium carries one or more programs. When the one or more programs are executed by a device, the device implements the following method: inputting a first frame sample image and a continuous plurality of frames of sample images including the first frame sample image into a neural network for supervised training, thereby training a lesion recognition model; wherein the neural network includes a classifier and a plurality of feature extraction layers in parallel, wherein one feature extraction layer is used to extract features of the first frame sample image, and the other feature extraction layers are used to extract features of the plurality of frames of sample images; inputting a first frame test image and a continuous plurality of frames of test images including the first frame test image into the lesion recognition model to identify lesions in the test image.

[0130] As another aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which implements the method described in any of the above embodiments when executed by a processor.

[0131] The technical solution according to the embodiments of the present invention overcomes the prior art problem of being unable to accurately determine message backlogs by inputting a first test image frame and a continuous multi-frame test image including the first test image frame into a lesion recognition model to identify lesions in the test images. By testing the first test image frame and a continuous multi-frame test image including the first test image frame, the embodiments of the present invention can effectively improve the accuracy of lesion recognition, thereby reducing the false positive rate.

[0132] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A lesion identification method, characterized in that: include: Inputting a first frame sample image and a plurality of consecutive frames of sample images including the first frame sample image into a neural network for supervised training, thereby training a lesion recognition model; wherein the neural network includes a classifier and a plurality of feature extraction layers connected in parallel, wherein one feature extraction layer is used to extract features of the first frame sample image, and the other feature extraction layers are used to extract features of the plurality of frames sample images; Inputting a first test image frame and a plurality of consecutive test images including the first test image frame into the lesion recognition model to output a recognition result for each abnormal region; the recognition result for each abnormal region includes location information of the abnormal region and a lesion probability of the abnormal region, and each abnormal region corresponds to one or more recognition results; If the probability of the abnormal region as a lesion in each recognition result decreases or increases successively, the abnormal region is identified as a lesion.

2. The method according to claim 1, characterized in that If the probability of the abnormal region as a lesion in each recognition result decreases or increases successively, then the abnormal region is identified as a lesion, including: If the probability of the abnormal area as a lesion in each recognition result decays in sequence and the decay amplitude between adjacent recognition results is less than or equal to a preset decay threshold, the abnormal area is identified as a lesion; or, If the probability of the abnormal region as a lesion in each recognition result increases successively and the enhancement amplitude between adjacent recognition results is less than or equal to a preset enhancement threshold, the abnormal region is identified as a lesion.

3. The method according to claim 1, characterized in that Inputting a first frame sample image and a plurality of consecutive frames of sample images including the first frame sample image into a neural network for supervised training, thereby training a lesion recognition model, including: The first frame sample image, the continuous first frame sample images and the second frame sample images, and the continuous first frame sample images, second frame sample images and third frame sample images are input into a neural network for supervised training, so as to train a lesion recognition model; wherein, the neural network includes a classifier and a first feature extraction layer, a second feature extraction layer and a third feature extraction layer in parallel, the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, and the third feature extraction layer is used to extract features of the third frame sample image.

4. The method according to claim 3, characterized in that Inputting a first test image frame and a plurality of consecutive test images including the first test image frame into the lesion recognition model to identify lesions in the test image includes: Inputting a first test image frame, a continuous first test image frame and a second test image frame, and a continuous first test image frame, a second test image frame and a third test image frame into the lesion recognition model to output a recognition result for each abnormal region; wherein each abnormal region corresponds to one or more recognition results; The lesions in the test image are identified according to the identification results of each abnormal area and preset lesion identification rules.

5. The method according to claim 1, characterized in that Inputting a first frame sample image and a plurality of consecutive frames of sample images including the first frame sample image into a neural network for supervised training, thereby training a lesion recognition model, including: The first frame sample image, the continuous first frame sample images and the second frame sample images, and the continuous first frame sample images, second frame sample images and third frame sample images are input into a neural network for supervised training, so as to train a lesion recognition model; wherein, the neural network includes a classifier, a first feature extraction layer, a second feature extraction layer and a third feature extraction layer in parallel, and a fusion layer connected between the classifier and the multiple parallel feature extraction layers; the first feature extraction layer is used to extract features of the first frame sample image, the second feature extraction layer is used to extract features of the second frame sample image, the third feature extraction layer is used to extract features of the third frame sample image, and the fusion layer fuses the features extracted by each feature extraction layer.

6. The method according to claim 5, characterized in that Inputting a first test image frame and a plurality of consecutive test images including the first test image frame into the lesion recognition model to identify lesions in the test image includes: Inputting a first test image frame, a continuous first test image frame and a second test image frame, and a continuous first test image frame, a second test image frame, and a third test image frame into the lesion recognition model to output a recognition result for each abnormal area, thereby identifying the lesion in the test image; The recognition result of the abnormal region includes the location information of the abnormal region and the lesion probability of the abnormal region, and each abnormal region corresponds to one recognition result.

7. A lesion identification device, characterized in that: include: a training module, configured to input a first frame of sample image and a plurality of consecutive frames of sample images including the first frame of sample image into a neural network for supervised training, thereby training a lesion recognition model; wherein the neural network includes a classifier and a plurality of feature extraction layers connected in parallel, wherein one feature extraction layer is configured to extract features of the first frame of sample image, and the other feature extraction layers are configured to extract features of the plurality of frames of sample images; An identification module is used to input a first frame test image and a plurality of consecutive frames of test images including the first frame test image into the lesion identification model to output an identification result for each abnormal area; the identification result of the abnormal area includes the location information of the abnormal area and the lesion probability of the abnormal area, and each abnormal area corresponds to one or more identification results; if the lesion probability of the abnormal area in each identification result decays or increases successively, the abnormal area is identified as a lesion.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images