Gastroscopic image target detection method, device, electronic device and storage medium

Abstract

This application provides a method, apparatus, electronic device, and storage medium for target detection in gastroscopy images. The method includes: acquiring a gastroscopy image to be detected; extracting multiple feature layers from the gastroscopy image to be detected; determining multiple target boxes of different scales based on the multiple feature layers; selecting predicted boxes from the target boxes based on the positional relationship between lesions in the gastroscopy image to be detected and each target box; obtaining the category label corresponding to the predicted box and the image recognition result of the gastroscopy image to be detected; when the category label and the image recognition result are the same, determining the target detection result of the gastroscopy image to be detected based on the predicted box and the image recognition result. The method provided by the above scheme not only determines the specific location information of the lesion but also determines the lesion recognition result, achieving the effect of manual detection by doctors while improving detection efficiency.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, electronic device and storage medium for detecting targets in gastroscopic images. Background Art

[0002] At present, gastroscopy is an important means of diagnosing gastric diseases. Specifically, it can be used to determine whether the patient is suffering from diseases such as gastric cancer or gastric ulcer based on the gastric mucosal epithelial images collected by gastroscopy.

[0003] In the prior art, doctors usually determine the location of the lesion and the diagnosis result by observing gastroscopy images.

[0004] However, when there are a large number of gastroscopic images, if the existing technology is still used to identify and analyze each gastroscopic image, it will take up a lot of doctors' working time, which is not conducive to ensuring detection efficiency. Summary of the invention

[0005] The present application provides a method, device, electronic device and storage medium for gastroscopic image target detection to solve the defects of low detection efficiency in the prior art.

[0006] The first aspect of the present application provides a method for detecting a target in a gastroscopic image, comprising:

[0007] Acquire a gastroscopic image to be detected;

[0008] Extracting multiple feature layers of the gastroscope image to be detected;

[0009] Determining target frames of multiple scales according to the multiple feature layers;

[0010] Selecting a prediction frame from the target frame according to the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame;

[0011] Obtaining a category label corresponding to the prediction frame and an image recognition result of the gastroscopic image to be detected;

[0012] When the category label is the same as the image recognition result, the target detection result of the gastroscope image to be detected is determined according to the prediction frame and the image recognition result.

[0013] Optionally, also include:

[0014] When the category label is different from the image recognition result, return to the step of selecting a prediction box from the target box.

[0015] Optionally, selecting a prediction frame from the target frame according to a positional relationship between a lesion point in the gastroscopic image to be detected and each target frame includes:

[0016] Obtaining the maximum distance between the lesion point and each frame in the target frame;

[0017] Filtering the prediction frame to be determined according to the maximum distance and the eccentric interval corresponding to each target frame;

[0018] Get the confidence level corresponding to each pending prediction box;

[0019] The prediction box is determined according to the confidence levels corresponding to the pending prediction boxes.

[0020] Optionally, the step of obtaining the image recognition result of the gastroscopic image to be detected includes:

[0021] Get the preset neural network model;

[0022] The gastroscope image to be detected is input into the neural network model to determine the image recognition result of the gastroscope image to be detected by using the neural network model.

[0023] Optionally, the extracting multiple feature layers of the gastroscopic image to be detected includes:

[0024] Determining a plurality of characteristic images according to the gastroscopic image to be detected;

[0025] A variety of feature layers are extracted from the feature image according to different downsampling step sizes.

[0026] Optionally, before extracting the multiple feature layers of the gastroscopic image to be detected, the method further includes:

[0027] Perform image preprocessing on the gastroscopic image to be detected.

[0028] Optionally, obtaining the confidence level corresponding to each pending prediction box includes:

[0029] Get the trained target detection model;

[0030] The target detection model is used to determine the confidence of the undetermined prediction box.

[0031] A second aspect of the present application provides a gastroscope image target detection device, comprising:

[0032] A first acquisition module is used to acquire a gastroscopic image to be detected;

[0033] An extraction module, used for extracting multiple feature layers of the gastroscopic image to be detected;

[0034] A determination module, used to determine target frames of multiple scales according to the multiple feature layers;

[0035] A selection module, configured to select a prediction frame from the target frame according to the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame;

[0036] A second acquisition module is used to acquire the category label corresponding to the prediction frame and the image recognition result of the gastroscopic image to be detected;

[0037] A detection module is used to determine the target detection result of the gastroscope image to be detected according to the prediction box and the image recognition result when the category label is the same as the image recognition result.

[0038] Optionally, the detection module is further used to:

[0039] When the category label is different from the image recognition result, return to the step of selecting a prediction box from the target box.

[0040] Optionally, the selection module is specifically used to:

[0041] Obtaining the maximum distance between the lesion point and each frame in the target frame;

[0042] Filtering the prediction frame to be determined according to the maximum distance and the eccentric interval corresponding to each target frame;

[0043] Get the confidence level corresponding to each pending prediction box;

[0044] The prediction box is determined according to the confidence levels corresponding to the pending prediction boxes.

[0045] Optionally, the second acquisition module is specifically used to:

[0046] Get the preset neural network model;

[0047] The gastroscope image to be detected is input into the neural network model to determine the image recognition result of the gastroscope image to be detected by using the neural network model.

[0048] Optionally, the extraction module is specifically used to:

[0049] Determining a plurality of characteristic images according to the gastroscopic image to be detected;

[0050] A variety of feature layers are extracted from the feature image according to different downsampling step sizes.

[0051] Optionally, the extraction module is further used to:

[0052] Perform image preprocessing on the gastroscopic image to be detected.

[0053] Optionally, the selection module is specifically used to:

[0054] Get the trained target detection model;

[0055] The target detection model is used to determine the confidence of the undetermined prediction box.

[0056] A third aspect of the present application provides an electronic device, comprising: at least one processor and a memory;

[0057] The memory stores computer-executable instructions;

[0058] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method described in the first aspect and various possible designs of the first aspect.

[0059] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect are implemented.

[0060] The technical solution of this application has the following advantages:

[0061] The gastroscopic image target detection method, device, electronic device and storage medium provided in the present application obtain the gastroscopic image to be detected; extract multiple feature layers of the gastroscopic image to be detected; determine multiple target frames of different scales based on the multiple feature layers; select a prediction frame from the target frame based on the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame; obtain the category label corresponding to the prediction frame and the image recognition result of the gastroscopic image to be detected; when the category label is the same as the image recognition result, determine the target detection result of the gastroscopic image to be detected based on the prediction frame and the image recognition result. The gastroscopic image target detection method provided by the above scheme determines the prediction frame based on the target frames of different scales obtained, and then determines the final target detection result based on the category label of the prediction frame and the image recognition result, that is, determines the specific location information of the lesion, and also determines the recognition result of the lesion, achieving the effect of manual detection by a doctor, while improving the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0063] Figure 1 This is a schematic diagram of the structure of the gastroscope image target detection system based on the embodiment of the present application;

[0064] Figure 2 A schematic diagram of the process of the gastroscope image target detection method provided in the embodiment of the present application;

[0065] Figure 3 A schematic diagram of the structure of an exemplary FCOS detection model provided in an embodiment of the present application;

[0066] Figure 4 A network architecture diagram of an exemplary gastroscope image object detection system provided in an embodiment of the present application;

[0067] Figure 5 A schematic diagram of the structure of a gastroscope image target detection device provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0069] The above drawings have shown clear embodiments of the present application, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0071] In addition, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. In the description of the following embodiments, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0072] In the prior art, doctors usually determine the location of lesions and the diagnosis results by observing gastroscopic images. However, when there are many gastroscopic images, if the existing technology is still used to identify and analyze each gastroscopic image, it will take up a lot of doctors' working time, which is not conducive to ensuring detection efficiency.

[0073] In response to the above problems, the gastroscopic image target detection method, device, electronic device and storage medium provided in the embodiments of the present application obtain the gastroscopic image to be detected; extract multiple feature layers of the gastroscopic image to be detected; determine multiple target frames of different scales based on the multiple feature layers; select a prediction frame from the target frame based on the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame; obtain the category label corresponding to the prediction frame and the image recognition result of the gastroscopic image to be detected; when the category label is the same as the image recognition result, determine the target detection result of the gastroscopic image to be detected based on the prediction frame and the image recognition result. The gastroscopic image target detection method provided by the above scheme determines the prediction frame based on the target frames of different scales obtained, and then determines the final target detection result based on the category label of the prediction frame and the image recognition result, that is, determines the specific location information of the lesion, and also determines the recognition result of the lesion, achieving the effect of manual detection by a doctor, while improving the detection efficiency.

[0074] The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0075] First, the structure of the gastroscope image target detection system on which this application is based is described:

[0076] The gastroscopic image target detection method, device, electronic device and storage medium provided in the embodiments of the present application are suitable for detecting the location information and lesion category of lesions in gastroscopic images. Figure 1 The figure is a schematic diagram of the structure of the gastroscopic image target detection system based on the embodiment of the present application, which mainly includes a gastroscopic image acquisition device and a gastroscopic image target detection device for performing target detection on the gastroscopic image. Specifically, the gastroscopic image acquisition device sends the acquired gastroscopic image to the gastroscopic image target detection device, which detects the location information and lesion category of the lesion in the gastroscopic image.

[0077] The embodiment of the present application provides a method for detecting a target in a gastroscopic image, which is used to detect the location information and the type of the lesion in the gastroscopic image. The execution subject of the embodiment of the present application is an electronic device, such as a server, a desktop computer, a laptop computer, a tablet computer, and other electronic devices that can be used for detecting a target in a gastroscopic image.

[0078] like Figure 2 FIG. 1 is a flow chart of a method for detecting a target in a gastroscopic image provided by an embodiment of the present application, and the method comprises:

[0079] Step 201, obtaining a gastroscopic image to be detected.

[0080] It should be noted that the gastroscopic image to be detected specifically refers to the gastric mucosal epithelial image collected by the gastroscope.

[0081] Step 202: extract multiple feature layers of the gastroscope image to be detected.

[0082] It should be noted that different feature layers reflect different features. For example, the first feature layer reflects the edge features of the gastroscopic image to be detected, and the second feature layer reflects the color features of the gastroscopic image.

[0083] Specifically, multiple feature layers can be extracted according to different downsampling scales (step sizes).

[0084] Specifically, in one embodiment, a plurality of feature images may be determined according to the gastroscopic image to be detected; and a plurality of feature layers may be extracted from the feature images according to different downsampling step sizes.

[0085] It should be noted that a feature image refers to an image that highlights certain features.

[0086] For example, Figure 3 As shown, it is a schematic diagram of the structure of an exemplary FCOS detection model provided in an embodiment of the present application. The features of the pre-image can be extracted by other learning models to obtain three feature images {C3, C4, C5}, and the downsampling steps used are: 8, 16 and 32 respectively. Then, the Feature Pyramid module of the FCOS detection model is used to further extract features from {C3, C4, C5} to obtain the final feature layer {P3, P4, P5, P6, P7}. P3, P4 and P5 are obtained by convolving the feature images C3, C4 and C5 from top to bottom with a 1*1 convolution kernel, and P6 and P7 are obtained by convolving P5 and P6 with a layer of convolution with a step size of 2. The strides of the feature layers {P3, P4, P5, P6, P7} are 8, 16, 32, 64 and 128 respectively.

[0087] Specifically, in one embodiment, in order to ensure the feature layer extraction effect, before extracting multiple feature layers of the gastroscope image to be detected, the gastroscope image to be detected may be subjected to image preprocessing.

[0088] Specifically, the gastroscope image to be detected may be cropped to scale the original image to a uniform size, and then operations such as de-averaging and normalization may be performed to improve the image quality of the image to be detected.

[0089] Step 203: Determine target boxes of multiple scales based on multiple feature layers.

[0090] Specifically, the preset FCOS detection model can be used to predict target frames of different scales based on different feature layers, so that most of the overlapping target frames can be separated, laying a good foundation for subsequent image target detection work and helping to improve the accuracy of target detection results.

[0091] Step 204 , selecting a prediction frame from the target frame according to the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame.

[0092] It should be noted that the purpose of performing target detection on gastroscopic images in the present embodiment is to detect the lesion type (such as gastric cancer or gastric ulcer, etc.) and the location information of the lesion (gastric antrum, greater curvature of stomach, lesser curvature of stomach and anterior and posterior walls, etc.). The target frame specifically refers to the predicted lesion location frame.

[0093] Specifically, the distance between the lesion point and the four borders of the target box is (l*, t*, r*, b*), max(l*, r*, t*, b*)>m i or max(l*,t*,r*,b*) <m i-1 It is regarded as a negative sample and no longer regressed. Otherwise, it continues to regress. i , i∈[2,7] can be set to 0, 64, 128, 256, 512 and ∞ respectively. Finally, the prediction box can be determined according to the regression result. The prediction box specifically refers to the prediction box that is predicted to match the current lesion, and the specific matching process can be implemented based on the FCOS detection model provided in the above embodiment.

[0094] Step 205, obtaining the category label corresponding to the prediction box and the image recognition result of the gastroscope image to be detected.

[0095] It should be noted that the image recognition result specifically refers to the lesion recognition result obtained based on other technologies, specifically refers to the lesion type corresponding to the current lesion, and the category label refers to the lesion type corresponding to the prediction box.

[0096] Step 206: When the category label is the same as the image recognition result, the target detection result of the gastroscope image to be detected is determined according to the prediction box and the image recognition result.

[0097] Specifically, if the category label is the same as the image recognition result, it can be determined that a reliable lesion recognition result and corresponding position information have been obtained. The position information is specifically determined according to the position of the prediction box. Further, the lesion recognition result and the position information are integrated to obtain the corresponding target detection result.

[0098] On the contrary, in one embodiment, when the category label is different from the image recognition result, the process returns to the step of selecting a prediction box from the target box.

[0099] It should be noted that the current image recognition technology is relatively mature, that is, the accuracy of the image recognition result is higher than the accuracy of the category label corresponding to the prediction box. Therefore, when the category label is different from the image recognition result, it can be determined that the current prediction box selection is inaccurate and the prediction box needs to be reselected. The specific reselection rules can refer to the above embodiment.

[0100] Exemplarily, if two pending prediction boxes are determined according to the regression result, wherein the category label of the currently selected prediction box is different from the image recognition result, the other pending prediction box is determined as the new prediction box.

[0101] Specifically, in one embodiment, in order to ensure the reliability of the image recognition result, a preset neural network model can be obtained; the gastroscopic image to be detected is input into the neural network model to determine the image recognition result of the gastroscopic image to be detected using the neural network model.

[0102] It should be noted that the obtained neural network model is a model that has completed model training, and the neural network model can be used to determine the image recognition result of the gastroscopic image. Among them, the specific training process of the neural network model can refer to the existing technology, and the embodiment of the present application is not limited.

[0103] For example, if the neural network model specifically uses the ResNet-50 network, the image can be first compressed to a size of 256*256, then cropped to a size of 224*224, randomly flipped horizontally, and finally standardized to obtain the image input features. The initial learning rate is set to 0.0001, and the Adam optimizer is used to optimize the grid parameters during training, and the binary cross entropy function is used as the loss function. The batch size is set to 16, and to prevent overfitting, the dropout is set to 0.7. During training, the ResNet-50 network is trained based on the training samples, allowing the network to learn the gastroscopy image classification task, and the optimizer is used to minimize the loss function of the image classification, so that the network finally converges.

[0104] Among them, the loss function of the ResNet-50 model is as follows:

[0105]

[0106] Among them, L cls represents the focal loss, L reg represents the cross-over loss, N pos represents the number of positive samples, λ is L reg The balance weight is 1 here. This function will be used to calculate the feature image F i All pixels in the calculation are is an indicator of this loss function, only when It is equal to 1 if it is, otherwise it is equal to 0.

[0107] On the basis of the above embodiments, in order to further improve the accuracy of the target detection result, as an implementable method, in one embodiment, according to the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame, a prediction frame is selected from the target frame, including:

[0108] Step 2041, obtaining the maximum distance between the lesion point and each frame in the target frame;

[0109] Step 2042, selecting the prediction frame to be determined according to the maximum distance and the eccentric interval corresponding to each target frame;

[0110] Step 2043, obtaining the confidence level corresponding to each pending prediction box;

[0111] Step 2044, determining the prediction box according to the confidence level corresponding to each pending prediction box.

[0112] For example, the present application embodiment uses the FCOS algorithm as an example, and obtains five target frames (five feature layers), and the eccentric intervals corresponding to each target frame are (0, 60), (64, 128), (128, 256), (256, 512) and (512, ∞). The above distance may specifically refer to the pixel distance.

[0113] Specifically, the target box whose maximum distance belongs to the corresponding centrifugal interval is determined as the pending prediction box. Then the pending prediction box with the highest confidence is determined as the prediction box. If the prediction box needs to be reselected later, the confidence of the current prediction box can be adjusted to half of the original confidence, and then the pending prediction box with the second highest confidence is determined as the new prediction box. And so on, until an accurate prediction box is obtained.

[0114] Specifically, in one embodiment, a trained target detection model may be obtained; and the confidence of the pending prediction box may be determined using the target detection model.

[0115] The target detection model may be a FCOS prediction model, and the confidence of each pending prediction box may be determined according to the training result of the FCOS prediction model.

[0116] In order to facilitate those skilled in the art to better understand the gastroscope image target detection method provided in the embodiment of the present application, the training process of the FCOS detection model provided in the embodiment of the present application can refer to the following content:

[0117] Read the position of the real target box in the training set image and text file, and then preprocess the image, including scaling, cropping, demeaning and normalization. After convolution, the image obtains 5 feature layers of different levels, and each feature layer is responsible for predicting target boxes of different scales. The output of target detection is divided into three branches: category branch, center degree branch, and regression branch. For each pixel in each layer, it is mapped to the original image. If the point falls into the target box, it is a positive sample, otherwise it is a negative sample. The category target is the category of the target box. At this time, the regression target is (l*, t*, r*, b*), which represents the distance from the center point to the left, top, right and bottom of the target box respectively. If the point falls into multiple overlapping target boxes at this time, the target box with a smaller area is selected for calculation. In order to make the result more accurate, a weight can be designed in this process: center degree. The output of the category is multiplied by a weight map to obtain the final classification confidence, and this weight map represents the distance from each point in the target box to the center point, that is, the center degree. The closer the distance, the greater the weight. Therefore, the final target detection result is divided into three parts: category label, confidence and prediction box. The learning rate is set to 0.002 at the beginning of training. As the number of training rounds increases, the learning rate is dynamically changed. The SGD (Stochastic Gradient Descent) optimizer is used to optimize the grid parameters, and the loss function provided in the above embodiment is used. The batch size is set to 4. During training, the FCOS network is trained based on the training samples, allowing the network to learn the gastroscope image target detection task, and the optimizer is used to minimize the target detection loss function, and finally the network converges.

[0118] For example, Figure 4 As shown, it is a network architecture diagram of an exemplary gastroscope image target detection system provided in an embodiment of the present application. Among them, the First Network in the figure represents a neural network used to determine the image recognition result of the gastroscope image. The FCOS network architecture can refer to the prior art and will not be repeated here.

[0119] The gastroscopic image target detection method provided in the embodiment of the present application obtains the gastroscopic image to be detected; extracts multiple feature layers of the gastroscopic image to be detected; determines multiple target frames of different scales according to the multiple feature layers; selects a prediction frame from the target frame according to the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame; obtains the category label corresponding to the prediction frame and the image recognition result of the gastroscopic image to be detected; when the category label is the same as the image recognition result, determines the target detection result of the gastroscopic image to be detected according to the prediction frame and the image recognition result. The gastroscopic image target detection method provided by the above scheme determines the prediction frame according to the target frames of different scales obtained, and then determines the final target detection result according to the category label and image recognition result of the prediction frame, that is, determines the specific location information of the lesion, and also determines the recognition result of the lesion, achieving the effect of manual detection by doctors, while improving the detection efficiency. In addition, the method provided in the embodiment of the present application is a target detection method that combines a neural network and an FCOS network, with a higher accuracy and more accurate model classification, thereby improving the effect of target detection.

[0120] An embodiment of the present application provides a gastroscopic image target detection device, which is used to execute the gastroscopic image target detection method provided in the above embodiment.

[0121] like Figure 5 , which is a schematic diagram of the structure of a gastroscope image target detection device provided in an embodiment of the present application. The gastroscope image target detection device 50 comprises a first acquisition module 501, an extraction module 502, a determination module 503, a selection module 504, a second acquisition module 505 and a detection module 506.

[0122] Among them, the first acquisition module is used to acquire the gastroscopic image to be detected; the extraction module is used to extract multiple feature layers of the gastroscopic image to be detected; the determination module is used to determine multiple target frames of different scales based on the multiple feature layers; the selection module is used to select the prediction frame from the target frame based on the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame; the second acquisition module is used to obtain the category label corresponding to the prediction frame and the image recognition result of the gastroscopic image to be detected; the detection module is used to determine the target detection result of the gastroscopic image to be detected based on the prediction frame and the image recognition result when the category label is the same as the image recognition result.

[0123] Specifically, in one embodiment, the detection module is further used to:

[0124] When the category label is different from the image recognition result, return to the step of selecting the prediction box from the target box.

[0125] Specifically, in one embodiment, the selection module is specifically used for:

[0126] Get the maximum distance between the lesion point and each frame in the target frame;

[0127] According to the maximum distance and the eccentric interval corresponding to each target frame, the undetermined prediction frame is selected;

[0128] Get the confidence level corresponding to each pending prediction box;

[0129] The prediction box is determined according to the confidence level corresponding to each pending prediction box.

[0130] Specifically, in one embodiment, the second acquisition module is specifically used to:

[0131] Get the preset neural network model;

[0132] The gastroscope image to be detected is input into the neural network model to determine the image recognition result of the gastroscope image to be detected using the neural network model.

[0133] Specifically, in one embodiment, the extraction module is specifically used to:

[0134] Determine multiple characteristic images according to the gastroscopic image to be detected;

[0135] According to different downsampling steps, multiple feature layers are extracted from the feature image.

[0136] Specifically, in one embodiment, the extraction module is further used to:

[0137] Perform image preprocessing on the gastroscope image to be detected.

[0138] Specifically, in one embodiment, the selection module is specifically used for:

[0139] Get the trained target detection model;

[0140] Use the target detection model to determine the confidence of the pending prediction box.

[0141] Regarding the gastroscope image target detection device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0142] The gastroscopic image target detection device provided in the embodiment of the present application is used to execute the gastroscopic image target detection method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0143] An embodiment of the present application provides an electronic device for executing the gastroscopic image target detection method provided in the above embodiment.

[0144] like Figure 6, which is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 60 includes: at least one processor 61 and a memory 62;

[0145] The memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the gastroscopic image target detection method provided in the above embodiment.

[0146] An electronic device provided in an embodiment of the present application is used to execute the gastroscopic image target detection method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0147] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the gastroscopic image target detection method provided in any of the above embodiments is implemented.

[0148] The storage medium containing computer executable instructions in the embodiment of the present application can be used to store computer executable instructions of the gastroscopic image target detection method provided in the aforementioned embodiment. Its implementation method and principle are the same and will not be repeated here.

[0149] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting a target in a gastroscope image, characterized in that: include: Acquire a gastroscopic image to be detected; Extracting multiple feature layers of the gastroscope image to be detected; Determining target frames of multiple scales according to the multiple feature layers; Selecting a prediction frame from the target frame according to the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame; The target frame refers to the predicted lesion location frame; Obtaining a category label corresponding to the prediction frame and an image recognition result of the gastroscopic image to be detected; When the category label is the same as the image recognition result, determining the target detection result of the gastroscopic image to be detected according to the prediction frame and the image recognition result; The extracting of multiple feature layers of the gastroscope image to be detected includes: Determining a plurality of characteristic images according to the gastroscopic image to be detected; Extracting multiple feature layers from the feature image according to different downsampling step sizes; The selecting a prediction frame from the target frame according to the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame comprises: Obtaining the maximum distance between the lesion point and each frame in the target frame; Filtering the prediction frame to be determined according to the maximum distance and the eccentric interval corresponding to each target frame; Get the confidence level corresponding to each pending prediction box; The prediction box is determined according to the confidence levels corresponding to the pending prediction boxes.

2. The method according to claim 1, characterized in that Also includes: When the category label is different from the image recognition result, return to the step of selecting a prediction box from the target box.

3. The method according to claim 1, characterized in that The step of obtaining the image recognition result of the gastroscopic image to be detected includes: Get the preset neural network model; The gastroscope image to be detected is input into the neural network model to determine the image recognition result of the gastroscope image to be detected by using the neural network model.

4. The method according to claim 1, characterized in that: Before extracting the multiple feature layers of the gastroscopic image to be detected, the method further includes: Perform image preprocessing on the gastroscopic image to be detected.

5. The method according to claim 1, characterized in that The obtaining of the confidence level corresponding to each pending prediction box includes: Get the trained target detection model; The target detection model is used to determine the confidence of the undetermined prediction box.

6. A gastroscope image target detection device, characterized in that: include: A first acquisition module is used to acquire a gastroscopic image to be detected; An extraction module, used for extracting multiple feature layers of the gastroscopic image to be detected; A determination module, used to determine target frames of multiple scales according to the multiple feature layers; The target frame refers to the predicted lesion location frame; A selection module, configured to select a prediction frame from the target frame according to the positional relationship between the lesion point in the gastroscopic image to be detected and each target frame; A second acquisition module is used to acquire the category label corresponding to the prediction frame and the image recognition result of the gastroscopic image to be detected; A detection module, configured to determine a target detection result of the gastroscopic image to be detected according to the prediction frame and the image recognition result when the category label is the same as the image recognition result; The extraction module is specifically used for: Determining a plurality of characteristic images according to the gastroscopic image to be detected; Extracting multiple feature layers from the feature image according to different downsampling step sizes; The selection module is specifically used for: Obtaining the maximum distance between the lesion point and each frame in the target frame; Filtering the prediction frame to be determined according to the maximum distance and the eccentric interval corresponding to each target frame; Get the confidence level corresponding to each pending prediction box; The prediction box is determined according to the confidence levels corresponding to the pending prediction boxes.

7. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 5 is implemented.

Images