Illumination robust digestive endoscopy image part identification method and system

Abstract

The invention relates to the technical field of medical auxiliary diagnosis, and provides an illumination-robust digestive endoscopy image part recognition method, which comprises the following steps: acquiring a digestive endoscopy image; inputting the digestive endoscopy image into a part recognition model and outputting a part recognition result; the part identification model comprises a backbone feature extraction network for extracting backbone features of a digestive endoscopy image; the normalization module is used for carrying out channel normalization on the backbone features to obtain normalized features; the differentiable gating module is used for carrying out channel pooling on the backbone features to obtain a gating channel selection vector of each channel; the fusion module fuses the backbone features and the normalized features based on gating channel selection vectors to obtain fusion features; and the classification module performs classification processing on the fusion features to obtain a part identification result. The invention further provides an intelligent auxiliary diagnosis system, a computer program product and electronic equipment. The method can reduce the influence of illumination on the detection precision of the part recognition model, and improves the part recognition precision.

Description

Technical Field

[0001] The present invention relates to the technical field of medical auxiliary diagnosis, and in particular to a method and system for identifying parts of digestive endoscopy images with robust illumination. Background Art

[0002] In the digestive endoscopy scene, rapid movement and lighting sources may cause large differences in the brightness of the collected images, affecting the detection accuracy of the artificial intelligence-assisted detection model. Existing methods mainly consider improving the detection effect in complex lighting environments through image enhancement. However, the method of performing image enhancement first and then target detection not only increases the model calculation amount and affects the real-time performance of detection, but also changes the statistical distribution law of pixels when enhancing the image. However, the evaluation indicators of image enhancement tasks and target detection tasks are not unified, and there are large differences between the two. Therefore, enhancing the image cannot guarantee the improvement of detection accuracy, but may lead to a decrease in detection accuracy. Summary of the invention

[0003] The present invention aims to at least solve the technical problems that the existing digestive endoscopy image part recognition scheme adopts a method of first enhancing the image to improve the brightness difference and then performing part recognition, which affects the real-time performance of detection, cannot guarantee the improvement of detection accuracy, and may lead to a decrease in detection accuracy. The present invention provides a digestive endoscopy image part recognition method and system with robust illumination.

[0004] In order to achieve the above-mentioned purpose of the present invention, according to the first aspect of the present invention, the present invention provides a method for illumination-robust digestive endoscopy image part recognition, comprising: acquiring a digestive endoscopy image; inputting the digestive endoscopy image into a part recognition model, and the part recognition model outputs a part recognition result; wherein the part recognition model comprises: a backbone feature extraction network, for extracting backbone features of the digestive endoscopy image; a normalization module, for performing channel normalization processing on the backbone features to obtain normalized features; a differentiable gating module, for performing channel pooling processing on the backbone features, and obtaining a gating channel selection vector for each channel according to the pooling processing result of each channel; a fusion module, for fusing the backbone features and the normalization features based on the gating channel selection vector of the channel to obtain a fused feature; and a classification module, for performing classification processing on the fused features to obtain a part recognition result.

[0005] In order to achieve the above-mentioned purpose of the present invention, according to the second aspect of the present invention, the present invention provides an intelligent auxiliary diagnosis system for implementing the illumination-robust digestive endoscopy image part recognition method described in the first aspect of the present invention, including: an image acquisition module for acquiring digestive endoscopy images; a recognition module for inputting the digestive endoscopy images into a part recognition model, and the part recognition model outputs a part recognition result; wherein the part recognition model includes: a backbone feature extraction network for extracting backbone features of digestive endoscopy images; a normalization module for performing channel normalization processing on the backbone features to obtain normalized features; a differentiable gating module for performing channel pooling processing on the backbone features, and obtaining a gating channel selection vector for each channel according to the pooling processing result of each channel; a fusion module for fusing the backbone features and the normalized features based on the gating channel selection vector of the channel to obtain a fused feature; and a classification module for performing classification processing on the fused features to obtain a part recognition result.

[0006] In order to achieve the above-mentioned purpose of the present invention, according to the third aspect of the present invention, the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of a lighting-robust digestive endoscopy image part recognition method described in the first aspect of the present invention.

[0007] In order to achieve the above-mentioned purpose of the present invention, according to the fourth aspect of the present invention, the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a lighting-robust digestive endoscopy image part recognition method as described in the first aspect of the present invention.

[0008] The beneficial technical effects of the present invention are as follows: the backbone features of the digestive endoscopy image are extracted using a backbone feature extraction network, the backbone features are channel normalized so that the obtained normalized features are not affected by illumination differences, and then a differentiable gating module is used to obtain a gating channel selection vector for each channel of the backbone features. The purpose of the gating channel selection vector is to select the channels in the backbone features that are more affected by illumination changes, and then a fusion module is used to fuse the backbone features and the normalized features based on the channel gating channel selection vector to obtain a fused feature that eliminates the illumination effect, and finally a classification module classifies the fused features to obtain a part recognition result. Through the above process, the present invention can reduce the influence of illumination on the detection accuracy of the part recognition model, and the present invention does not enhance the original digestive endoscopy image, does not change the statistical distribution law of pixels, improves the part recognition accuracy and efficiency, and solves the problem of reduced part detection accuracy and low real-time performance in the prior art under complex illumination conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a flow chart of a method for recognizing parts of digestive endoscopy images with robust illumination in a preferred embodiment of the present invention;

[0010] Figure 2 It is a structural schematic diagram of a part recognition model in a preferred embodiment of the present invention;

[0011] Figure 3 It is an implementation flow chart of a method for recognizing parts of digestive endoscopy images with robust illumination in an application scenario of the present invention;

[0012] Figure 4 is a structural schematic diagram of an intelligent auxiliary diagnosis system in a preferred embodiment of the present invention;

[0013] Figure 5 It is a schematic structural diagram of an electronic device in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0014] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0015] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0016] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0017] The present invention provides a method for robust illumination of digestive endoscopy image part recognition, and the execution subject of the method includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided in the embodiment of the present application. In other words, the method for robust illumination of digestive endoscopy image part recognition provided by the present invention can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0018] The present invention discloses a method for robust illumination of digestive endoscopy image part recognition. In a preferred embodiment, Figure 1 As shown, the method includes:

[0019] Step S1, obtaining a digestive endoscopy image.

[0020] In this embodiment, the digestive endoscopic image may be derived from an image collected by an endoscope system used for the examination of a certain digestive tract organ. The digestive endoscopic image may be read from the memory of the endoscope system, or it may be extracted from a digestive endoscopic image database. The digestive tract organs include the pharynx, esophagus, stomach, small intestine, large intestine, etc., and the corresponding endoscope systems include esophagoscopic system, gastroscopic system, small enteroscope system, colonoscope system, duodenoscope system, etc. There are multiple anatomical parts (referred to as parts) for each organ in the digestive tract. For example, the pharynx is divided into three parts: the nasopharynx, oropharynx, and hypopharynx; the stomach is divided into four parts: the gastric cardia, the gastric fundus, the gastric body, and the pylorus; and the small intestine is divided into three parts: the duodenum, the jejunum, and the ileum. More than one part of the digestive tract organ may be located within the field of view of the endoscope system camera at the same time. Therefore, there may be more than one part that needs to be identified in a digestive endoscopic image.

[0021] Step S2, inputting the digestive endoscopy image into a part recognition model, and the part recognition model outputs a part recognition result. The part recognition model is a pre-trained neural network model. In one example, the part recognition result includes the predicted probability of all parts in a certain digestive tract organ, and the predicted probability of a certain part indicates the probability of the part existing in the digestive endoscopy image. The predicted probability value range can be 0 to 1, and can also be expressed as a percentage, ranging from 0 to 100%.

[0022] Among them, refer to Figure 2 As shown, the part recognition model includes:

[0023] (1) Backbone feature extraction network, used to extract the backbone features of digestive endoscopy images.

[0024] In this embodiment, the backbone feature extraction network is mainly used to extract the semantic features of digestive endoscopy images, and is not limited to selecting existing VGGNet, Darknet, Ghostnet, CSPDarkNet53, MobileNetV3 and other backbone feature extraction network structures, and the CSPDarkNet53 model is preferably used as the backbone network. The processing process of the backbone feature extraction network can be expressed as:

[0025] x=f(I)

[0026] Where x represents the backbone features extracted by the backbone feature extraction network, f(·) represents the processing process of the backbone feature extraction network, and I represents the digestive endoscopy image input into the backbone feature extraction network.

[0027] (2) Normalization module: performs channel normalization on the backbone features to obtain normalized features.

[0028] In this embodiment, the number of channels of the backbone feature is equal to the number of convolution kernels of the backbone feature extraction network, and is also equal to the number of output channels of the backbone feature extraction network, and one channel of the backbone feature corresponds to one feature map. Each channel of the backbone feature is normalized, and the features after normalization of all channels constitute the normalized features. The channels of the normalized features correspond one to one with the channels of the backbone features. Each channel of the backbone feature maps the characteristic value of the channel to the interval [0, 1] through the normalization operation. Existing normalization methods such as Min-Max normalization, z-score standardization, log function normalization, and inverse tangent function normalization can be used. By performing normalization operations on each channel of the backbone feature, the normalized features are not affected by illumination differences.

[0029] (3) The differentiable gating module performs channel pooling on the backbone features and obtains the gating channel selection vector of each channel according to the pooling processing result of each channel.

[0030] In this embodiment, pooling processing is performed in each channel of the backbone feature, and the feature map of the channel (i.e., channel feature) is divided into multiple pooling regions according to the pooling window on each channel, each pooling region includes multiple feature points, and each pooling region obtains a pooling feature value through pooling processing, and the pooling processing result of each channel includes multiple pooling feature values. The pooling processing is preferably, but not limited to, average pooling processing or maximum pooling processing. The size of the pooling feature value characterizes the intensity of the corresponding pooling area affected by light.

[0031] (4) A fusion module fuses the backbone features and normalized features based on the gated channel selection vector of the channel to obtain a fused feature. Specifically, the backbone features and normalized features are fused feature point by feature point.

[0032] (5) Classification module: classifies the fused features to obtain part recognition results.

[0033] In this embodiment, the classification module is preferably but not limited to a fully connected network, and the fully connected network includes more than one cascaded fully connected layer. x It is expressed as:

[0034] c x =R(f c (Wx n+1 + b));

[0035] Where W and b represent the weight matrix and bias parameter of the fully connected network of the classification module, respectively, and f c (·) represents a fully connected network, and R(·) represents the activation function of the fully connected network. n+1 Represents fusion features.

[0036] In this embodiment, preferably, the differentiable gating module performs:

[0037] Step A1: pool the channel features of the backbone feature channel c to obtain the channel pooling feature vector p x,c :

[0038] p x，c ={p x,c,1 , p x,c,2 , ..., p x,c,M}

[0039] Step A2: Pooling feature vector p based on channels x，c Get the gated channel selection vector g for channel c x,c :

[0040] g x，c = {g x,c,1 , g x，c，2 , …, g x,c,M}

[0041] g x，c，m =max(ε,abs(p x,c,m ))

[0042] Where m represents the index of the pooling region of the channel, m∈[1,M], M represents the number of pooling regions in channel c, and the number of pooling regions in each channel can be the same; p x,c,m Represents the channel pooling feature vector p of the backbone feature x channel c x,cThe pooling feature value corresponding to the mth pooling area in g; x,c,m Represents the gate selection value corresponding to the mth pooling area in channel c of the backbone feature x; max(,) represents the maximum value function; abs(·) represents the absolute value function; ε represents the selection coefficient, ε∈[0,0.7], preferably 0.5; c is a positive integer, c∈[1,C], and C represents the total number of channels of the backbone feature.

[0043] In this embodiment, the above steps A1 and A2 are performed for each channel of the backbone features to obtain the gated channel selection vector of each channel. The backbone features and normalized features can be efficiently selected along the channel dimension through the differentiable gating module. x ,c,m=max(ε,abs(p x，c，m )) can select channels that are greatly affected by illumination changes. Further preferably, the pooling process combines pooling and normalization functions, so that the obtained gated channel selection vector is a series of binary indicators to improve computational efficiency, specifically:

[0044]

[0045] Among them, R m represents the mth pooling region in channel c, |R m | represents the size of the mth pooling area, which can be the number of feature points in the mth pooling area. In an example, the mth pooling area is 3×3, then |R m |=9; p represents the regularization coefficient, which is a positive number. i Represents the feature value of the i-th feature point in the m-th pooling area.

[0046] In this implementation, preferably, the fusion module performs:

[0047] Step B1, multiply the feature values ​​of the area corresponding to the position of the mth pooling area in the backbone feature channel c in the normalized feature by the gated selection value corresponding to the mth pooling area in the channel c, and obtain the intermediate feature map of the mth pooling area in the channel c.

[0048] Specifically, the intermediate feature map x1 of the mth pooling area in channel c c，m The calculation formula is:

[0049] x1 c，m =g x，c，m ·x c，m

[0050] x′ c，m It represents the feature subgraph represented by the region corresponding to the position of the mth pooling region in the channel c of the backbone feature channel c in the normalized feature x′. The “·” symbol represents gx，c，m and x′ c，m Multiply the eigenvalues ​​one by one.

[0051] Step B2: The mth pooled region in the backbone feature channel c corresponds to the feature map x c，m and the intermediate feature map x1 of the mth pooling region in channel c c，m Perform weighted summation of each eigenvalue to obtain the fused feature map x of the mth pooling area in channel c n+1 (c, m):

[0052] x n+1 (c, m) = (1-α)·x c,m +α·x1 c,m

[0053] =(1-α)·x c,m +α·g x，c，m ·x c,m

[0054] The symbol "+" represents (1-α)·x c,m and α·x1 c,m The eigenvalues ​​are added one by one, and α represents the fusion weight parameter, which ranges from 0.1 to 0.5.

[0055] Step B3, integrating the fusion feature maps of all pooled areas of C channels to obtain fusion features.

[0056] Specifically, the fused feature maps of all pooled regions of each channel are combined together to obtain the fused feature map of the channel. For example, the fused feature map x of channel c is n+1 (c) The fused feature map x of the M pooling regions through channel c n+1 (c, m) are concatenated in order of position. The fusion feature maps of C channels are then combined to obtain the fusion feature.

[0057] In this embodiment, the training process of the part recognition model includes:

[0058] Step (1) obtains a sample image set and sets a part category label for each sample image.

[0059] In actual application, each digestive tract organ may correspond to a part recognition model. During training, multiple digestive endoscopy images taken by the endoscope system corresponding to the digestive tract organ are obtained as a sample image set. A part category label can be manually set for each sample image. Specifically, the part category label of the part that actually exists in the sample image is set to 1, and the part category label of the part that does not exist is set to 0;

[0060] Step (2) constructs the network structure of the part recognition model and initializes the network parameters.

[0061] Step (3) uses the sample image set to train the part recognition model, calculates the loss function during the training process and updates the network parameters of the part recognition model through the back propagation algorithm until the training stop condition is reached.

[0062] In this embodiment, the training stop condition is preferably but not limited to the number of training times reaching a preset maximum number of training times, or the loss function value is less than or equal to a preset loss threshold. Step (3) also includes dividing the sample image set into a training set, a test set, and a validation set, using the training set to perform multiple batches of training on the network of the part recognition model, calculating the loss function after each batch of training, calculating the gradient of the loss function and using the back propagation algorithm to update the network parameters of the model. The trained model is tested and verified using the test set and the validation set. When the test and verification pass, the network parameters of the model are saved to obtain a trained part recognition model. The loss function Loss is expressed as:

[0063]

[0064] Among them, y i′，b represents the part category label of the i′th sample image part b, which is 0 or 1; p i′，b represents the predicted probability of the output part b of the part recognition model of the i-th sample image, N represents the number of training sample images, i′∈[1,N], B represents the number of parts of the digestive tract organs corresponding to the part recognition model, b∈[1,B].

[0065] In a preferred embodiment, the normalization module performs:

[0066] Step B1, obtain the channel mean μ of the backbone feature channel c c (x c ) and channel variance σ c (x c ).

[0067] Step B2, determine the channel normalized feature x of channel c c 'for:

[0068]

[0069] Where c is a positive integer, c∈[1,C], C represents the total number of channels of the backbone feature; x c represents the channel feature of the backbone feature channel c; γ represents the feature mean correction coefficient obtained by learning; β represents the feature variance correction coefficient obtained by learning;

[0070] Step B3: The channel normalized features of the C channels of the backbone features form the normalized features.

[0071] In this embodiment, γ and β are learned during the part recognition model training process. In step (2) of the above training process, γ and β are initialized, and they can be initialized with random values ​​or based on empirical values. In step (3), the gradient of the loss function is calculated and the network parameters of the model as well as γ and β are updated using the back propagation algorithm. γ and β are dynamically adjusted during the model training process. In this embodiment, γ and β are used to enable the normalized features to follow the characteristic distribution characteristics of the input digestive endoscopy image, thereby avoiding the loss of the distribution information of the original image and improving the accuracy of part recognition.

[0072] In a preferred embodiment, in step S1, obtaining a digestive endoscopic image includes:

[0073] Step S11, obtaining the original digestive endoscopic image I 0 ;

[0074] Step S12: the original digestive endoscopy image I is processed according to the following formula: 0 Processing to obtain digestive endoscopic image I:

[0075]

[0076] Among them, μ(I 0 ) represents the original digestive endoscopic image I 0 The pixel mean, σ(I 0 ) represents the original digestive endoscopic image I 0 The pixel variance, σ g represents the illumination adjustment variance, μ g Represents the illumination adjustment mean.

[0077] In this embodiment, the pixel distribution of the original digestive endoscopic image is corrected and preprocessed through steps S11 and S12, and the obtained digestive endoscopic image I is made consistent with the original digestive endoscopic image I while removing noise. 0 Having the same mean and variance distribution improves the accuracy of subsequent part recognition.

[0078] In this embodiment, before executing step S11, the method further includes:

[0079] Step S01, obtaining a plurality of digestive endoscopy reference images satisfying the illumination condition. Assume that there are N1 digestive endoscopy reference images satisfying the illumination condition, and N1 is greater than or equal to 100. The illumination condition refers to good illumination, and all brightness components of pixels in the image are within a preset brightness range.

[0080] Step S02, calculating the average of the pixel means of multiple digestive endoscopy reference images, recording it as a first average value, and using the first average value as the illumination adjustment mean μ g .

[0081]

[0082] Among them, μ q It represents the pixel mean of the qth digestive endoscopy reference image, that is, the mean pixel value.

[0083] Step S03, calculating the average value of the pixel variances of multiple digestive endoscopy reference images, recording it as the second average value, and using the second average value as the illumination adjustment variance σ g .

[0084]

[0085] Among them, σ q It represents the pixel variance of the qth digestive endoscopy reference image, that is, the pixel value variance.

[0086] Figure 3 The present invention provides Figure 3 It is a schematic diagram of the implementation process of a method for recognizing parts of digestive endoscopy images with robust illumination in an application scenario of the present invention. In this application scenario, the digestive endoscopy image is a gastrointestinal endoscopy image. The classification head corresponds to the classification module.

[0087] The present invention also discloses an intelligent auxiliary diagnosis system for implementing the above-mentioned robust illumination digestive endoscopy image part recognition method. In a preferred embodiment, Figure 4 As shown, the device comprises:

[0088] An image acquisition module, for acquiring digestive endoscopy images;

[0089] The recognition module inputs the digestive endoscopy image into the part recognition model, and the part recognition model outputs the part recognition result;

[0090] Among them, the part recognition model includes:

[0091] Backbone feature extraction network, used to extract the backbone features of digestive endoscopy images;

[0092] Normalization module, which performs channel normalization processing on backbone features to obtain normalized features;

[0093] The differentiable gating module performs channel pooling on the backbone features and obtains the gating channel selection vector of each channel according to the pooling processing result of each channel;

[0094] The fusion module fuses the backbone features and normalized features based on the channel-based gated channel selection vector to obtain the fused features;

[0095] The classification module classifies the fused features to obtain the part recognition results.

[0096] In this embodiment, the image acquisition module and the recognition module correspond one-to-one to step S1 and step S2 of the above-mentioned illumination-robust digestive endoscopy image part recognition method, and are not described in detail here.

[0097] The present invention also discloses a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned illumination-robust digestive endoscopy image part recognition method provided by the present invention. The computer program product should be understood as a software product that mainly implements its solution through a computer program, such as a program product integrated in the cloud or a software library.

[0098] The present invention also discloses an electronic device. In one embodiment, the electronic device includes at least one processor; and a memory connected to the at least one processor in communication; wherein:

[0099] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a method for identifying parts of digestive endoscopy images with robust illumination provided by the present invention.

[0100] like Figure 5 , which is a schematic diagram of the structure of an electronic device for a method for recognizing parts of digestive endoscopy images with robust illumination provided by an embodiment of the present invention. The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a program for a method for recognizing parts of digestive endoscopy images with robust illumination.

[0101] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (for example, executing a method for robust digestive endoscopy image site recognition, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device and process data.

[0102] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 can also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a method program for robust illumination of digestive endoscopy image parts recognition, but also can be used to temporarily store data that has been output or is to be output.

[0103] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0104] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0105] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 5The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0106] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0107] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0108] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0109] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "an implementation", "a preferred implementation" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0110] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for identifying parts of digestive endoscopy images with robust illumination, characterized in that: include: Obtain digestive endoscopic images; The digestive endoscopy image is input into the part recognition model, and the part recognition model outputs the part recognition result; Among them, the part recognition model includes: Backbone feature extraction network, used to extract the backbone features of digestive endoscopy images; Normalization module, which performs channel normalization processing on backbone features to obtain normalized features; The differentiable gating module performs channel pooling on the backbone features and obtains the gating channel selection vector of each channel according to the pooling processing result of each channel; The fusion module fuses the backbone features and normalized features based on the channel-based gated channel selection vector to obtain the fused features; The classification module classifies the fused features to obtain the part recognition results.

2. The method for identifying parts of digestive endoscopy images with robust illumination according to claim 1, characterized in that: The normalization module performs: Get the channel mean μ of the backbone feature channel c c (x c ) and channel variance σ c (x c ); Determine the channel-normalized feature x of channel c c 'for: Where c is a positive integer, c∈[1,C], C represents the total number of channels of the backbone feature; x c represents the channel feature of the backbone feature channel c; γ represents the feature mean correction coefficient obtained by learning; β represents the feature variance correction coefficient obtained by learning; The channel normalized features of the C channels of the backbone feature constitute the normalized features.

3. The method for identifying parts of digestive endoscopy images with robust illumination according to claim 1 or 2, characterized in that: The differentiable gating module performs: Pool the channel features of the backbone feature channel c to obtain the channel pooling feature vector p x,c : p x,c ={p x,c,1 ,p x,c,2 ,...,p x,c,M } Based on the channel pooling feature vector p x,c Get the gated channel selection vector g for channel c x,c : g x,c ={g x,c,1 ,g x,c,2 ,...,g x,c,M } g x,c,m =max(ε,abs(p x,c,m )) Where m represents the pooling region index of the channel, m∈[1,M], M represents the number of pooling regions in channel c; p x,c,m Represents the channel pooling feature vector p x,c The pooling feature value corresponding to the mth pooling area in g; x,c,m represents the gate selection value corresponding to the mth pooling area in channel c; max(,) represents the maximum value function; abs(·) represents the absolute value function; ε represents the selection coefficient, ε∈[0,0.7]; c is a positive integer, c∈[1,C], C represents the total number of channels of the backbone feature.

4. The method for identifying parts of digestive endoscopy images with robust illumination according to claim 3, characterized in that: The pooling process is an average pooling process or a maximum pooling process.

5. The method for identifying parts of digestive endoscopy images with robust illumination according to claim 3, characterized in that: The fusion module performs: Multiply the feature values ​​of the area corresponding to the position of the mth pooling area in the backbone feature channel c in the normalized feature by the gated selection value corresponding to the mth pooling area in the channel c to obtain the intermediate feature map of the mth pooling area in the channel c; Perform weighted summation of feature values ​​one by one on the feature map corresponding to the mth pooling area in the backbone feature channel c and the intermediate feature map of the mth pooling area in the channel c to obtain a fused feature map of the mth pooling area in the channel c; The fusion feature maps of all pooling areas of C channels are integrated to obtain the fusion feature.

6. The method for identifying parts of digestive endoscopy images with robust illumination as described in claim 1, 2, 4 or 5, characterized in that: The step of obtaining a digestive endoscopic image comprises: Obtaining the original digestive endoscopic image I0; The original digestive endoscopic image I0 is processed according to the following formula to obtain the digestive endoscopic image I: Among them, μ(I0) represents the pixel mean of the original digestive endoscopic image I0, σ(I0) represents the pixel variance of the original digestive endoscopic image I0, and σ g represents the illumination adjustment variance, μ g Represents the illumination adjustment mean.

7. The method for identifying parts of digestive endoscopy images with robust illumination according to claim 6, characterized in that: The method further comprises: Acquire multiple digestive endoscopy reference images that meet lighting conditions; Calculating an average of pixel means of a plurality of digestive endoscopy reference images, recording the average as a first average, and using the first average as a lighting adjustment mean; The average value of the pixel variances of the plurality of digestive endoscopy reference images is calculated and recorded as a second average value, and the second average value is used as the illumination adjustment variance.

8. An intelligent auxiliary diagnosis system, characterized in that: The method for realizing the illumination-robust digestive endoscopy image part recognition as claimed in any one of claims 1 to 7 comprises: An image acquisition module, for acquiring digestive endoscopy images; The recognition module inputs the digestive endoscopy image into the part recognition model, and the part recognition model outputs the part recognition result; Among them, the part recognition model includes: Backbone feature extraction network, used to extract the backbone features of digestive endoscopy images; Normalization module, which performs channel normalization processing on backbone features to obtain normalized features; The differentiable gating module performs channel pooling on the backbone features and obtains the gating channel selection vector of each channel according to the pooling processing result of each channel; The fusion module fuses the backbone features and normalized features based on the channel-based gated channel selection vector to obtain the fused features; The classification module classifies the fused features to obtain the part recognition results.

9. A computer program product, comprising a computer program, characterized in that When the computer program / instructions are executed by a processor, the steps of a method for illumination-robust digestive endoscopy image part recognition as described in any one of claims 1-7 are implemented.

10. An electronic device, characterized in that: The electronic device comprises: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a method for identifying parts of digestive endoscopy images with robust illumination as described in any one of claims 1 to 7.

Images