Oropharyngeal lesion feature extraction method and device and electronic equipment

Abstract

The application provides an oropharyngeal lesion feature extraction method and device and electronic equipment. The method calls a trained lesion detection model to detect white light images, NBI images and CT images of the same case of the oropharynx, respectively, to obtain white light lesion images, NBI lesion images and CT lesion images of each lesion. Then, roughness quantitative features, ulcer / corrosion learning features of the target lesion are obtained from the white light lesion images, spot / microvessel dilation learning features of the target lesion are obtained from the NBI lesion images, and symmetry quantitative features, boundary definition quantitative features, shape quantitative features and invasion quantitative features of the target lesion are obtained from the CT lesion images. Finally, according to all quantitative features and all learning features, oropharyngeal lesion features are obtained. The oropharyngeal lesion features obtained by the application can provide accurate and comprehensive reference, and the type determination accuracy of the oropharyngeal lesion is also high.

Description

Technical Field

[0001] This application relates to the field of medical auxiliary technology, and in particular to a method, device and electronic device for extracting features of oropharyngeal lesions. Background Technology

[0002] Malignant lesions of the oropharynx mainly include carcinoma and lymphoma. Oropharyngeal carcinoma primarily includes carcinoma of the base of the tongue, tonsil, and soft palate. Oropharyngeal carcinoma accounts for approximately 7% of head and neck malignancies. Due to its deep location and high invasiveness, it is a tumor with a poor prognosis. Because the clinical manifestations of oropharyngeal carcinoma and oropharyngeal lymphoma are quite similar, the lesion characteristics are difficult to distinguish, leading to misdiagnosis and affecting subsequent treatment plans.

[0003] Therefore, the current method for identifying oropharyngeal lesion types suffers from low accuracy in lesion feature recognition, and this needs to be improved. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for extracting features of oropharyngeal lesions, in order to alleviate the technical problem of low accuracy in lesion feature identification when determining the type of oropharyngeal lesions.

[0005] To address the aforementioned technical problems, this application provides the following technical solutions:

[0006] This application provides a method for extracting features of oropharyngeal lesions, including:

[0007] Acquire white light images, NBI images, and CT images of the oropharynx of the same case, and call the trained lesion detection model to detect the white light images, NBI images, and CT images respectively to obtain white light lesion images, NBI lesion images, and CT lesion images of each lesion;

[0008] The roughness information of the target lesion is obtained from the target white light lesion image. Based on the roughness information, the first quantitative feature of the target lesion is obtained. The trained ulcer / erosion recognition model is called to recognize the target white light lesion image to obtain the first recognition result of the ulcer / erosion. Based on the first recognition result, the first learned feature of the target lesion is obtained.

[0009] The trained speckle / microvascular dilatation recognition model is invoked to identify the target NBI lesion image, and a second recognition result of speckle / microvascular dilatation is obtained. Based on the second recognition result, a second learned feature of the target lesion is obtained.

[0010] Symmetry information of the target lesion is obtained from the CT images of each lesion, and a second quantitative feature of the target lesion is obtained based on the symmetry information; boundary clarity information of the target lesion is obtained from the target CT images, and a third quantitative feature of the target lesion is obtained based on the boundary clarity information; shape information of the target lesion is obtained from the target CT images, and a fourth quantitative feature of the target lesion is obtained based on the shape information; invasiveness information of the target lesion is obtained from the target CT images, and a fifth quantitative feature of the target lesion is obtained based on the invasiveness information.

[0011] Based on all quantitative features and all learned features, the features of oropharyngeal lesions are obtained.

[0012] Meanwhile, this application also provides an oropharyngeal lesion feature extraction device, including:

[0013] The acquisition module is used to acquire white light images, NBI images and CT images of the oropharynx of the same case, and call the trained lesion detection model to detect the white light images, NBI images and CT images respectively to obtain white light lesion images, NBI lesion images and CT lesion images of each lesion;

[0014] The first obtaining module is used to obtain roughness information of the target lesion from the target white light lesion image, obtain the first quantitative feature of the target lesion based on the roughness information, call the trained ulcer / erosion recognition model to recognize the target white light lesion image, obtain the first recognition result of ulcer / erosion, and obtain the first learned feature of the target lesion based on the first recognition result.

[0015] The second module is used to call the trained speckle / microvascular dilatation recognition model to recognize the target NBI lesion image, obtain the second recognition result of speckle / microvascular dilatation, and obtain the second learned feature of the target lesion based on the second recognition result;

[0016] The third module is used to obtain symmetry information of the target lesion from the CT images of each lesion, and obtain a second quantitative feature of the target lesion based on the symmetry information; obtain boundary clarity information of the target lesion from the target CT images, and obtain a third quantitative feature of the target lesion based on the boundary clarity information; obtain shape information of the target lesion from the target CT images, and obtain a fourth quantitative feature of the target lesion based on the shape information; obtain invasiveness information of the target lesion from the target CT images, and obtain a fifth quantitative feature of the target lesion based on the invasiveness information.

[0017] The fourth module is used to obtain oropharyngeal lesion features based on all quantified features and all learned features.

[0018] This application also provides an electronic device, including a memory and a processor; the memory stores an application program, and the processor is used to run the application program in the memory to perform the steps in the oropharyngeal lesion feature extraction method described in any of the above claims.

[0019] Beneficial Effects: This application provides a method, apparatus, and electronic device for extracting features of oropharyngeal lesions. The method first acquires white light images, NBI images, and CT images of the oropharynx of the same case, and then calls a trained lesion detection model to detect the white light images, NBI images, and CT images respectively, obtaining white light lesion images, NBI lesion images, and CT lesion images of each lesion. Then, from the white light lesion images, the roughness quantitative features and ulcer / erosion learning features of the target lesion are obtained; from the NBI lesion images, the spot / microvascular dilation learning features of the target lesion are obtained; and from the CT lesion images, the symmetry quantitative features, boundary clarity quantitative features, shape quantitative features, and invasiveness quantitative features of the target lesion are obtained. Finally, based on all quantitative features and all learning features, the oropharyngeal lesion features are obtained. After acquiring white light images, NBI images, and CT images of the oropharynx, this application can automatically quantify and learn the lesion status of the target lesion in the oropharynx from multiple dimensions, and synthesize the features of the oropharyngeal lesion. These features can provide an accurate and comprehensive reference for determining the type of the target lesion, with high efficiency and high accuracy in determining the type of the oropharyngeal lesion. Attached Figure Description

[0020] The technical solution and other beneficial effects of this application will become apparent from the following detailed description of specific embodiments in conjunction with the accompanying drawings.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario of the oropharyngeal lesion feature extraction method provided in the embodiments of this application.

[0022] Figure 2 This is a flowchart illustrating the oropharyngeal lesion feature extraction method provided in the embodiments of this application.

[0023] Figure 3 This is a comparative schematic diagram of the rough and smooth surfaces of lesions in the embodiments of this application.

[0024] Figure 4 This is a schematic diagram of a white light image and a target white light lesion image in an embodiment of this application.

[0025] Figure 5 This is a comparative schematic diagram of the speckled microvascular dilation on the surface of the lesion in the embodiments of this application.

[0026] Figure 6 This is a schematic diagram of the asymmetrical distribution of lesions in an embodiment of this application.

[0027] Figure 7 This is a comparative schematic diagram showing the difference between unclear and clear lesion boundaries in the embodiments of this application.

[0028] Figure 8 This is a schematic diagram illustrating the calculation of lesion shape in an embodiment of this application.

[0029] Figure 9 This is a schematic diagram illustrating the calculation of lesion invasiveness in an embodiment of this application.

[0030] Figure 10 This is a schematic diagram of the oropharyngeal lesion feature extraction device provided in the embodiments of this application.

[0031] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0033] This application provides a method, apparatus, electronic device, and computer-readable storage medium for extracting features of oropharyngeal lesions. The oropharyngeal lesion feature extraction apparatus can be integrated into an electronic device, which can be a server or a terminal or other similar device.

[0034] Please see Figure 1 , Figure 1 This is a schematic diagram of an application scenario for the oropharyngeal lesion feature extraction method provided in this application embodiment. The scenario may include a terminal and a server. Terminals, servers, and terminals and servers are connected and communicate through the Internet composed of various gateways. The application scenario includes an image receiving device 11 and a server 12. The image receiving device 11 may be a device with human-computer interaction function. The server 12 includes a local server and / or a remote server.

[0035] The image receiving device 11 and the server 12 are located in a wireless network or a wired network to enable data interaction between them, wherein:

[0036] The user inputs white light images, NBI images, and CT images of the oropharynx from the same case into the image receiving device 11. The image receiving device 11 sends the three types of images and the oropharyngeal lesion feature extraction request to the server 12. The server 12 first calls the trained lesion detection model to detect the white light images, NBI images, and CT images respectively, obtaining white light lesion images, NBI lesion images, and CT lesion images of each lesion. Then, it calls a preset algorithm to obtain the roughness quantification features and ulcer / erosion learning features of the target lesion from the white light lesion image, the spot / microvascular dilation learning features of the target lesion from the NBI lesion image, and the symmetry quantification features, boundary clarity quantification features, shape quantification features, and invasiveness quantification features of the target lesion from the CT lesion image. Finally, based on all quantification features and all learning features, the oropharyngeal lesion features are obtained and returned to the image receiving device 11, which then displays the results to the user. This method can automatically quantify, learn, and synthesize the lesion status of target lesions in the oropharynx from multiple dimensions, and return the final oropharyngeal lesion features to the user. The user can use these features as a reference to determine whether the target lesion is oropharyngeal carcinoma or oropharyngeal lymphoma. This method is highly efficient, and because the extracted oropharyngeal lesion features are highly accurate and comprehensive, the accuracy in determining the type of oropharyngeal lesion is also high.

[0037] It should be noted that, Figure 1 The system scenario diagram shown is merely an example. The servers and scenarios described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of systems and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems. Detailed descriptions are provided below. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of embodiments.

[0038] Please see Figure 2 , Figure 2 This is a schematic flowchart of the first method for extracting oropharyngeal lesion features provided in this application embodiment. The method specifically includes:

[0039] S1: Obtain white light images, NBI images, and CT images of the oropharynx of the same case, and call the trained lesion detection model to detect the white light images, NBI images, and CT images respectively to obtain white light lesion images, NBI lesion images, and CT lesion images of each lesion.

[0040] In this embodiment, a case refers to a patient with at least one lesion in the oropharynx. Images of the oropharynx from the same case are acquired using medical-aided instruments, yielding white light images, NBI images, and CT images, each including all lesions in the oropharynx. The lesion detection model can be a YOLOv3 model. Before use, a professional endoscopist marks the lesions in the three types of images from a certain number of historical cases with rectangular bounding boxes, using this as training data. This training data is then used to train the lesion detection model.

[0041] After obtaining the three types of images, the trained lesion detection model is called to detect the lesions in the three types of images respectively. The model marks each detected lesion in the form of a rectangle and crops it by expanding it outward by a certain multiple according to the size of the rectangle (or it can be without cropping, and only the expanded size is recorded as the image size). This yields white light lesion images, NBI lesion images and CT lesion images for each lesion. When there are N lesions, N sets of white light lesion images, NBI lesion images and CT lesion images are obtained respectively.

[0042] S2: Obtain the roughness information of the target lesion from the target white light lesion image, and obtain the first quantitative feature of the target lesion based on the roughness information; call the trained ulcer / erosion recognition model to recognize the target white light lesion image, obtain the first recognition result of ulcer / erosion, and obtain the first learned feature of the target lesion based on the first recognition result.

[0043] When only one lesion exists in the oropharynx of a case, that lesion is designated as the target lesion. When two or more lesions exist, each lesion is designated as the target lesion in turn, and the white light image corresponding to the target lesion is the target white light lesion image. The ulcer / erosion recognition model can be the VGG16 model. Before using this model, a professional endoscopist first identifies the ulcer / erosion status of lesions in white light images of a certain number of historical cases, labels them with corresponding ulcer / erosion labels, and uses this training data to train the ulcer / erosion recognition model.

[0044] like Figure 3 As shown, when the lesion type is oropharyngeal carcinoma, the lesion surface appears rough in the white light image, i.e., the roughness is relatively high. When the lesion type is oropharyngeal lymphoma, the lesion surface appears smooth in the white light image, i.e., the roughness is relatively low. Since the two lesion types exhibit different surface roughness in the white light image, roughness information of the target lesion can be obtained from the target white light lesion image, and the roughness can be quantified to obtain the first quantitative feature. This method measures the type of the target lesion from the roughness dimension.

[0045] In one embodiment, the target white light lesion image includes a first lesion frame of the target lesion. The target white light lesion image is obtained based on the center of the first lesion frame and a preset magnification factor. S2 specifically includes: generating a first diagonal line and a second diagonal line intersecting at the center in the target white light lesion image to obtain four diagonal regions. Each diagonal region is divided by the edge of the first lesion frame, forming a first region close to the center and a second region far from the center. The first region forms a first image, and the second region forms a second image. First size information of the target white light lesion image and second size information of the first lesion frame in the target white light lesion image are obtained. Based on the first size information, the second size information, and the four diagonal regions, each first image is obtained. The image obtains the third size information of the first image and the fourth size information of each second image; it acquires the first color information of the target white light lesion image; for each diagonal region, it obtains the image roughness quantification value of the corresponding first image based on the first color information and the third size information of the corresponding first image; it obtains the image roughness quantification value of the corresponding second image based on the first color information and the fourth size information of the corresponding second image; for each diagonal region, it obtains the region roughness quantification value of each diagonal region based on the image roughness quantification value of the corresponding first image and the image roughness quantification value of the corresponding second image; it obtains the roughness information of the target lesion based on the four region roughness quantification values; and it obtains the first quantitative feature of the target lesion based on the roughness information.

[0046] like Figure 4 As shown, a target lesion exists in the lower right corner of the white light image. After detection by the trained lesion detection model, the first lesion bounding box of the target lesion is obtained, which is the smaller rectangle in the lower right corner. Based on the center of the first lesion bounding box and a preset magnification factor, a larger rectangle in the lower right corner is obtained. All contents within this rectangle constitute the target white light lesion image. The preset magnification factor can be set as needed, such as 1.2 times or 1.5 times. For ease of explanation, the larger rectangle is referred to as the image rectangle in the following embodiments. Based on this scaling relationship, a first diagonal and a second diagonal are generated in the target lesion white light image, which are the two dashed lines in the figure. The two diagonals intersect at the center of the first lesion bounding box, and the four vertices of the first lesion bounding box also fall on these two diagonals respectively. The first and second diagonals divide the image rectangle into four diagonal regions. For each diagonal region, with one side of the first lesion bounding box as the dividing line, it can be further divided into two regions: a first region closer to the center, which is triangular, and a second region farther from the center, which is trapezoidal. All content within the first region constitutes the first image, and all content within the second region constitutes the second image.

[0047] The first size information of the target white light lesion image includes the width and height of the image rectangle. The second size information of the first lesion frame includes the width and height of the first lesion frame. After obtaining this size information, combined with the division of the four diagonal regions, the third size information of each first image and the fourth size information of each second image can be obtained. The third size information includes the width and height of the first image, and the fourth size information includes the width and height of the second image. The first color information of the target white light lesion image includes the color value of each pixel. For each diagonal region, based on the color values ​​of all pixels and the height and width of the first image, the roughness of all content in the first image can be quantized to obtain the image roughness quantification value of the first image. Similarly, based on the color values ​​of all pixels and the height and width of the second image, the roughness of all content in the second image can be quantized to obtain the image roughness quantification value of the second image.

[0048] like Figure 4 As shown, taking the right diagonal region as the target diagonal region, the first image is denoted as P11, and the second image is denoted as P12. Using C... P11 The image coarseness quantification of the first image is represented by C. P12 The image coarseness quantification value of the second image is represented by C. P1 The region roughness quantification value representing the right diagonal region satisfies the following formula:

[0049]

[0050] Similarly, by successively taking the upper diagonal region, the left diagonal region, and the lower diagonal region as the target diagonal regions, and substituting the relevant data of each target diagonal region into Formula 1 above, the quantified value C of the regional roughness of the upper diagonal region can be obtained. P2 The regional roughness quantification value C of the left diagonal region P3 And the regional roughness quantification value C of the lower diagonal region. P4 Let label1 represent the roughness of the target lesion, then it satisfies the following formula:

[0051]

[0052] The image roughness quantification value of a given image represents the overall roughness difference between that image and the white light lesion image. When the lesion surface is rough, the overall roughness difference between the first image and the white light lesion image is small, while the overall roughness difference between the second image and the white light lesion image is large. Therefore, the image roughness quantification value of the first image is small, and the image roughness quantification value of the second image is large, resulting in a smaller regional roughness quantification value for the diagonal region. Conversely, when the lesion surface is smooth, the overall roughness difference between the first image and the white light lesion image is small, and the overall roughness difference between the second image and the white light lesion image is also small. Therefore, the image roughness quantification values ​​of both the first and second images are small, resulting in a larger regional roughness quantification value for the diagonal region relative to the roughness of the lesion surface. Therefore, when the four ratios are combined in both cases, there will be differences between the two combined values. The magnitude of the two combined values ​​is negatively correlated with the roughness of the target lesion. When the roughness of the target lesion is large, the combined value is small, and when the roughness of the target lesion is small, the combined value is large. Therefore, the combined value can be used as the first quantitative feature label1 of the roughness of the target lesion.

[0053] In one embodiment, the step of obtaining the image coarseness quantification values ​​of each first image and each second image specifically includes: sequentially determining each diagonal region as a target diagonal region, and sequentially determining the first image and second image corresponding to the target diagonal region as target images; obtaining the first color value of each pixel in the target white light lesion image; for each target image, obtaining the first pixel color average value of the target image based on each first color value, as well as the width and height of the target image; selecting a horizontal baseline and a vertical baseline from the target image according to preset selection conditions; obtaining the first coarseness quantification value of the horizontal baseline based on the first color value of each pixel on the horizontal baseline, the first pixel color average value, and the width of the target image; obtaining the second coarseness quantification value of the vertical baseline based on the first color value of each pixel on the vertical baseline, the first pixel color average value, and the height of the target image; and obtaining the image coarseness quantification value of the target image based on the first coarseness quantification value and the second coarseness quantification value.

[0054] like Figure 4 As shown, taking the right diagonal region as the target diagonal region, the first image is denoted as P11, and the second image is denoted as P12. First, the first image is taken as the target image, and the first color value of each pixel in the first image is denoted as img. P11 Let W be the width of the first image, H be the height, and P be the average color value of the first pixel of the first image. mean Then P mean Satisfy the following formula:

[0055]

[0056] According to preset selection conditions, a horizontal baseline W0 and a vertical baseline H0 are selected from the first image. W0 is a baseline parallel to a row of the first image, and H0 is a baseline parallel to a column of the first image. The value range of W0 satisfies 0 < W0 < W, and the value range of H0 satisfies 0 < H0 < H. In this embodiment, W0 can be W / 2 and H0 can be H / 2, but other values ​​within the above ranges are also possible. The first color value of each pixel on the horizontal baseline W0 is denoted as img. P11 (i, W0), the first color value of each pixel on the vertical baseline H0 is denoted as img. P11 (H0, j), using H _c The first roughness measure of the horizontal baseline is represented by W. _c Let represent the second roughness measure of the vertical baseline. Then, the two satisfy the following formula:

[0057]

[0058]

[0059] Use C P11 The image coarseness quantification value of the first image satisfies the following formula:

[0060]

[0061] By combining and averaging the differences between the first color value of each pixel on the horizontal baseline and the average first pixel color value of the white light lesion image, and combining and averaging the differences between the first color value of each pixel on the vertical baseline and the average first pixel color value of the white light lesion image, and then combining and averaging the two, the resulting image roughness quantification values ​​reflect the overall roughness difference between each image and the white light lesion image, with high accuracy.

[0062] Similarly, by using the second image as the target image and substituting its relevant data into formulas 3 to 6 above, the image roughness quantification value CP of the second image can be obtained. 12 Based on the above method, the image roughness quantification values ​​of the first and second images for each diagonal region can be calculated.

[0063] When the lesion type is oropharyngeal carcinoma, the lesion surface appears ulcerated in white light images. When the lesion type is oropharyngeal lymphoma, the lesion surface appears eroded in white light images. Since the two lesion types exhibit different surface ulceration / erosion in white light images, a trained ulcer / erosion recognition model can be invoked to identify the target lesion in the white light image, obtaining a first recognition result of whether the surface is ulcerated or eroded. The model represents and outputs the first recognition result for different situations with different values, obtaining the first learned feature of the target lesion. This method measures the type of the target lesion from the ulcer / erosion dimension.

[0064] Let label2 represent the first learned feature of the target lesion. The trained ulcer / erosion recognition model is called to identify the target white light lesion image. When the first recognition result is surface ulcer or surface ulcer, the value of label2 satisfies the following formula:

[0065]

[0066] S3: Call the trained speckle / microvascular dilatation recognition model to recognize the target NBI lesion image, obtain the second recognition result of speckle / microvascular dilatation, and obtain the second learning feature of the target lesion based on the second recognition result.

[0067] The NBI image corresponding to the target lesion is the target NBI lesion image. The speckle / microvascular dilatation recognition model can be a ResNet50 model. Before using this model, a professional endoscopist first identifies the speckle / microvascular dilatation of lesions in a certain number of historical cases' NBI images, labels the corresponding speckle / microvascular dilatation, and uses this training data as training data. Then, the speckle / microvascular dilatation recognition model is trained using this training data.

[0068] like Figure 5 As shown, when the lesion type is oropharyngeal carcinoma, the lesion surface appears as spots in the NBI image; when the lesion type is oropharyngeal lymphoma, the lesion surface appears as microvascular dilation in the NBI image. The two lesion types exhibit different surface spots / microvascular dilation in the NBI image. Based on this, a trained spot / microvascular dilation recognition model can be invoked to identify the target NBI lesion image, obtaining a second recognition result: either the surface is spots or the surface is microvascular dilation. The model represents and outputs the second recognition result for different cases with different values, obtaining the second learned feature of the target lesion. This method measures the type of the target lesion from the spot / microvascular dilation dimension.

[0069] Let label3 represent the second learned feature of the target lesion. The trained speckle / microvascular dilatation recognition model is called to identify the target NBI lesion image. When the second recognition result is surface speckle / microvascular dilatation, the value of label3 satisfies the following formula:

[0070]

[0071] S4: Obtain the symmetry information of the target lesion from the CT images of each lesion, and obtain the second quantitative feature of the target lesion based on the symmetry information; obtain the boundary clarity information of the target lesion from the CT images of the target lesion, and obtain the third quantitative feature of the target lesion based on the boundary clarity information; obtain the shape information of the target lesion from the CT images of the target lesion, and obtain the fourth quantitative feature of the target lesion based on the shape information; obtain the invasiveness information of the target lesion from the CT images of the target lesion, and obtain the fifth quantitative feature of the target lesion based on the invasiveness information.

[0072] like Figure 6 As shown, when the lesion type is oropharyngeal carcinoma, the lesions in the CT images exhibit an asymmetrical distribution, as indicated by the circular box, meaning unilateral onset. Conversely, when the lesion type is oropharyngeal lymphoma, the lesions in the CT images show a symmetrical distribution, meaning bilateral onset. The two lesion types exhibit different levels of symmetry in CT images, and symmetry analysis requires combining at least two CT images. Therefore, based on this, symmetry information of the target lesion can be obtained from all CT lesion images, and the symmetry can be quantified to obtain a second quantitative feature. This method measures the type of the target lesion from the dimension of symmetry in lesion distribution.

[0073] In one embodiment, S4 specifically includes: acquiring lesion quantity information in the oropharynx; when the lesion quantity information indicates that there is a unique lesion in the oropharynx, generating symmetry information that the target lesion is asymmetrically distributed, and obtaining a second quantitative feature of the target lesion based on the symmetry information; when the lesion quantity information indicates that there are at least two lesions in the oropharynx, acquiring the first coordinate information, the fifth size information, and the second color value of each pixel within the second lesion frame of the target lesion from the target CT lesion image, while sequentially treating other lesions as non-target lesions, and acquiring the third lesion frame of the non-target lesion from the non-target CT lesion image. The system uses two coordinate information, a sixth dimension information, and the third color value of each pixel within the third lesion bounding box; based on the first and second coordinate information, it obtains the positional difference between the target lesion and the non-target lesion; based on the fifth dimension information, each second color value, the sixth dimension information, and each third color value, it obtains the color difference between the target lesion and the non-target lesion; it compares the positional difference with the first threshold, and the color difference with the second threshold, and determines whether the target lesion and the non-target lesion are the same lesion with symmetrical distribution based on the comparison results, and generates corresponding symmetry information; based on all the symmetry information, it obtains the second quantitative feature of the target lesion.

[0074] Label4 is used to represent the second quantitative feature of the target lesion. When there is only one lesion in the oropharynx, it is definitely asymmetrically distributed, i.e., unilateral, and the label4 value is 0. When there are two lesions in the oropharynx, they may be a symmetrical distribution of the same lesion, or they may be two independent lesions with asymmetrical distribution. In this case, it is necessary to judge the two situations. When making a judgment, the location information and color information of the two lesions should be used as a reference.

[0075] Specifically, the first coordinate information, the fifth size information, and the second color value of each pixel within the second lesion frame are first obtained from the target CT lesion image. The first coordinate information includes the coordinates (x, y) of the second lesion frame in the CT image. 11 y 11 x 12 y 12 The fifth dimension information includes the width w1 and height h1 of the second lesion frame, and the second color value of each pixel within the second lesion frame is represented as img. 31 (i, j). Simultaneously, other lesions are sequentially treated as non-target lesions, and the second coordinate information, sixth size information, and third color value of each pixel within the third lesion bounding box of the non-target lesion are obtained from the non-target CT lesion image. The second coordinate information includes the coordinates (x, j) of the third lesion bounding box in the CT image. 21 y 21 x 22 y22 The fifth dimension information includes the width w2 and height h2 of the third lesion frame, and the third color value of each pixel within the third lesion frame is represented as img. 32 (m, m). Let △1 represent the positional difference between the target lesion and the non-target lesion, and let △2 represent the color difference between the target lesion and the non-target lesion. Then, it satisfies the following formula:

[0076]

[0077]

[0078] Compare the positional difference Δ1 with the first threshold τ1, and compare the color difference Δ2 with the second threshold τ2. If the comparison result is Δ1 < τ1 and Δ2 < τ2, then the target lesion and non-target lesions are determined to be the same lesion with a symmetrical distribution; otherwise, they are different lesions with an asymmetrical distribution. At this time, the value of the second quantification feature label4 satisfies the following formula:

[0079]

[0080] When the lesion type is oropharyngeal carcinoma, the lesion boundary is unclear in CT images, while when the lesion type is oropharyngeal lymphoma, the lesion boundary is clear in CT images. Since the two lesion types exhibit different boundary clarity in CT images, boundary clarity information of the target lesion can be obtained from the target lesion image, and this boundary clarity can be quantified to obtain a third quantitative feature. This method measures the type of the target lesion from the dimension of lesion boundary clarity.

[0081] In one embodiment, S4 specifically includes: performing boundary line detection on the target lesion in the target CT lesion image based on the Canny algorithm to obtain multiple effective boundary points and obtain the total number of multiple effective boundary points; determining the connected component of the target lesion based on the multiple effective boundary points and obtaining the perimeter of the connected component; comparing the ratio of the total number to the perimeter with a third threshold, judging whether the boundary of the target lesion is clear based on the comparison result, generating corresponding boundary clarity information, and obtaining the third quantitative feature of the target lesion based on the boundary clarity information.

[0082] Let label5 represent the third quantization feature of the target lesion. First, the boundary line of the target lesion in the target CT image is extracted based on the Canny algorithm. Based on the double threshold condition in the Canny algorithm, valid boundary points with gradient values ​​greater than the first threshold are obtained from the extracted boundary lines and marked as 1. Invalid boundary points with gradient values ​​less than the second threshold are obtained and marked as 0. After binarization, the values ​​of all boundary points are summed to obtain l1, which is the number of all valid boundary points. The connected components of the target lesion are determined based on these valid boundary points, and the perimeter l2 of the connected components is calculated. The ratio of l1 to l2 is compared with the third threshold α. If the ratio is less than α, the boundary of the target lesion is considered unclear; if the ratio is not less than α, the boundary of the target lesion is considered clear. The two states are as follows: Figure 7 As shown in the figure. At this time, the value of the third quantization feature label5 satisfies the following formula:

[0083]

[0084] When the lesion type is oropharyngeal carcinoma, the shape of the lesion is irregular in CT images, while when the lesion type is oropharyngeal lymphoma, the shape of the lesion is nearly circular in CT images. Based on this difference in shape between the two lesion types in CT images, the shape information of the target lesion can be obtained from the target lesion image, and the shape can be quantified to obtain a fourth quantitative feature. This method measures the type of the target lesion from the dimension of lesion shape.

[0085] In one embodiment, S4 specifically includes: obtaining the centroid position of the target lesion region in the target CT lesion image, obtaining the maximum distance between all boundary points of the target lesion region and the centroid, generating an circumcircle with the centroid as the center based on the maximum distance, and obtaining the distance between each boundary point and the circumcircle along the radial direction; comparing the standard deviation of all distances with a fourth threshold, determining whether the shape of the target lesion is close to a circle based on the comparison result, generating corresponding shape information, and obtaining the fourth quantitative feature of the target lesion based on the shape information.

[0086] like Figure 8 As shown, based on the connected components of the target CT lesion image, the centroid of the target lesion region is determined. Then, the distances between all boundary points of the target lesion region and the centroid are calculated, and the maximum distance is found. Using the maximum distance as the radius, a circumcircle is drawn with the centroid as the center. The distance d between each boundary point and the circumcircle along the radial direction is calculated. i Then calculate the standard deviation s of the k boundary points, which satisfies the following formula:

[0087]

[0088] The value of s is compared with the fourth threshold β. If the comparison result is that s is not less than β, the target lesion is determined to have an irregular shape; if the comparison result is that s is less than β, the target lesion is determined to have a nearly circular shape. In this case, the value of the fourth quantization feature label6 satisfies the following formula:

[0089]

[0090] When the lesion type is oropharyngeal carcinoma, the lesion appears invasive on CT images, while when the lesion type is oropharyngeal lymphoma, the lesion appears non-invasive on CT images. Since the two lesion types exhibit different degrees of invasiveness on CT images, invasiveness information of the target lesion can be obtained from the target CT lesion image, and the invasiveness can be quantified to obtain a fourth quantitative feature. This method measures the type of the target lesion from the dimension of lesion invasiveness.

[0091] In one embodiment, S4 specifically includes: determining the target lesion expansion region corresponding to the target lesion region in the target CT lesion image according to preset expansion parameters; acquiring the seventh size information of the target lesion region, the fourth color value of each pixel in the target lesion region, the eighth size information of the target lesion expansion region, and the fifth color value of each pixel in the target lesion expansion region; obtaining the second pixel color mean of the target lesion region according to the seventh size information and each fourth color value, obtaining the third pixel color mean of the target lesion expansion region according to the eighth size information and each fifth color value, obtaining the color gradient of the target lesion region and the target lesion expansion region according to the second pixel color mean, the third pixel color mean, and the preset expansion parameters; comparing the color gradient with the fifth threshold, determining whether the target lesion has invaded according to the comparison result, and generating corresponding invasion information; obtaining the fifth quantitative feature of the target lesion according to the invasion information.

[0092] like Figure 9 As shown, based on the preset invasion parameter K, taking the target lesion region Y1 in the target CT lesion image as a reference, the region is expanded outward by K pixels to obtain the corresponding expanded target lesion region Y2. The seventh size information of the target lesion region and the fourth color value of each pixel within the target lesion region are obtained, and the average second pixel color q1 of the target lesion region Y1 is calculated. The eighth size information of the expanded target lesion region and the fifth color value of each pixel within the expanded target lesion region are obtained, and the average third pixel color q2 of the expanded target lesion region Y2 is calculated. The color gradients of Y1 and Y2 are represented by Δcolor, which satisfies the following formula:

[0093]

[0094] The △color is compared with the fifth threshold γ. If the comparison result is that △color is less than γ, the morphology of the target lesion is determined to be invasive; if the comparison result is that △color is not less than γ, the morphology of the target lesion is determined to be non-invasive. In this case, the value of the fifth quantification feature label7 satisfies the following formula:

[0095]

[0096] S5: Based on all quantitative features and all learned features, obtain the oropharyngeal lesion features.

[0097] In the above steps, multiple quantitative features and multiple learned features of the target lesion were obtained. These features together form the oropharyngeal lesion features, which can provide an accurate and comprehensive reference for the subsequent determination of lesion type.

[0098] In one embodiment, after S5, the method further includes: obtaining the weights of each quantitative feature and each learned feature of the target lesion; obtaining the lesion severity coefficient of the target lesion based on each quantitative feature, each learned feature and the corresponding weight; and determining the lesion type information of the target lesion based on the lesion severity coefficient and the coefficient threshold.

[0099] In the above steps, the quantitative features label1 to label7 of the target lesion are obtained. The weights λ1 to λ7 of each quantitative feature are obtained through machine learning models such as decision trees and random forests. Then, the seven weights are summed to obtain the lesion severity coefficient θ of the target lesion, which satisfies the following formula:

[0100]

[0101] Will Compare with the coefficient threshold η, if the comparison result is If the value is not greater than η, the target lesion is determined to be oropharyngeal carcinoma. If the comparison result is... If the value is greater than η, the target lesion is determined to be oropharyngeal lymphoma, as shown in the following formula:

[0102]

[0103] Through the above process, the type of target lesion in the oropharynx can be determined, providing an accurate reference for subsequent diagnosis and treatment.

[0104] Based on the above process, Table 1 illustrates the differences between the prior art and the embodiments of this application in terms of lesions across various dimensions, as well as the feature extraction methods in each dimension. In the prior art, when differentiating between oropharyngeal carcinoma and oropharyngeal lymphoma based on clinical manifestations, the extraction of lesion features is difficult because both present with pharyngeal discomfort, pain, and a foreign body sensation. In this application, after acquiring white light images, NBI images, and CT images of the oropharynx, the lesion status of the target oropharyngeal lesion can be automatically quantified and learned from multiple dimensions, and the oropharyngeal lesion features can be comprehensively obtained. These features can provide an accurate and comprehensive reference for determining the type of the target lesion, with high efficiency and thus high accuracy in determining the type of oropharyngeal lesion.

[0105] Table 1

[0106]

[0107] Based on the methods described in the above embodiments, this embodiment will further describe the method from the perspective of an oropharyngeal lesion feature extraction device. Please refer to [link to relevant documentation]. Figure 10 The oropharyngeal lesion feature extraction device may include:

[0108] The acquisition module 10 is used to acquire white light images, NBI images and CT images of the oropharynx of the same case, and call the trained lesion detection model to detect the white light images, the NBI images and the CT images respectively to obtain white light lesion images, NBI lesion images and CT lesion images of each lesion;

[0109] The first obtaining module 20 is used to obtain roughness information of the target lesion from the target white light lesion image, obtain the first quantitative feature of the target lesion based on the roughness information, call the trained ulcer / erosion recognition model to recognize the target white light lesion image, obtain the first recognition result of ulcer / erosion, and obtain the first learned feature of the target lesion based on the first recognition result.

[0110] The second module 30 is used to call the trained speckle / microvascular dilatation recognition model to recognize the target NBI lesion image, obtain the second recognition result of speckle / microvascular dilatation, and obtain the second learned feature of the target lesion based on the second recognition result;

[0111] The third obtaining module 40 is used to obtain symmetry information of the target lesion from the CT images of each lesion, and obtain a second quantitative feature of the target lesion based on the symmetry information; obtain boundary clarity information of the target lesion from the target CT images, and obtain a third quantitative feature of the target lesion based on the boundary clarity information; obtain shape information of the target lesion from the target CT images, and obtain a fourth quantitative feature of the target lesion based on the shape information; obtain invasiveness information of the target lesion from the target CT images, and obtain a fifth quantitative feature of the target lesion based on the invasiveness information.

[0112] The fourth module 50 is used to obtain oropharyngeal lesion features based on all quantified features and all learned features.

[0113] In one embodiment, the target white light lesion image includes a first lesion frame of the target lesion, and the target white light lesion image is obtained based on the center of the first lesion frame and a preset magnification factor. The first obtaining module 20 includes:

[0114] The first submodule is used to generate a first diagonal line and a second diagonal line intersecting the center in the target white light lesion image, resulting in four diagonal regions. Each diagonal region is divided by the edge of the first lesion frame, forming a first region close to the center and a second region far from the center. The first region forms a first image, and the second region forms a second image.

[0115] The second submodule is used to obtain the first size information of the target white light lesion image and the second size information of the first lesion frame in the target white light lesion image, and to obtain the third size information of each first image and the fourth size information of each second image based on the first size information, the second size information and the four diagonal regions.

[0116] The third submodule is used to obtain the first color information of the target white light lesion image, and for each diagonal region, obtain the image roughness quantification value of the corresponding first image based on the first color information and the third size information of the corresponding first image, and obtain the image roughness quantification value of the corresponding second image based on the first color information and the fourth size information of the corresponding second image.

[0117] The fourth submodule is used to obtain the region roughness quantification value of each diagonal region based on the image roughness quantification value of the corresponding first image and the image roughness quantification value of the corresponding second image, obtain the roughness information of the target lesion based on the four region roughness quantification values, and obtain the first quantitative feature of the target lesion based on the roughness information.

[0118] In one embodiment, the third obtaining submodule includes:

[0119] The determining unit is used to sequentially determine each of the diagonal regions as target diagonal regions, and sequentially determine the first image and the second image corresponding to the target diagonal regions as target images;

[0120] The first obtaining unit is used to obtain the first color value of each pixel in the target white light lesion image, and for each target image, obtain the first pixel color average value of the target image based on each first color value and the width and height of the target image;

[0121] The second obtaining unit is used to select a horizontal baseline and a vertical baseline from the target image according to preset selection conditions, obtain a first coarseness quantification value of the horizontal baseline based on the first color value of each pixel on the horizontal baseline, the average color value of the first pixel, and the width of the target image, and obtain a second coarseness quantification value of the vertical baseline based on the first color value of each pixel on the vertical baseline, the average color value of the first pixel, and the height of the target image.

[0122] The third obtaining unit is used to obtain the image roughness metric of the target image based on the first roughness metric and the second roughness metric.

[0123] In one embodiment, the third obtaining module 40 includes:

[0124] The first acquisition submodule is used to acquire information on the number of lesions in the oropharynx;

[0125] The fifth submodule is used to generate symmetry information that the target lesion is asymmetrically distributed when the lesion quantity information indicates that there is a unique lesion in the oropharynx, and to obtain the second quantitative feature of the target lesion based on the symmetry information.

[0126] The second acquisition submodule is used to acquire, when the lesion quantity information indicates that there are at least two lesions in the oropharynx, the first coordinate information, the fifth size information, and the second color value of each pixel in the second lesion frame of the target lesion from the target CT lesion image, and simultaneously treat other lesions as non-target lesions, and acquire, from the non-target CT lesion image, the second coordinate information, the sixth size information, and the third color value of each pixel in the third lesion frame of the non-target lesion;

[0127] The sixth submodule is used to obtain the positional difference between the target lesion and the non-target lesion based on the first coordinate information and the second coordinate information; and to obtain the color difference between the target lesion and the non-target lesion based on the fifth size information, each second color value, the sixth size information, and each third color value.

[0128] The seventh submodule is used to compare the positional difference degree with the first threshold, as well as the color difference degree and the second threshold. Based on the comparison results, it is determined whether the target lesion and the non-target lesion are the same lesion with symmetrical distribution, and corresponding symmetry information is generated. Based on all the symmetry information, the second quantitative feature of the target lesion is obtained.

[0129] In one embodiment, the third obtaining module 40 includes:

[0130] The third acquisition submodule is used to perform boundary line detection on the target lesion in the target CT lesion image based on the Canny algorithm, obtain multiple valid boundary points, and obtain the total number of the multiple valid boundary points;

[0131] The fourth acquisition submodule is used to determine the connected domain of the target lesion based on the multiple valid boundary points, and to acquire the perimeter of the connected domain;

[0132] The eighth submodule is used to compare the ratio of the total number to the perimeter with the third threshold, determine whether the boundary of the target lesion is clear based on the comparison result, generate corresponding boundary clarity information, and obtain the third quantitative feature of the target lesion based on the boundary clarity information.

[0133] In one embodiment, the third obtaining module 40 includes:

[0134] The fifth acquisition submodule is used to acquire the centroid position of the target lesion region in the target CT lesion image, acquire the maximum distance between all boundary points of the target lesion region and the centroid, generate an circumcircle with the centroid as the center based on the maximum distance, and acquire the distance between each boundary point and the circumcircle along the radial direction.

[0135] The ninth submodule is used to compare the standard deviation of all distances with the fourth threshold, determine whether the shape of the target lesion is close to a circle based on the comparison result, generate corresponding shape information, and obtain the fourth quantitative feature of the target lesion based on the shape information.

[0136] In one embodiment, the third obtaining module 40 includes:

[0137] The determination submodule is used to determine the target lesion expansion area corresponding to the target lesion area in the target lesion image based on preset expansion parameters;

[0138] The sixth acquisition submodule is used to acquire the seventh size information of the target lesion area, the fourth color value of each pixel in the target lesion area, the eighth size information of the outer expansion area of ​​the target lesion, and the fifth color value of each pixel in the outer expansion area of ​​the target lesion.

[0139] The tenth submodule is used to obtain the average second pixel color of the target lesion area based on the seventh size information and each fourth color value, obtain the average third pixel color of the target lesion expansion area based on the eighth size information and each fifth color value, and obtain the color gradient of the target lesion area and the target lesion expansion area based on the average second pixel color, the average third pixel color and the preset expansion parameter.

[0140] The eleventh submodule is used to compare the color gradient with the fifth threshold, determine whether the target lesion has invaded based on the comparison result, generate corresponding invasion information, and obtain the fifth quantitative feature of the target lesion based on the invasion information.

[0141] In one embodiment, the oropharyngeal lesion feature extraction device further includes:

[0142] The fifth module is used to obtain the weights of each quantitative feature and each learned feature of the target lesion, and to obtain the lesion severity coefficient of the target lesion based on each quantitative feature, each learned feature and the corresponding weight.

[0143] The sixth module is used to determine the lesion type information of the target lesion based on the lesion severity coefficient and the coefficient threshold.

[0144] Unlike existing technologies, the oropharyngeal lesion feature extraction device provided in this application first acquires white light images, NBI images, and CT images of the oropharynx of the same case, and then calls the trained lesion detection model to detect the white light images, NBI images, and CT images respectively to obtain white light lesion images, NBI lesion images, and CT lesion images of each lesion. Then, it obtains the roughness quantitative features and ulcer / erosion learning features of the target lesion from the white light lesion images, the spot / microvascular dilation learning features of the target lesion from the NBI lesion images, and the symmetry quantitative features, boundary clarity quantitative features, shape quantitative features, and invasiveness quantitative features of the target lesion from the CT lesion images. Finally, based on all quantitative features and all learning features, the oropharyngeal lesion features are obtained. After acquiring white light images, NBI images, and CT images of the oropharynx, this application can automatically quantify and learn the lesion status of the target lesion in the oropharynx from multiple dimensions, and synthesize the features of the oropharyngeal lesion. These features can provide an accurate and comprehensive reference for determining the type of the target lesion, with high efficiency and high accuracy in determining the type of the oropharyngeal lesion.

[0145] Accordingly, embodiments of this application also provide an electronic device, such as... Figure 11 As shown, the electronic device may include a radio frequency (RF) circuit 1001, a memory 1002 including one or more computer-readable storage media, an input unit 1003, a display unit 1004, a sensor 1005, an audio circuit 1006, a WiFi module 1007, a processor 1008 including one or more processing cores, and a power supply 1009, among other components. Those skilled in the art will understand that... Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0146] The radio frequency circuit 1001 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and hands it over to one or more processors 1008 for processing; additionally, it transmits uplink data to the base station. The memory 1002 can be used to store software programs and modules. The processor 1008 executes various functional applications and oropharyngeal lesion feature extraction by running the software programs and modules stored in the memory 1002. The input unit 1003 can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to customer settings and function control.

[0147] The display unit 1004 can be used to display information input by the customer or information provided to the customer, as well as various graphical client interfaces of the server. These graphical client interfaces can be composed of graphics, text, icons, videos, and any combination thereof.

[0148] The electronic device may also include at least one sensor 1005, such as a light sensor, a motion sensor, and other sensors. The audio circuitry 1006 includes a speaker that provides an audio interface between the client and the electronic device.

[0149] WiFi is a short-range wireless transmission technology. Electronic devices using the WiFi module 1007 can help customers send and receive emails, browse web pages, and access streaming media, providing customers with wireless broadband internet access. Although Figure 11 WiFi module 1007 is shown, but it is understood that it is not a necessary component of electronic devices and can be omitted as needed without changing the nature of the application.

[0150] The processor 1008 is the control center of the electronic device. It connects various parts of the phone through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 1002, and calling data stored in the memory 1002, it performs various functions of the electronic device and processes data, thereby monitoring the phone as a whole.

[0151] The electronic device also includes a power supply 1009 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 1008 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system.

[0152] Although not shown, electronic devices may also include cameras, Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1008 in the server loads the executable files corresponding to the processes of one or more applications into the memory 1002 according to the following instructions, and the processor 1008 runs the applications stored in the memory 1002 to achieve the following functions:

[0153] Acquire white light images, NBI images, and CT images of the oropharynx of the same case, and call the trained lesion detection model to detect the white light images, NBI images, and CT images respectively to obtain white light lesion images, NBI lesion images, and CT lesion images of each lesion;

[0154] The roughness information of the target lesion is obtained from the target white light lesion image. Based on the roughness information, the first quantitative feature of the target lesion is obtained. The trained ulcer / erosion recognition model is called to recognize the target white light lesion image to obtain the first recognition result of the ulcer / erosion. Based on the first recognition result, the first learned feature of the target lesion is obtained.

[0155] The trained speckle / microvascular dilatation recognition model is invoked to identify the target NBI lesion image, and a second recognition result of speckle / microvascular dilatation is obtained. Based on the second recognition result, a second learned feature of the target lesion is obtained.

[0156] Symmetry information of the target lesion is obtained from the CT images of each lesion, and a second quantitative feature of the target lesion is obtained based on the symmetry information; boundary clarity information of the target lesion is obtained from the target CT images, and a third quantitative feature of the target lesion is obtained based on the boundary clarity information; shape information of the target lesion is obtained from the target CT images, and a fourth quantitative feature of the target lesion is obtained based on the shape information; invasiveness information of the target lesion is obtained from the target CT images, and a fifth quantitative feature of the target lesion is obtained based on the invasiveness information.

[0157] Based on all quantitative features and all learned features, the features of oropharyngeal lesions are obtained.

[0158] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description above, and they will not be repeated here.

[0159] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0160] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to perform the following functions:

[0161] Acquire white light images, NBI images, and CT images of the oropharynx of the same case, and call the trained lesion detection model to detect the white light images, NBI images, and CT images respectively to obtain white light lesion images, NBI lesion images, and CT lesion images of each lesion;

[0162] The roughness information of the target lesion is obtained from the target white light lesion image. Based on the roughness information, the first quantitative feature of the target lesion is obtained. The trained ulcer / erosion recognition model is called to recognize the target white light lesion image to obtain the first recognition result of the ulcer / erosion. Based on the first recognition result, the first learned feature of the target lesion is obtained.

[0163] The trained speckle / microvascular dilatation recognition model is invoked to identify the target NBI lesion image, and a second recognition result of speckle / microvascular dilatation is obtained. Based on the second recognition result, a second learned feature of the target lesion is obtained.

[0164] Symmetry information of the target lesion is obtained from the CT images of each lesion, and a second quantitative feature of the target lesion is obtained based on the symmetry information; boundary clarity information of the target lesion is obtained from the target CT images, and a third quantitative feature of the target lesion is obtained based on the boundary clarity information; shape information of the target lesion is obtained from the target CT images, and a fourth quantitative feature of the target lesion is obtained based on the shape information; invasiveness information of the target lesion is obtained from the target CT images, and a fifth quantitative feature of the target lesion is obtained based on the invasiveness information.

[0165] Based on all quantitative features and all learned features, the features of oropharyngeal lesions are obtained.

[0166] The foregoing has provided a detailed description of the oropharyngeal lesion feature extraction method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the technical solutions and core ideas of this application. Those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An oropharyngeal lesion feature extraction method, characterized by, The method comprises the following steps: Obtaining white light images, NBI images and CT images of the oropharynx of the same case, calling a trained lesion detection model to detect the white light images, the NBI images and the CT images respectively to obtain white light lesion images, NBI lesion images and CT lesion images of each lesion; Obtaining roughness information of the target lesion from the target white light lesion image, and obtaining a first quantitative feature of the target lesion according to the roughness information; Calling a trained ulcer / corrosion recognition model to recognize the target white light lesion image to obtain a first recognition result of ulcer / corrosion, and obtaining a first learning feature of the target lesion according to the first recognition result; Calling a trained spot / microvessel dilation recognition model to recognize the target NBI lesion image to obtain a second recognition result of spot / microvessel dilation, and obtaining a second learning feature of the target lesion according to the second recognition result; Obtaining symmetry information of the target lesion from the CT lesion images of each lesion, and obtaining a second quantitative feature of the target lesion according to the symmetry information; Obtaining boundary sharpness information of the target lesion from the target CT lesion image, and obtaining a third quantitative feature of the target lesion according to the boundary sharpness information; Obtaining shape information of the target lesion from the target CT lesion image, and obtaining a fourth quantitative feature of the target lesion according to the shape information; Obtaining invasion degree information of the target lesion from the target CT lesion image, and obtaining a fifth quantitative feature of the target lesion according to the invasion degree information; Obtaining oropharyngeal lesion features according to all quantitative features and all learning features; The target white light lesion image includes a first lesion frame of the target lesion, and the target white light lesion image is obtained based on the center of the first lesion frame and a preset magnification factor. The step of obtaining the first quantitative feature of the target lesion according to the roughness information of the target lesion from the target white light lesion image comprises the following steps: Generating a first diagonal line and a second diagonal line intersecting at the center in the target white light lesion image to obtain four diagonal regions, each diagonal region being divided by the edge of the first lesion frame to form a first region close to the center and a second region away from the center, the first region forming a first image and the second region forming a second image; Obtaining first size information of the target white light lesion image and second size information of the first lesion frame in the target white light lesion image, and obtaining third size information of each first image and fourth size information of each second image according to the first size information, the second size information and the four diagonal regions; Obtaining first color information of the target white light lesion image, for each diagonal region, obtaining an image roughness quantitative value of the corresponding first image according to the first color information and the third size information of the corresponding first image, and obtaining an image roughness quantitative value of the corresponding second image according to the first color information and the fourth size information of the corresponding second image; For each of the diagonal regions, a region roughness quantization value of each of the diagonal regions is obtained according to the image roughness quantization value of the corresponding first image and the image roughness quantization value of the corresponding second image, and a roughness information of the target lesion is obtained according to the four region roughness quantization values, and a first quantization feature of the target lesion is obtained according to the roughness information.

2. The oropharyngeal lesion feature extraction method according to claim 1, characterized in that, The step of obtaining the first color information of the target white light lesion image, and obtaining, for each diagonal region, the image roughness quantization value of the corresponding first image according to the first color information and third size information of the corresponding first image, and obtaining the image roughness quantization value of the corresponding second image according to the first color information and fourth size information of the corresponding second image, comprises: sequentially determining each of the diagonal regions as a target diagonal region, and sequentially determining the first image and the second image corresponding to the target diagonal region as target images; obtaining first color values of each pixel point in the target white light lesion image, and obtaining, for each of the target images, a first pixel color mean value of the target image according to the first color values and a width and a height of the target image; selecting a horizontal reference line and a vertical reference line from the target image according to a preset selection condition, obtaining a first roughness quantization value of the horizontal reference line according to the first color values of each pixel point on the horizontal reference line, the first pixel color mean value and the width of the target image, and obtaining a second roughness quantization value of the vertical reference line according to the first color values of each pixel point on the vertical reference line, the first pixel color mean value and the height of the target image; obtaining an image roughness quantization value of the target image according to the first roughness quantization value and the second roughness quantization value.

3. The method according to claim 1, wherein, The step of obtaining the symmetry information of the target lesion from the CT lesion images of each lesion, and obtaining a second quantization feature of the target lesion according to the symmetry information, comprises: obtaining lesion number information of the oropharyngeal part; when the lesion number information indicates that there is only one lesion in the oropharyngeal part, generating symmetry information that the target lesion is asymmetrically distributed, and obtaining a second quantization feature of the target lesion according to the symmetry information; when the lesion number information indicates that there are at least two lesions in the oropharyngeal part, obtaining first coordinate information, fifth size information of a second lesion frame of the target lesion and second color values of each pixel point in the second lesion frame from the target CT lesion image, and sequentially taking other lesions as non-target lesions and obtaining second coordinate information, sixth size information of a third lesion frame of the non-target lesion and third color values of each pixel point in the third lesion frame from the non-target CT lesion image; obtaining a position difference degree of the target lesion and the non-target lesion according to the first coordinate information and the second coordinate information, and obtaining a color difference degree of the target lesion and the non-target lesion according to the fifth size information, the second color values, the sixth size information and the third color values. Comparing the position difference degree with a first threshold value and the color difference degree with a second threshold value, judging whether the target lesion and the non-target lesion are the same lesion with symmetric distribution according to a comparison result, and generating corresponding symmetry information, and obtaining a second quantitative feature of the target lesion according to all the symmetry information.

4. The method according to claim 1, wherein, The step of obtaining boundary definition information of the target lesion from the target CT lesion image and obtaining a third quantitative feature of the target lesion according to the boundary definition information comprises: Performing boundary line detection on the target lesion in the target CT lesion image based on a Canny algorithm to obtain a plurality of effective boundary points and obtain a total number of the plurality of effective boundary points; Determining a connected domain of the target lesion according to the plurality of effective boundary points and obtaining a perimeter of the connected domain; Comparing a ratio of the total number to the perimeter with a third threshold value, judging whether the boundary of the target lesion is clear according to a comparison result, and generating corresponding boundary definition information, and obtaining the third quantitative feature of the target lesion according to the boundary definition information.

5. The method according to claim 1, wherein, The step of obtaining shape information of the target lesion from the target CT lesion image and obtaining a fourth quantitative feature of the target lesion according to the shape information comprises: Obtaining a center position of a target lesion region in the target CT lesion image and obtaining a maximum distance between all boundary points of the target lesion region and the center, generating a circumscribed circle with the center as the center based on the maximum distance, and obtaining a distance of each boundary point from the circumscribed circle along the radial direction; Comparing a standard deviation of all distances with a fourth threshold value, judging whether the shape of the target lesion is nearly circular according to a comparison result, and generating corresponding shape information, and obtaining the fourth quantitative feature of the target lesion according to the shape information.

6. The oropharyngeal lesion feature extraction method according to claim 1, characterized by, The step of obtaining invasiveness information of the target lesion from the target CT lesion image and obtaining a fifth quantitative feature of the target lesion according to the invasiveness information comprises: Determining a target lesion expansion region corresponding to the target lesion region in the target CT lesion image according to a preset expansion parameter; Obtaining seventh size information of the target lesion region, fourth color values of each pixel point in the target lesion region, eighth size information of the target lesion expansion region, and fifth color values of each pixel point in the target lesion expansion region; Obtaining a second pixel color mean value of the target lesion region according to the seventh size information and each fourth color value, obtaining a third pixel color mean value of the target lesion expansion region according to the eighth size information and each fifth color value, and obtaining a color gradient of the target lesion region and the target lesion expansion region according to the second pixel color mean value, the third pixel color mean value and the preset expansion parameter; Comparing the color gradient with a fifth threshold value, judging whether the target lesion has occurred invasion according to a comparison result, and generating corresponding invasiveness information, and obtaining the fifth quantitative feature of the target lesion according to the invasiveness information.

7. The method according to claim 1, wherein, After the step of obtaining oropharyngeal lesion features according to all quantitative features and all learning features, further comprising: obtain a weight of each quantitative feature and each learning feature of the target lesion, and obtain a lesion degree coefficient of the target lesion according to each quantitative feature, each learning feature, and the corresponding weight; determine lesion type information of the target lesion according to the lesion degree coefficient and a coefficient threshold.

8. An oropharyngeal lesion feature extraction apparatus characterized by comprising: Comprise: The acquisition module is used for acquiring a white light image, an NBI image, and a CT image of the oropharynx of the same case, calling a trained lesion detection model to detect the white light image, the NBI image, and the CT image respectively, and obtaining a white light lesion image, an NBI lesion image, and a CT lesion image of each lesion; The first obtaining module is used for obtaining roughness information of a target lesion from a target white light lesion image, and obtaining a first quantitative feature of the target lesion according to the roughness information; The trained ulcer / corrosion recognition model is called to recognize the target white light lesion image, a first recognition result of ulcer / corrosion is obtained, and a first learning feature of the target lesion is obtained according to the first recognition result; The trained spot / microvessel dilation recognition model is called to recognize the target NBI lesion image, a second recognition result of spot / microvessel dilation is obtained, and a second learning feature of the target lesion is obtained according to the second recognition result; The third obtaining module is used for obtaining symmetry information of the target lesion from the CT lesion image of each lesion, and obtaining a second quantitative feature of the target lesion according to the symmetry information; The boundary definition information of the target lesion is obtained from the target CT lesion image, and a third quantitative feature of the target lesion is obtained according to the boundary definition information; The shape information of the target lesion is obtained from the target CT lesion image, and a fourth quantitative feature of the target lesion is obtained according to the shape information; The invasion degree information of the target lesion is obtained from the target CT lesion image, and a fifth quantitative feature of the target lesion is obtained according to the invasion degree information; The fourth obtaining module is used for obtaining oropharyngeal lesion features according to all quantitative features and all learning features; The target white light lesion image comprises a first lesion frame of the target lesion, the target white light lesion image is obtained based on a center of the first lesion frame and a preset magnification coefficient, and the first obtaining module comprises: The first obtaining submodule is used for generating a first diagonal line and a second diagonal line intersecting at the center in the target white light lesion image, obtaining four diagonal regions, each diagonal region taking an edge of the first lesion frame as a boundary, forming a first region close to the center and a second region away from the center, the first region forming a first image, and the second region forming a second image; The second obtaining submodule is used for obtaining first size information of the target white light lesion image and second size information of the first lesion frame in the target white light lesion image, and obtaining third size information of each first image and fourth size information of each second image according to the first size information, the second size information, and the four diagonal regions. The third obtaining sub-module is configured to obtain first color information of the target white light lesion image, obtain, for each diagonal region, an image roughness quantization value of the corresponding first image according to the first color information and third size information of the corresponding first image, and obtain an image roughness quantization value of the corresponding second image according to the first color information and fourth size information of the corresponding second image. The fourth obtaining sub-module is configured to obtain, for each diagonal region, a region roughness quantization value of each diagonal region according to the image roughness quantization value of the corresponding first image and the image roughness quantization value of the corresponding second image, obtain roughness information of the target lesion according to four region roughness quantization values, and obtain a first quantization feature of the target lesion according to the roughness information.

9. An electronic device, comprising: The oropharyngeal lesion feature extraction method comprises a memory and a processor; the memory stores an application program; and the processor is configured to run the application program in the memory to execute the steps in the oropharyngeal lesion feature extraction method according to any one of claims 1 to 7.

Images