Method and system for identifying laryngeal space-occupying lesions based on image recognition

Abstract

The present invention relates to the field of medical image recognition technology, and specifically to a method and system for identifying laryngeal space-occupying lesions based on image recognition, including moving an ultrasonic probe in a patient's larynx along a preset path and collecting a two-dimensional ultrasonic image, mapping pixels of the two-dimensional ultrasonic image corresponding to different scanning points to a spatial three-dimensional coordinate system through scanning positions and scanning directions to obtain a three-dimensional voxel model of the patient's larynx; extracting edges from the three-dimensional voxel model and inputting them into a detection network to obtain a cyst center coordinate position, a cyst shape, a mass center coordinate position, and a mass shape; connecting the cyst center coordinate position and the mass center coordinate position to obtain a path, and obtaining a mass correction center coordinate position through the path and the mass center coordinate position artifact. The present invention reconstructs a three-dimensional image from a two-dimensional image through ultrasound, and when a mass artifact is found in the larynx, adjusts the ultrasound frequency to form a new artifact, and obtains the actual coordinate parameters of the mass through multiple measurements.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image recognition, and in particular to a method and system for recognizing laryngeal space-occupying lesions based on image recognition. Background Art

[0002] Laryngeal space-occupying lesions include benign lesions such as vocal cord polyps and masses, laryngeal contact granulomas, precancerous lesions such as vocal cord leukoplakia and laryngeal keratosis, and malignant lesions such as laryngeal cancer. Ultrasonic imaging technology plays an important role in the field of medical diagnosis. Ultrasonic imaging technology draws images of the detected object based on the relevant parameters of the emission and reception of high-frequency sound waves. The specific principle is that when sound waves pass through the human body, the difference in acoustic impedance of different tissues will cause the reflection of sound waves. These reflected waves are received by the probe and converted into images, and doctors can diagnose diseases based on the images. For example, the patent with publication number CN115471484A discloses an image processing method and system for intravascular ultrasound, which processes the intravascular ultrasound image frame by frame, calculates the translation component and rotation component in the imaging process according to the translation and rotation amount between two adjacent frames of images caused by the heartbeat, so as to correct the acquired intravascular ultrasound image, but the current ultrasonic imaging technology has refraction artifacts, which are a common artifact caused by the refraction of sound waves when passing through the interface of different media. Such artifacts can create false structures in the image or cause the true anatomical structures to shift in position, which may interfere with the accuracy of the diagnosis. Summary of the invention

[0003] In order to overcome the defects of the above-mentioned prior art, the purpose of the present invention is to provide a method and system for identifying laryngeal space-occupying lesions based on image recognition, reconstruct a three-dimensional image from a two-dimensional image through ultrasound, and when a mass artifact is found in the larynx, adjust the ultrasound frequency to form a new artifact, and obtain the actual coordinate parameters of the mass through multiple measurements to solve the problem of refraction artifacts caused by laryngeal space-occupying lesions in ultrasonic image reconstruction.

[0004] To achieve the above object, the technical solution of the present invention is:

[0005] The method for identifying laryngeal space-occupying lesions based on image recognition comprises the following steps:

[0006] (1) moving the ultrasonic probe along a preset path in the patient's throat and acquiring a two-dimensional ultrasonic image, wherein the preset path is not limited to linear, rotational or free movement; and recording the scanning position and scanning direction of the ultrasonic probe when acquiring the two-dimensional ultrasonic image at different scanning points;

[0007] (2) After filtering and enhancing the two-dimensional ultrasonic image of the patient's larynx, the pixels of the two-dimensional ultrasonic image corresponding to different scanning points are mapped to a three-dimensional spatial coordinate system by scanning position and scanning direction to obtain a three-dimensional voxel model of the patient's larynx; the blank spaces of the voxels in the three-dimensional voxel model are interpolated, the color and transparency of each voxel are calculated by the three-dimensional voxel model, the penetration path of the light at the observation point passing through the three-dimensional voxel model is obtained, and the cumulative transparency and cumulative color of each voxel in the three-dimensional image of the patient's larynx are obtained by the penetration path;

[0008] (3) obtaining a three-dimensional gradient of a three-dimensional image of the patient's larynx, detecting the edge of the three-dimensional image of the patient's larynx through the three-dimensional gradient, inputting the edge into a cyst detection network to obtain a cyst morphology, and inputting the edge into a mass detection network to obtain a mass morphology, wherein the cyst morphology includes a cyst center coordinate position and a cyst boundary, and the mass morphology includes a mass center coordinate position and a mass boundary; when the cyst morphology and the mass morphology exist at the same time, connecting the cyst center coordinate position and the mass center coordinate position to obtain a first path, again changing the ultrasonic frequency of the ultrasonic probe to move in the patient's larynx along a preset path and constructing a three-dimensional image of the patient's larynx and identifying the cyst morphology and the mass morphology, and again connecting the cyst center coordinate position and the mass center coordinate position to obtain a second path, obtaining a mass correction center coordinate position through the first path and the second path, and obtaining a laryngeal space-occupying lesion parameter through the mass correction center coordinate position and the mass boundary; the cyst detection network is a cyst edge feature classifier established by a neural network algorithm, and the mass detection network is a mass edge feature classifier established by a neural network algorithm.

[0009] Furthermore, the method for filtering and enhancing the two-dimensional ultrasonic image of the patient's larynx includes:

[0010] The Gaussian kernel of size n×n and the standard deviation σ are preset, and the kernel value G(i,j) at the kernel position (i,j) of the Gaussian kernel is:

[0011]

[0012] The Gaussian kernel is normalized so that the sum of the kernel values ​​of the Gaussian kernel is equal to 1, and the normalized Gaussian kernel is convolved with each pixel of the two-dimensional ultrasonic image; the convolution operation includes aligning the Gaussian kernel with the pixels of the two-dimensional ultrasonic image in sequence, and denoising and enhancing the pixel value I(u,v) of the coordinate (u,v) to obtain the corresponding pixel value I'(u,v):

[0013]

[0014] Where k is the radius of the Gaussian kernel, and I(u+i, v+j) is the pixel value of the offset position (u+i, v+j) relative to the current coordinate (u, v); the convolution operation is repeated until all pixel values ​​I(u, v) of the two-dimensional ultrasonic image are denoised and enhanced to obtain the pixel value I′(u, v).

[0015] Furthermore, the method of mapping pixels of a two-dimensional ultrasonic image corresponding to different scanning points into a three-dimensional spatial coordinate system by scanning position and scanning direction to obtain a three-dimensional voxel model of the patient's larynx includes:

[0016] The translation vector is obtained by scanning the change of position The rotation vector is obtained by changing the scanning direction The coordinates (u, v) of the pixels of the two-dimensional ultrasonic image are mapped to the coordinates (u′, v′, w′) of the voxels in the three-dimensional spatial coordinate system, where the coordinates (u′, v′, w′) are calculated as follows:

[0017]

[0018] Where T is the transformation matrix, the transformation matrix T is:

[0019]

[0020] Furthermore, the method for interpolating the blank voxel spaces in the three-dimensional voxel model includes:

[0021] Get the coordinate point P(u′, v′, w′) of the blank voxel in the three-dimensional coordinate system of space, get the voxel values ​​adjacent to the coordinate point P(u′, v′, w′) in three mutually perpendicular directions in the three-dimensional coordinate system of space, and interpolate the voxel value V of the coordinate point P(u′, v′, w′) P for:

[0022] V P (u′, v′, w′) = V 000 (1-u′)(1-v′)(1-w′)+V 001 (1-u′)(1-v′)w′+V 010 (1-u′)v′(1-w′)+V 011 (1-u′)v′w′+V 100 u′(1-v′)(1-w′)+V 101 u′(1-v′)w′+V 110 u′v′(1-w′)+V 111 u′v′w′;

[0023] Among them, V 000 is the minimum value of the voxel value in the x, y and z axis directions, V 001is the maximum voxel value in the z-axis direction and the minimum voxel value in the x- and y-axis directions, V 010 is the maximum voxel value in the y-axis direction and the minimum voxel value in the x- and z-axis directions, V 011 is the maximum value of the voxel value in the z and y axis directions and the minimum value of the voxel value in the x axis direction, V 100 is the maximum value of the voxel in the x-axis direction and the minimum value of the voxel in the y- and z-axis directions, V 101 is the maximum value of the voxel value in the x- and z-axis directions and the minimum value of the voxel value in the y-axis direction, V 110 is the maximum value of the voxel value in the x- and y-axis directions and the minimum value of the voxel value in the z-axis direction, V 111 It is the maximum value of the voxel value in the x, y and z axis directions.

[0024] Furthermore, the method for obtaining the corrected center coordinate position of the mass through the first path and the second path includes:

[0025] Obtain the incident point positions L1(u′, v′, w′) and L2(u′, v′, w′) of the ultrasound entering the cyst corresponding to the first path and the second path, respectively, and record the direction vectors corresponding to the ultrasound along the first path and the second path as and The center coordinates of the mass corresponding to the first path and the second path are recorded as P1(u′, v′, w′) and P2(u′, v′, w′), the corrected center coordinates of the mass are recorded as P(u′, v′, w′), and the refraction points of the ultrasound penetrating the cyst corresponding to the first path and the second path are recorded as R1(u′, v′, w′) and R2(u′, v′, w′); the ultrasonic refraction equations are established as follows:

[0026]

[0027] Wherein, d1 is the distance from R1(u′, v′, w′) to P1(u′, v′, w′), d2 is the distance from R2(u′, v′, w′) to P2(u′, v′, w′); the corrected center coordinate position of the mass is obtained by using the ultrasonic refraction equations and recorded as P(u′, v′, w′).

[0028] The present invention also provides a laryngeal space-occupying lesion recognition system based on image recognition, the system comprising:

[0029] An image acquisition module is used to move the ultrasonic probe in the patient's throat along a preset path and acquire a two-dimensional ultrasonic image. The preset path includes but is not limited to linear, rotational or free movement; and record the scanning position and scanning direction of the ultrasonic probe when acquiring the two-dimensional ultrasonic image at different scanning points;

[0030] A three-dimensional reconstruction module is used to filter and enhance the two-dimensional ultrasonic image of the patient's larynx, and then map the pixels of the two-dimensional ultrasonic image corresponding to different scanning points to a spatial three-dimensional coordinate system through scanning positions and scanning directions to obtain a three-dimensional voxel model of the patient's larynx; interpolate the blank spaces of the voxels in the three-dimensional voxel model, calculate the color and transparency of each voxel through the three-dimensional voxel model, obtain the penetration path of the light at the observation point through the three-dimensional voxel model, and obtain the cumulative transparency and cumulative color of each voxel in the three-dimensional image of the patient's larynx through the penetration path;

[0031] The laryngeal space-occupying lesion recognition module is used to obtain the three-dimensional gradient of the three-dimensional image of the patient's larynx, detect the edge of the three-dimensional image of the patient's larynx through the three-dimensional gradient, input the edge into the cyst detection network to obtain the cyst morphology, and input the edge into the mass detection network to obtain the mass morphology. The cyst morphology includes the cyst center coordinate position and the cyst boundary, and the mass morphology includes the mass center coordinate position and the mass boundary. When the cyst morphology and the mass morphology exist at the same time, the cyst center coordinate position and the mass center coordinate position are connected to obtain a first path, and the ultrasonic probe is changed again to move in the patient's larynx along a preset path to construct the patient's larynx three-dimensional image and identify the cyst morphology and the mass morphology. The cyst center coordinate position and the mass center coordinate position are connected again to obtain a second path, and the mass correction center coordinate position is obtained through the first path and the second path, and the laryngeal space-occupying lesion parameters are obtained through the mass correction center coordinate position and the mass boundary. The cyst detection network is a cyst edge feature classifier established by a neural network algorithm, and the mass detection network is a mass edge feature classifier established by a neural network algorithm.

[0032] Furthermore, the system further comprises:

[0033] The image filtering module is used to preset the Gaussian kernel of size n×n and the standard deviation σ. The kernel value G(i,j) at the kernel position (i,j) of the Gaussian kernel is:

[0034]

[0035] The Gaussian kernel is normalized so that the sum of the kernel values ​​of the Gaussian kernel is equal to 1, and the normalized Gaussian kernel is convolved with each pixel of the two-dimensional ultrasonic image; the convolution operation includes aligning the Gaussian kernel with the pixels of the two-dimensional ultrasonic image in sequence, and denoising and enhancing the pixel value I(u,v) of the coordinate (u,v) to obtain the corresponding pixel value I'(u,v):

[0036]

[0037] Where k is the radius of the Gaussian kernel, and I(u+i,v+j) is the pixel value at the offset position (u+i,v+j) relative to the current coordinate (u,v); the convolution operation is repeated until all pixel values ​​I(u,v) of the two-dimensional ultrasonic image are denoised and enhanced to obtain the pixel value I′(u,v).

[0038] Furthermore, the system further comprises:

[0039] Coordinate transformation module, used to obtain the translation vector by scanning position changes The rotation vector is obtained by changing the scanning direction The coordinates (u, v) of the pixels of the two-dimensional ultrasound image are mapped to the coordinates (u', v', w') of the voxels in the three-dimensional spatial coordinate system, where the coordinates (u', v', w') are calculated as follows:

[0040]

[0041] Among them, T is the transformation matrix, and the transformation matrix T is:

[0042]

[0043] Furthermore, the system further comprises:

[0044] The interpolation module is used to obtain the coordinate point P(u′, v′, w′) of the blank voxel in the three-dimensional coordinate system of the space, obtain the voxel values ​​adjacent to the coordinate point P(u′, v′, w′) in three mutually perpendicular directions in the three-dimensional coordinate system of the space, and interpolate the voxel value V of the coordinate point P(u′, v′, w′) P for:

[0045] V P (u′, v′, w′) = V 000 (1-u′)(1-v′)(1-w′)+V 001 (1-u′)(1-v′)w′+V 010 (1-u′)v′(1-w′)+V 011 (1-u′)v′w′+V 100 u′(1-v′)(1-w′)+V 101 u′(1-v′)w′+V 110 u′v′(1-w′)+V 111 u′v′w′;

[0046] Among them, V 000 is the minimum value of the voxel value in the x, y and z axis directions, V 001 is the maximum voxel value in the z-axis direction and the minimum voxel value in the x- and y-axis directions, V 010is the maximum voxel value in the y-axis direction and the minimum voxel value in the x- and z-axis directions, V 011 is the maximum value of the voxel value in the z and y axis directions and the minimum value of the voxel value in the x axis direction, V 100 is the maximum value of the voxel in the x-axis direction and the minimum value of the voxel in the y- and z-axis directions, V 101 is the maximum value of the voxel value in the x- and z-axis directions and the minimum value of the voxel value in the y-axis direction, V 110 is the maximum value of the voxel value in the x- and y-axis directions and the minimum value of the voxel value in the z-axis direction, V 111 It is the maximum value of the voxel value in the x, y and z axis directions.

[0047] Furthermore, the system further comprises:

[0048] The artifact correction module is used to obtain the incident point positions L1(u′, v′, w′) and L2(u′, v′, w′) of the ultrasound entering the cyst corresponding to the first path and the second path, and record the direction vectors corresponding to the ultrasound along the first path and the second path as and The center coordinates of the mass corresponding to the first path and the second path are recorded as P1(u′, v′, w′) and P2(u′, v′, w′), the corrected center coordinates of the mass are recorded as P(u′, v′, w′), and the refraction points of the ultrasound penetrating the cyst corresponding to the first path and the second path are recorded as R1(u′, v′, w′) and R2(u′, v′, w′); the ultrasonic refraction equations are established as follows:

[0049]

[0050] Wherein, d1 is the distance from R1(u′, v′, w′) to P1(u′, v′, w′), d2 is the distance from R2(u′, v′, w′) to P2(u′, v′, w′); the corrected center coordinate position of the mass is obtained by using the ultrasonic refraction equations and recorded as P(u′, v′, w′).

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] (1) The two-dimensional ultrasonic image of the present invention is first filtered and enhanced and then interpolated, so that the data is smoother and more accurate.

[0053] (2) The present invention performs convolution operation on each pixel of the two-dimensional ultrasonic image, thereby solving the problem of image aliasing. The image quality after processing is better than the image reconstructed from the original signal.

[0054] (3) The present invention reconstructs a three-dimensional image from a two-dimensional image through ultrasound, and when a mass artifact is found in the larynx, adjusts the ultrasound frequency to form a new artifact, and obtains the actual coordinate parameters of the mass through multiple measurements.

[0055] In summary, the present invention reconstructs a three-dimensional image based on a two-dimensional image through ultrasound, and when a mass artifact is found in the larynx, adjusts the ultrasound frequency to form a new artifact, and obtains the actual coordinate parameters of the mass through multiple measurements to solve the problem of refraction artifacts caused by laryngeal space-occupying lesions in the reconstruction of ultrasound images. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a method block diagram of a method for identifying laryngeal space-occupying lesions based on image recognition according to Example 1 of the present invention.

[0057] Figure 2 This is a schematic diagram of the principle of artifacts generated by ultrasonic refraction in Example 1 of the present invention.

[0058] Figure 3 This is a module block diagram of a laryngeal space-occupying lesion recognition system based on image recognition according to Example 2 of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Before giving examples, it is necessary to explain the application scenarios of the present invention. The present invention is to solve the refraction artifact problem in ultrasonic medical imaging. For example, if a patient with laryngeal space-occupying lesions has a cyst in the larynx, the core of the cyst is fluid, and when the ultrasound penetrates the cyst, it will be refracted. Figure 2 As shown, after the ultrasound partially penetrates the cystic effusion, it will continue to move forward until it hits a hard nucleus mass and is reflected. The emitted ultrasound penetrates the cystic effusion along the original path and reaches the ultrasound receiving device. After the reflected waves of the ultrasound are collected, an image of the larynx is drawn. However, the imaging position of the mass in the laryngeal image is the first position 1, while the actual position of the mass is the second position 2.

[0061] like Figure 1 As shown, this embodiment provides a method for identifying laryngeal space-occupying lesions based on image recognition, the method comprising:

[0062] Move the ultrasonic probe in the patient's throat along a preset path and collect two-dimensional ultrasonic images, wherein the preset path includes but is not limited to linear, rotational or free movement; record the scanning position and scanning direction of the ultrasonic probe when collecting two-dimensional ultrasonic images at different scanning points;

[0063] After filtering and enhancing the two-dimensional ultrasonic image of the patient's larynx, the pixels of the two-dimensional ultrasonic image corresponding to different scanning points are mapped to a three-dimensional spatial coordinate system through the scanning position and scanning direction to obtain a three-dimensional voxel model of the patient's larynx; the blank spaces of the voxels in the three-dimensional voxel model are interpolated, the color and transparency of each voxel are calculated through the three-dimensional voxel model, the penetration path of the light at the observation point passing through the three-dimensional voxel model is obtained, and the cumulative transparency and cumulative color of each voxel in the three-dimensional image of the patient's larynx are obtained through the penetration path;

[0064] The three-dimensional gradient of the three-dimensional image of the patient's larynx is obtained, and the edge of the three-dimensional image of the patient's larynx is detected by the three-dimensional gradient. The edge is input into the cyst detection network to obtain the cyst morphology, and the edge is input into the mass detection network to obtain the mass morphology. The cyst morphology includes the cyst center coordinate position and the cyst boundary, and the mass morphology includes the mass center coordinate position and the mass boundary. When the cyst morphology and the mass morphology exist at the same time, the cyst center coordinate position and the mass center coordinate position are connected to obtain a first path, and the ultrasonic probe is changed again to move in the patient's larynx along a preset path to construct the patient's larynx three-dimensional image and identify the cyst morphology and the mass morphology. The cyst center coordinate position and the mass center coordinate position are connected again to obtain a second path, and the mass correction center coordinate position is obtained through the first path and the second path, and the laryngeal space-occupying lesion parameters are obtained through the mass correction center coordinate position and the mass boundary. The cyst detection network is a cyst edge feature classifier established by a neural network algorithm, and the mass detection network is a mass edge feature classifier established by a neural network algorithm.

[0065] For example, a patient goes to the hospital for throat discomfort, and the doctor uses ultrasound to examine the larynx to identify possible space-occupying lesions. However, when there is a cyst in the patient's larynx, ultrasound will be refracted when it penetrates the cyst effusion, resulting in artifacts in the position of the mass in the imaging, thereby interfering with the doctor's diagnosis. First, an ultrasonic probe is used to move along a preset path in the patient's larynx, which includes linear, rotational and free movement, to collect two-dimensional ultrasonic images and record the position and direction of the probe at each scanning point. The collected image is enhanced by applying Gaussian filtering technology to remove noise and improve image quality. Using the enhanced two-dimensional image, combined with the scanning position and direction information, the image is reconstructed into a three-dimensional voxel model of the patient's larynx through a pixel mapping method, and the voxel blanks are filled in the model, and the color and transparency of each voxel are calculated. The edges of the image are detected by three-dimensional gradient information, and then the edges are input into a specially designed neural network to detect the morphology of cysts and masses respectively. These neural networks are based on the characteristics of ultrasonic images and can distinguish the specific location and boundaries of cysts and masses. After the cysts and masses are identified, their center position coordinates are used for path analysis. First, determine the path of the ultrasound wave penetrating the cyst and entering the mass. Then, by changing the ultrasound frequency and recollecting the data, the changes in the coordinates of the center of the mass in the two scan results are compared to calculate the true location of the mass.

[0066] Furthermore, the method for filtering and enhancing the two-dimensional ultrasonic image of the patient's larynx includes:

[0067] The Gaussian kernel of size n×n and the standard deviation σ are preset, and the kernel value G(i,j) at the kernel position (i,j) of the Gaussian kernel is:

[0068]

[0069] The Gaussian kernel is normalized so that the sum of the kernel values ​​of the Gaussian kernel is equal to 1, and the normalized Gaussian kernel is convolved with each pixel of the two-dimensional ultrasonic image; the convolution operation includes aligning the Gaussian kernel with the pixels of the two-dimensional ultrasonic image in sequence, and denoising and enhancing the pixel value I(u,v) of the coordinate (u,v) to obtain the corresponding pixel value I'(u,v):

[0070]

[0071] Where k is the radius of the Gaussian kernel, and I(u+i,v+j) is the pixel value at the offset position (u+i,v+j) relative to the current coordinate (u,v); the convolution operation is repeated until all pixel values ​​I(u,v) of the two-dimensional ultrasonic image are denoised and enhanced to obtain the pixel value I′(u,v).

[0072] For example, a two-dimensional image of the patient's throat is collected by an ultrasonic probe, and the image is filtered and enhanced by, for example, setting a Gaussian kernel of size 5x5 and selecting an appropriate standard deviation such as σ=1.5 to effectively remove random noise in the image while retaining important structural features in the image, such as the edge of a mass or cyst in the throat. In the specific operation, each pixel value is replaced with the weighted average of its neighboring pixels, and the weight is determined by the Gaussian kernel to smooth the image and enhance the edges in the image.

[0073] Furthermore, the method of mapping pixels of a two-dimensional ultrasonic image corresponding to different scanning points into a three-dimensional spatial coordinate system by scanning position and scanning direction to obtain a three-dimensional voxel model of the patient's larynx includes:

[0074] The translation vector is obtained by scanning the change of position The rotation vector is obtained by changing the scanning direction The coordinates (u, v) of the pixels of the two-dimensional ultrasound image are mapped to the coordinates (u', v', w') of the voxels in the three-dimensional spatial coordinate system, where the coordinates (u', v', w') are calculated as follows:

[0075]

[0076] Where T is the transformation matrix, the transformation matrix T is:

[0077]

[0078] Exemplarily, a 3D voxel model of the patient's larynx is constructed based on the enhanced 2D images and the scanning position and orientation information corresponding to each image. Each pixel of the 2D image is mapped to a voxel in 3D space. The mapping process uses the pre-recorded probe position and orientation data to calculate the exact position of each pixel in 3D space through geometric transformation. Interpolation techniques are used to fill in the blank areas in the 3D model, and the color and transparency of the blank voxels are estimated based on the color and position of the known voxels.

[0079] Furthermore, the method for interpolating the blank voxel spaces in the three-dimensional voxel model includes:

[0080] Get the coordinate point P(u′, v′, w′) of the blank voxel in the three-dimensional coordinate system of space, get the voxel values ​​adjacent to the coordinate point P(u′, v′, w′) in three mutually perpendicular directions in the three-dimensional coordinate system of space, and interpolate the voxel value V of the coordinate point P(u′, v′, w′) P for:

[0081] V P (u′, v′, w′) = V 000 (1-u′)(1-v′)(1-w′)+V 001(1-u′)(1-v′)w′+V 010 (1-u′)v′(1-w′)+V 011 (1-u′)v′w′+V 100 u′(1-v′)(1-w′)+V 101 u′(1-v′)w′+V 110 u′v′(1-w′)+V 111 u′v′w′;

[0082] Among them, V 000 is the minimum value of the voxel value in the x, y and z axis directions, V 001 is the maximum voxel value in the z-axis direction and the minimum voxel value in the x- and y-axis directions, V 010 is the maximum voxel value in the y-axis direction and the minimum voxel value in the x- and z-axis directions, V 011 is the maximum value of the voxel value in the z and y axis directions and the minimum value of the voxel value in the x axis direction, V 100 is the maximum value of the voxel in the x-axis direction and the minimum value of the voxel in the y- and z-axis directions, V 101 is the maximum value of the voxel value in the x- and z-axis directions and the minimum value of the voxel value in the y-axis direction, V 110 is the maximum value of the voxel value in the x- and y-axis directions and the minimum value of the voxel value in the z-axis direction, V 111 It is the maximum value of the voxel value in the x, y and z axis directions.

[0083] Exemplarily, by utilizing enhanced two-dimensional ultrasonic images and their corresponding scanning positions and directions, these two-dimensional images are mapped into three-dimensional space to form a three-dimensional model. Specifically, the scanning position and direction information is used to determine the exact position of each two-dimensional image pixel in three-dimensional space, thereby organizing these pixels into a continuous three-dimensional voxel model. Due to the limitations of the discontinuity of the scan, there may be blank areas in the model. Interpolation technology is used to estimate and fill the voxel values ​​of these blank areas. Specifically, the values ​​of the blank voxels are calculated using an interpolation method by obtaining the values ​​of the known voxels around the blank voxels.

[0084] Furthermore, the method for obtaining the corrected center coordinate position of the mass through the first path and the second path includes:

[0085] Obtain the incident point positions L1(u′, v′, w′) and L2(u′, v′, w′) of the ultrasound entering the cyst corresponding to the first path and the second path, respectively, and record the direction vectors corresponding to the ultrasound along the first path and the second path as and The center coordinates of the mass corresponding to the first path and the second path are recorded as P1(u′, v′, w′) and P2(u′, v′, w′), the corrected center coordinates of the mass are recorded as P(u′, v′, w′), and the refraction points of the ultrasound penetrating the cyst corresponding to the first path and the second path are recorded as R1(u′, v′, w′) and R2(u′, v′, w′); the ultrasonic refraction equations are established as follows:

[0086]

[0087] Wherein, d1 is the distance from R1(u′, v′, w′) to P1(u′, v′, w′), d2 is the distance from R2(u′, v′, w′) to P2(u′, v′, w′); the corrected center coordinate position of the mass is obtained by using the ultrasonic refraction equations and recorded as P(u′, v′, w′).

[0088] For example, Figure 2 As shown in the figure, in the process of accurately locating the mass, artifacts are generated by the refraction of ultrasound waves by cystic effusions. The change in the propagation direction of ultrasound waves when passing through different media (such as cystic effusions and surrounding tissues) directly causes false artifacts in the display of the mass location. The solution is to change the position of the ultrasound probe and adjust the scanning parameters, re-collect data on the cyst and mass area, and record the specific path of penetrating the cyst to reach the mass, including the position of the ultrasound's incident point 3 and refraction point 4. By comparing the scan results of the initial scan and the scan results after adjusting the parameters, the position deviation caused by refraction can be identified.

[0089] Embodiment 2:

[0090] Based on the same inventive concept, Figure 3 As shown, this embodiment also provides a laryngeal space-occupying lesion recognition system based on image recognition, the system comprising:

[0091] An image acquisition module is used to move the ultrasonic probe in the patient's throat along a preset path and acquire a two-dimensional ultrasonic image. The preset path includes but is not limited to linear, rotational or free movement; and record the scanning position and scanning direction of the ultrasonic probe when acquiring the two-dimensional ultrasonic image at different scanning points;

[0092] A three-dimensional reconstruction module is used to filter and enhance the two-dimensional ultrasonic image of the patient's larynx, and then map the pixels of the two-dimensional ultrasonic image corresponding to different scanning points to a spatial three-dimensional coordinate system through scanning positions and scanning directions to obtain a three-dimensional voxel model of the patient's larynx; interpolate the blank spaces of the voxels in the three-dimensional voxel model, calculate the color and transparency of each voxel through the three-dimensional voxel model, obtain the penetration path of the light at the observation point through the three-dimensional voxel model, and obtain the cumulative transparency and cumulative color of each voxel in the three-dimensional image of the patient's larynx through the penetration path;

[0093] The laryngeal space-occupying lesion recognition module is used to obtain the three-dimensional gradient of the three-dimensional image of the patient's larynx, detect the edge of the three-dimensional image of the patient's larynx through the three-dimensional gradient, input the edge into the cyst detection network to obtain the cyst morphology, and input the edge into the mass detection network to obtain the mass morphology. The cyst morphology includes the cyst center coordinate position and the cyst boundary, and the mass morphology includes the mass center coordinate position and the mass boundary. When the cyst morphology and the mass morphology exist at the same time, the cyst center coordinate position and the mass center coordinate position are connected to obtain a first path, and the ultrasonic probe is changed again to move in the patient's larynx along a preset path to construct the patient's larynx three-dimensional image and identify the cyst morphology and the mass morphology. The cyst center coordinate position and the mass center coordinate position are connected again to obtain a second path, and the mass correction center coordinate position is obtained through the first path and the second path, and the laryngeal space-occupying lesion parameters are obtained through the mass correction center coordinate position and the mass boundary. The cyst detection network is a cyst edge feature classifier established by a neural network algorithm, and the mass detection network is a mass edge feature classifier established by a neural network algorithm.

[0094] Furthermore, the system further comprises:

[0095] The image filtering module is used to preset the Gaussian kernel of size n×n and the standard deviation σ. The kernel value G(i,j) at the kernel position (i,j) of the Gaussian kernel is:

[0096]

[0097] The Gaussian kernel is normalized so that the sum of the kernel values ​​of the Gaussian kernel is equal to 1, and the normalized Gaussian kernel is convolved with each pixel of the two-dimensional ultrasonic image; the convolution operation includes aligning the Gaussian kernel with the pixels of the two-dimensional ultrasonic image in sequence, and denoising and enhancing the pixel value I(u,v) of the coordinate (u,v) to obtain the corresponding pixel value I'(u,v):

[0098]

[0099] Where k is the radius of the Gaussian kernel, and I(u+i,v+j) is the pixel value of the offset position (u+i,v+j) relative to the current coordinate (u,v); the convolution operation is repeated until all pixel values ​​I(u,v) of the two-dimensional ultrasonic image are denoised and enhanced to obtain the pixel value I′(u,v).

[0100] Furthermore, the system further comprises:

[0101] Coordinate transformation module, used to obtain the translation vector by scanning position changes The rotation vector is obtained by changing the scanning direction The coordinates (u, v) of the pixels of the two-dimensional ultrasonic image are mapped to the coordinates (u′, v′, w′) of the voxels in the three-dimensional spatial coordinate system, where the coordinates (u′, v′, w′) are calculated as follows:

[0102]

[0103] Where T is the transformation matrix, the transformation matrix T is:

[0104]

[0105] Furthermore, the system further comprises:

[0106] The interpolation module is used to obtain the coordinate point P(u′, v′, w′) of the blank voxel in the three-dimensional coordinate system of the space, obtain the voxel values ​​adjacent to the coordinate point P(u′, v′, w′) in three mutually perpendicular directions in the three-dimensional coordinate system of the space, and interpolate the voxel value V of the coordinate point P(u′, v′, w′) P for:

[0107] V P (u′, v′, w′) = V 000 (1-u′)(1-v′)(1-w′)+V 001 (1-u′)(1-v′)w′+V 010 (1-u′)v′(1-w′)+V 011 (1-u′)v′w′+V 100 u′(1-v′)(1-w′)+V 101 u′(1-v′)w′+V 110 u′v′(1-w′)+V 111 u′v′w′;

[0108] Among them, V 000 is the minimum value of the voxel value in the x, y and z axis directions, V 001 is the maximum voxel value in the z-axis direction and the minimum voxel value in the x- and y-axis directions, V 010is the maximum voxel value in the y-axis direction and the minimum voxel value in the x- and z-axis directions, V 011 is the maximum value of the voxel value in the z and y axis directions and the minimum value of the voxel value in the x axis direction, V 100 is the maximum value of the voxel in the x-axis direction and the minimum value of the voxel in the y- and z-axis directions, V 101 is the maximum value of the voxel value in the x- and z-axis directions and the minimum value of the voxel value in the y-axis direction, V 110 is the maximum value of the voxel value in the x- and y-axis directions and the minimum value of the voxel value in the z-axis direction, V 111 It is the maximum value of the voxel value in the x, y and z axis directions.

[0109] Furthermore, the system further comprises:

[0110] The artifact correction module is used to obtain the incident point positions L1(u′, v′, w′) and L2(u′, v′, w′) of the ultrasound entering the cyst corresponding to the first path and the second path, and record the direction vectors corresponding to the ultrasound along the first path and the second path as and The center coordinates of the mass corresponding to the first path and the second path are recorded as P1(u′, v′, w′) and P2(u′, v′, w′), the corrected center coordinates of the mass are recorded as P(u′, v′, w′), and the refraction points of the ultrasound penetrating the cyst corresponding to the first path and the second path are recorded as R1(u′, v′, w′) and R2(u′, v′, w′); the ultrasonic refraction equations are established as follows:

[0111]

[0112] Wherein, d1 is the distance from R1(u′, v′, w′) to P1(u′, v′, w′), d2 is the distance from R2(u′, v′, w′) to P2(u′, v′, w′); the corrected center coordinate position of the mass is obtained by using the ultrasonic refraction equations and recorded as P(u′, v′, w′).

[0113] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0114] Finally, it should be noted that: Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. Identification method of laryngeal space-occupying lesions based on image recognition, It is characterized in that The method comprises: Move the ultrasonic probe in the patient's throat along a preset path and collect two-dimensional ultrasonic images, wherein the preset path includes linear, rotational or free movement; record the scanning position and scanning direction of the ultrasonic probe when collecting two-dimensional ultrasonic images at different scanning points; After filtering and enhancing the two-dimensional ultrasonic image of the patient's larynx, the pixels of the two-dimensional ultrasonic image corresponding to different scanning points are mapped to a three-dimensional spatial coordinate system through the scanning position and scanning direction to obtain a three-dimensional voxel model of the patient's larynx; the blank spaces of the voxels in the three-dimensional voxel model are interpolated, the color and transparency of each voxel are calculated through the three-dimensional voxel model, the penetration path of the light at the observation point passing through the three-dimensional voxel model is obtained, and the cumulative transparency and cumulative color of each voxel in the three-dimensional image of the patient's larynx are obtained through the penetration path; The three-dimensional gradient of the three-dimensional image of the patient's larynx is obtained, and the edge of the three-dimensional image of the patient's larynx is detected by the three-dimensional gradient, and the edge is input into the cyst detection network to obtain the cyst morphology, and the edge is input into the mass detection network to obtain the mass morphology, wherein the cyst morphology includes the cyst center coordinate position and the cyst boundary, and the mass morphology includes the mass center coordinate position and the mass boundary; when the cyst morphology and the mass morphology exist at the same time, the cyst center coordinate position and the mass center coordinate position are connected to obtain a first path, and the ultrasonic probe is again changed in the ultrasonic frequency to move in the patient's larynx along a preset path to construct the patient's larynx three-dimensional image and identify the cyst morphology and the mass morphology, and the cyst center coordinate position and the mass center coordinate position are connected again to obtain a second path, and the mass correction center coordinate position is obtained through the first path and the second path, and the laryngeal space-occupying lesion parameters are obtained through the mass correction center coordinate position and the mass boundary; the cyst detection network is a cyst edge feature classifier established by a neural network algorithm, and the mass detection network is a mass edge feature classifier established by a neural network algorithm; The method for mapping pixels of a two-dimensional ultrasonic image corresponding to different scanning points to a three-dimensional coordinate system in space by scanning position and scanning direction to obtain a three-dimensional voxel model of the patient's larynx comprises: The translation vector is obtained by scanning the change of position The rotation vector is obtained by changing the scanning direction The coordinates (u, v) of the pixels of the two-dimensional ultrasound image are mapped to the coordinates (u', v', w') of the voxels in the three-dimensional spatial coordinate system, where the coordinates (u', v', w') The calculation formula is: Where T is the transformation matrix, the transformation matrix T is: The method for interpolating the blank voxel space in the three-dimensional voxel model comprises: Get the coordinate point P(u',v',w') of the blank voxel in the three-dimensional coordinate system of space, get the voxel values ​​adjacent to the coordinate point P(u',v',w') in three mutually perpendicular directions in the three-dimensional coordinate system of space, and interpolate the voxel value V of the coordinate point P(u',v',w') P for: V P (u′,v′,w′)=V 000 (1-u′)(1-v′)(1-w′)+V 001 (1-u′)(1-v′)w′+V 010 (1-u′)v′(1-w′)+V 011 (1-u′)v′w′+V 100 u′(1-v′)(1-w′)+V 101 u′(1-v′)w′+V 110 u′v′(1-w′)+V 111 u′v′w′; Among them, V 000 is the minimum value of the voxel value in the x, y and z axis directions, V 001 is the maximum voxel value in the z-axis direction and the minimum voxel value in the x- and y-axis directions, V 010 is the maximum voxel value in the y-axis direction and the minimum voxel value in the x- and z-axis directions, V 011 is the maximum value of the voxel value in the z and y axis directions and the minimum value of the voxel value in the x axis direction, V 100 is the maximum value of the voxel in the x-axis direction and the minimum value of the voxel in the y- and z-axis directions, V 101 is the maximum value of the voxel value in the x- and z-axis directions and the minimum value of the voxel value in the y-axis direction, V 110 is the maximum value of the voxel value in the x- and y-axis directions and the minimum value of the voxel value in the z-axis direction, V 111 is the maximum value of the voxel value in the x, y and z axis directions; The method for obtaining the corrected center coordinate position of the mass through the first path and the second path includes: Obtain the incident point position L of the ultrasound entering the cyst corresponding to the first path and the second path respectively 1 (u',v',w') and L 2 (u', ​​v', w'), and the direction vectors corresponding to the ultrasonic wave along the first path and the second path are respectively recorded as and The coordinate positions of the center of the mass corresponding to the first path and the second path are denoted as P 1 (u',v',w') and P 2 (u', ​​v', w'), the corrected center coordinate position of the mass is recorded as P(u', v', w'), and the refraction point position of the ultrasound penetrating the cyst corresponding to the first path and the second path is recorded as R 1 (u',v',w') and R 2 (u',v',w'); establish the ultrasonic refraction equation group as: Among them, d 1 YesR 1 (u',v',w') to P 1 The distance between (u',v',w'), d 2 YesR 2 (u',v',w') to P 2 The distance between (u',v',w'); t 1 YesL 1 (u',v',w') to P 1 The distance between (u',v',w'), t 2 YesL 2 (u',v',w') to P 1 (u',v',w'); the corrected center coordinate position of the mass is obtained by the ultrasonic refraction equation group and recorded as P(u',v',w').

2. The method for identifying laryngeal space-occupying lesions based on image recognition according to claim 1, It is characterized in that The method for filtering and enhancing the two-dimensional ultrasonic image of the patient's larynx comprises: The Gaussian kernel of size n×n and the standard deviation σ are preset, and the kernel value G(i,j) at the kernel position (i,j) of the Gaussian kernel is: The Gaussian kernel is normalized so that the sum of the kernel values ​​of the Gaussian kernel is equal to 1, and the normalized Gaussian kernel is convolved with each pixel of the two-dimensional ultrasonic image; the convolution operation includes aligning the Gaussian kernel with the pixels of the two-dimensional ultrasonic image in sequence, and denoising and enhancing the pixel value I(u,v) of the coordinate (u,v) to obtain the corresponding pixel value I'(u,v): Where k is the radius of the Gaussian kernel, and I(u+i,v+j) is the pixel value at the offset position (u+i,v+j) relative to the current coordinate (u,v); the convolution operation is repeated until all pixel values ​​I(u,v) of the two-dimensional ultrasonic image are denoised and enhanced to obtain the pixel value I′(u,v).

3. Laryngeal space-occupying lesion recognition system based on image recognition, It is characterized in that The system comprises: An image acquisition module is used to move the ultrasonic probe in the patient's throat along a preset path and acquire a two-dimensional ultrasonic image, wherein the preset path includes linear, rotational or free movement; and record the scanning position and scanning direction of the ultrasonic probe when acquiring a two-dimensional ultrasonic image at different scanning points; The three-dimensional reconstruction module is used to filter and enhance the two-dimensional ultrasonic image of the patient's larynx, and then map the pixels of the two-dimensional ultrasonic image corresponding to different scanning points to the three-dimensional coordinate system of the space to obtain a three-dimensional voxel model of the patient's larynx through the scanning position and scanning direction; interpolate the blank voxels in the three-dimensional voxel model, and calculate the color of each voxel through the three-dimensional voxel model. Color and transparency, obtain the penetration path of the light at the observation point through the three-dimensional voxel model, and obtain the cumulative transparency and cumulative color of each voxel in the three-dimensional image of the patient's larynx through the penetration path; The laryngeal space-occupying lesion recognition module is used to obtain the three-dimensional gradient of the three-dimensional image of the patient's larynx, detect the edge of the three-dimensional image of the patient's larynx through the three-dimensional gradient, input the edge into the cyst detection network to obtain the cyst morphology, and input the edge into the mass detection network to obtain the mass morphology. The cyst morphology includes the cyst center coordinate position and the cyst boundary, and the mass morphology includes the mass center coordinate position and the mass boundary. When the cyst morphology and the mass morphology exist at the same time, the cyst center coordinate position and the mass center coordinate position are connected to obtain a first path, and the ultrasonic probe is changed again to move in the patient's larynx along a preset path to construct the patient's larynx three-dimensional image and identify the cyst morphology and the mass morphology. The cyst center coordinate position and the mass center coordinate position are connected again to obtain a second path, and the mass correction center coordinate position is obtained through the first path and the second path, and the laryngeal space-occupying lesion parameters are obtained through the mass correction center coordinate position and the mass boundary. The cyst detection network is a cyst edge feature classifier established by a neural network algorithm, and the mass detection network is a mass edge feature classifier established by a neural network algorithm. The system also includes a coordinate transformation module for obtaining a translation vector by changing the scanning position. The rotation vector is obtained by changing the scanning direction The coordinates (u, v) of the pixels of the two-dimensional ultrasound image are mapped to the coordinates (u', v', w') of the voxels in the three-dimensional spatial coordinate system, where the coordinates (u', v', w') are calculated as follows: Where T is the transformation matrix, the transformation matrix T is: The system also includes an interpolation module for obtaining a coordinate point P (u', v', w') at a blank voxel in a three-dimensional spatial coordinate system, obtaining voxel values ​​adjacent to the coordinate point P (u', v', w') in three mutually perpendicular directions in the three-dimensional spatial coordinate system, and interpolating the voxel value V of the coordinate point P (u', v', w') P for: V P (u′,v′,w′)=V 000 (1-u′)(1-v′)(1-w′)+V 001 (1-u′)(1-v′)w′+V 010 (1-u′)v′(1-w′)+V 011 (1-u′)v′w′+V 100 u′(1-v′)(1-w′)+V 101 u′(1-v′)w′+V 110 u′v′(1-w′)+V 111 u′v′w′; Among them, V 000 is the minimum value of the voxel value in the x, y and z axis directions, V 001 is the maximum voxel value in the z-axis direction and the minimum voxel value in the x- and y-axis directions, V 010 is the maximum voxel value in the y-axis direction and the minimum voxel value in the x- and z-axis directions, V 011 is the maximum value of the voxel value in the z and y axis directions and the minimum value of the voxel value in the x axis direction, V 100 is the maximum value of the voxel in the x-axis direction and the minimum value of the voxel in the y- and z-axis directions, V 101 is the maximum value of the voxel value in the x- and z-axis directions and the minimum value of the voxel value in the y-axis direction, V 110 is the maximum value of the voxel value in the x- and y-axis directions and the minimum value of the voxel value in the z-axis direction, V 111 is the maximum value of the voxel value in the x, y and z axis directions; The system also includes an artifact correction module for obtaining the incident point position L of the ultrasound entering the cyst corresponding to the first path and the second path respectively. 1 (u',v',w') and L 2 (u', ​​v', w'), and the direction vectors corresponding to the ultrasonic wave along the first path and the second path are respectively recorded as and The coordinate positions of the center of the mass corresponding to the first path and the second path are denoted as P 1 (u',v',w') and P 2 (u', ​​v', w'), the corrected center coordinate position of the mass is recorded as P(u', v', w'), and the refraction point position of the ultrasound penetrating the cyst corresponding to the first path and the second path is recorded as R 1 (u',v',w') and R 2 (u',v',w'); establish the ultrasonic refraction equation group as: Among them, d 1 YesR 1 (u',v',w') to P 1 The distance between (u',v',w'), d 2 YesR 2 (u',v',w') to P 2 The distance between (u',v',w'); t 1 YesL 1 (u',v',w') to P 1 The distance between (u',v',w'), t 2 YesL 2 (u',v',w') to P 1 (u',v',w'); the corrected center coordinate position of the mass is obtained by the ultrasonic refraction equation group and recorded as P(u',v',w').

4. The laryngeal space-occupying lesion recognition system based on image recognition according to claim 3, It is characterized in that The system further comprises: The image filtering module is used to preset the Gaussian kernel of size n×n and the standard deviation σ. The kernel value G(i,j) at the kernel position (i,j) of the Gaussian kernel is: The Gaussian kernel is normalized so that the sum of the kernel values ​​of the Gaussian kernel is equal to 1, and the normalized Gaussian kernel is convolved with each pixel of the two-dimensional ultrasonic image; the convolution operation includes aligning the Gaussian kernel with the pixels of the two-dimensional ultrasonic image in sequence, and denoising and enhancing the pixel value I(u,v) of the coordinate (u,v) to obtain the corresponding pixel value I'(u,v): Where k is the radius of the Gaussian kernel, and I(u+i,v+j) is the pixel value at the offset position (u+i,v+j) relative to the current coordinate (u,v); the convolution operation is repeated until all pixel values ​​I(u,v) of the two-dimensional ultrasonic image are denoised and enhanced to obtain the pixel value I′(u,v).

Images