Nasointestinal tube positioning method based on artificial intelligence and related device

Abstract

The invention discloses a nasointestinal tube positioning method based on artificial intelligence and a related device. The method comprises the following steps: acquiring a first ultrasonic image and a first ultrasonic signal of a nasointestinal tube inserted into a digestive tract through a nasal cavity in real time; extracting a multi-channel image of the first ultrasonic image through a plurality of groups of residual networks to obtain a multi-channel image feature; multi-channel ultrasound of the first ultrasonic signal is extracted by combining a scale invariant feature transformation network and a U-NET neural network, and multi-channel ultrasonic features are obtained; inputting the multi-channel image features and the multi-channel ultrasonic features into an ultrasonic synthesis neural network for waveform mask estimation, and outputting a waveform mask estimation result; and inputting the waveform mask estimation result and the first ultrasonic signal into an inverse short-time Fourier transform network, and outputting a double-track acoustic image containing the nasointestinal tube. According to the method, the features of the image and the ultrasonic signal are extracted from multiple dimensions, and the waveform mask estimation result and the original ultrasonic signal are fused, so that the double-track acoustic image containing the nasointestinal tube can be displayed more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly relates to a method and device for positioning a nasoenteric tube based on artificial intelligence, and a computing device. Background Art

[0002] Enteral nutrition (EN) is the preferred method for nutritional support of critically ill patients, which can ensure that patients obtain early nutritional supplementation, maintain the gastrointestinal function of patients, and play an important role in promoting the recovery of the immune function of patients and accelerating the recovery of diseases. A nasoenteric tube is a tube inserted through the nasal cavity, passing through the pharynx, esophagus, and stomach, and placed in the duodenum or jejunum for enteral nutrition infusion. Through the nasoenteric tube, liquid food or nutrient solution can be directly injected into the small intestine, which can maximize the absorption of nutrients and effectively avoid aspiration caused by gastric retention, gastric discomfort and other refluxes. Blind insertion of the nasoenteric tube by hand is the most commonly used method in clinical practice, but it is necessary to repeatedly confirm the position of the catheter tip to ensure the safety and effectiveness of enteral nutrition administration after the nasoenteric tube is inserted.

[0003] Currently, methods such as auscultation, vacuum test, detection of pH value by extracting digestive juice, ultrasonic determination, and magnetic navigation trajectory line can be used to determine the position of the nasoenteric tube tip. However, since the jejunum does not have a storage function and has a fast emptying rate, it often occurs that digestive juice cannot be extracted during actual operation. In addition, the gastric juice of patients who have been using proton pump inhibitors or acid-suppressing drugs for a long time may also show weak acidity, or even alkalinity due to bile reflux, which will interfere with the determination results, and the start time of enteral nutrition is often postponed due to waiting to confirm the nasoenteric tube positioning. In addition, ultrasonic determination and magnetic navigation trajectory line of nasoenteric tube positioning have high requirements for personnel and are more complex to operate.

[0004] To solve the above problems, the present invention proposes a method for positioning a nasoenteric tube based on artificial intelligence to more accurately display the double-track sonogram including the nasoenteric tube. Summary of the Invention

[0005] In view of the above problems, the present invention provides a method and device for positioning a nasoenteric tube based on artificial intelligence, and a computing device.

[0006] According to one aspect of the present invention, there is provided a method for positioning a nasoenteric tube based on artificial intelligence, including:

[0007] Real-time acquiring a first ultrasonic image and a first ultrasonic signal of the nasoenteric tube inserted through the nasal cavity into the digestive tract;

[0008] Extracting multi-channel images of the first ultrasonic image through multiple groups of residual networks to obtain multi-channel image features; extracting multi-channel ultrasounds of the first ultrasonic signal through a combination of a scale-invariant feature transform network and a U-NET neural network to obtain multi-channel ultrasound features;

[0009] Input the multi-channel image features and the multi-channel ultrasound features into an ultrasound synthesis neural network for waveform mask estimation, and output the waveform mask estimation result; input the waveform mask estimation result and the first ultrasound signal into an inverse short-time Fourier transform network, and output a dual-track sonogram containing a nasointestinal tube.

[0010] In an alternative approach, the ultrasound synthesis neural network employs a generative adversarial network, which includes a generator and a discriminator;

[0011] Among them, the generator is used to extract features of different scales through a multi-scale feature fusion module and fuse them to generate a waveform mask estimation result.

[0012] The discriminator is used to input the first ultrasound image into the discriminator, enabling the discriminator to evaluate the matching degree between the waveform mask estimation result output by the generator and the true nasointestinal tube sonogram.

[0013] In an alternative approach, the inverse short-time Fourier transform network uses a convolutional neural network to fuse the waveform mask estimation result and the first ultrasound signal to obtain a fused feature;

[0014] The inverse short-time Fourier transform network converts the fused feature into a segmented time-domain signal;

[0015] Process the segmented time-domain signal according to the Overlap-Add method to obtain a dual-track sonogram containing a nasointestinal tube.

[0016] In an alternative approach, the generator gradually generates a waveform mask from low resolution to high resolution, where the output of each resolution stage is used as the input of the next stage, and the generation result is gradually refined.

[0017] In an alternative approach, the step of inputting the waveform mask estimation result and the first ultrasound signal into the inverse short-time Fourier transform network further includes:

[0018] Obtain a restored time-domain signal based on the waveform mask estimation result and the first ultrasound signal;

[0019] Input the above restored time-domain signal into the above inverse short-time Fourier transform network.

[0020] In an alternative approach, the calculation formula for the restored time-domain signal is:

[0021]

[0022] Among them, s(t) is the restored time-domain signal; STFT -1is the inverse short-time Fourier transform; X[k] is the k-th component in the frequency domain; k and l are frequency components; t is a point on the time axis.

[0023] In an alternative manner, the loss function for each resolution stage is:

[0024]

[0025] where L (k) is the loss function for the k-th resolution stage; D KL is the KL divergence; is the data distribution generated in the k-th resolution stage; P data (x) is the true data distribution; D(x) is the output of the discriminator; λ is the regularization coefficient; is the expectation of the random variable z, where z follows the distribution P z (z); P z (z) is the Gaussian probability distribution of the random variable z.

[0026] In an alternative manner, the generator adopts a pyramid network structure;

[0027] where each layer of the pyramid network includes a feature extraction module, a feature fusion module, and an upsampling module for extracting, fusing, and upsampling features; and, each layer of the pyramid network constrains the generated waveform mask estimation result through a loss function to ensure the similarity between the generated waveform mask and the true nasointestinal tube sonogram.

[0028] According to another aspect of the present invention, there is provided an artificial intelligence-based nasointestinal tube positioning device, including:

[0029] An ultrasonic acquisition module for real-time acquisition of the first ultrasonic image and the first ultrasonic signal of the nasointestinal tube inserted into the digestive tract through the nasal cavity;

[0030] A feature extraction module for extracting multi-channel images of the first ultrasonic image through multiple groups of residual networks to obtain multi-channel image features; and extracting multi-channel ultrasounds of the first ultrasonic signal by combining a scale-invariant feature transform network and a U-NET neural network to obtain multi-channel ultrasound features;

[0031] An acoustic image output module for inputting the multi-channel image features and the multi-channel ultrasound features into an ultrasonic synthesis neural network for waveform mask estimation, and outputting a waveform mask estimation result; and inputting the waveform mask estimation result and the first ultrasonic signal into an inverse short-time Fourier transform network to output a dual-track acoustic image including the nasointestinal tube.

[0032] According to another aspect of the present invention, there is provided a computing device, including: a processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0033] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the operations corresponding to the above-mentioned nasoenteric tube positioning method based on artificial intelligence.

[0034] According to the solution provided by the present invention, a first ultrasound image and a first ultrasound signal of the nasoenteric tube inserted into the digestive tract through the nasal cavity are obtained in real time; multi-channel images of the first ultrasound image are extracted through multiple groups of residual networks to obtain multi-channel image features; multi-channel ultrasounds of the first ultrasound signal are extracted by combining a scale-invariant feature transform network and a U-NET neural network to obtain multi-channel ultrasound features; the multi-channel image features and the multi-channel ultrasound features are input into an ultrasound synthesis neural network for waveform mask estimation, and a waveform mask estimation result is output; the waveform mask estimation result and the first ultrasound signal are input into an inverse short-time Fourier transform network, and a double-track sonogram including the nasoenteric tube is output. The present invention extracts features of images and ultrasound signals from multiple dimensions and by fusing the waveform mask estimation result and the original ultrasound signal, can more accurately display the double-track sonogram including the nasoenteric tube.

[0035] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates the specific embodiments of the present invention. Description of the Drawings

[0036] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0037] Figure 1 A schematic flow chart of the nasoenteric tube positioning method based on artificial intelligence according to an embodiment of the present invention is shown;

[0038] Figure 2 A schematic diagram of the neural network structure for nasoenteric tube positioning according to an embodiment of the present invention is shown;

[0039] Figure 3 A schematic diagram showing the B-ultrasound positioning of the nasoenteric tube entering the esophagus according to an embodiment of the present invention is shown;

[0040] Figure 4Shows a schematic diagram of the double-track sonogram of the B-ultrasound positioned nasointestinal tube in the intestine according to an embodiment of the present invention;

[0041] Figure 5 Shows a schematic diagram of the framework of the nasointestinal tube positioning device based on artificial intelligence according to an embodiment of the present invention;

[0042] Figure 6 Shows a schematic diagram of the structure of a computing device according to an embodiment of the present invention. Detailed implementation manners

[0043] Hereinafter, exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be fully conveyed to those skilled in the art.

[0044] Figure 1 Shows a schematic diagram of the flow of the nasointestinal tube positioning method based on artificial intelligence according to an embodiment of the present invention. Specifically, as Figure 1 shown, it includes the following steps:

[0045] Step S101, obtain the first ultrasonic image and the first ultrasonic signal of the nasointestinal tube inserted into the digestive tract through the nasal cavity in real time.

[0046] Specifically, the nasointestinal tube inserted into the patient through the nasal cavity is inserted to the pharynx. Let the patient perform a swallowing action to enable the nasointestinal tube to smoothly pass through the pharynx, and continue to insert the nasointestinal tube until the nasointestinal tube reaches the target position (such as in the stomach, near the pylorus, or the duodenum). During the catheterization process, the ultrasonic image and the ultrasonic signal output by the ultrasonic device are obtained in real time.

[0047] Step S102, extract multi-channel images of the first ultrasonic image through multiple groups of residual networks to obtain multi-channel image features; extract multi-channel ultrasounds of the first ultrasonic signal by combining the scale-invariant feature transform network and the U-NET neural network to obtain multi-channel ultrasound features.

[0048] As Figure 2 shown, features with scale invariance in the ultrasonic signal are extracted through the scale-invariant feature transform network (SIFT), which has good adaptability to ultrasonic images at different scales. The encoder-decoder architecture of the U-NET neural network can effectively capture the context information in the image and can significantly improve the extraction effect of the multi-channel features of the ultrasonic signal.

[0049] Specifically, the features of the first ultrasound image are extracted through multiple groups of residual networks. Each group of residual networks contains multiple convolutional layers and residual connections to extract image features at different levels. The outputs of each group of residual networks are concatenated to obtain multi-channel image features. The preprocessed ultrasound signal is input into the SIFT network to extract scale-invariant features. At the same time, the ultrasound signal is input into the U-NET neural network, and the encoder-decoder architecture is used to capture context information to extract multi-channel ultrasound features. The outputs of the SIFT network and the U-NET network are fused to obtain comprehensive multi-channel ultrasound features.

[0050] Step S103, input the multi-channel image features and the multi-channel ultrasound features into the ultrasound synthesis neural network for waveform mask estimation, and output the waveform mask estimation result; input the waveform mask estimation result and the first ultrasound signal into the inverse short-time Fourier transform network, and output the double-track sonogram containing the nasoenteric tube.

[0051] In this embodiment, by fusing multi-channel image features and multi-channel ultrasound features, the information of the nasoenteric tube in the ultrasound image and ultrasound signal can be more comprehensively reflected, and the accuracy of waveform mask estimation is improved. The ultrasound synthesis neural network performs waveform mask estimation based on the input features, effectively distinguishing the ultrasound signals of the nasoenteric tube from other tissue structures. By combining the waveform mask estimation result with the first ultrasound signal through the inverse short-time Fourier transform network, the double-track sonogram containing the nasoenteric tube is restored. As Figure 3 shown, the B-ultrasound locates the nasoenteric tube entering the esophagus, and the catheter can be seen in the lower left corner of the trachea, confirming that the nasoenteric tube has entered the esophagus. As Figure 4 shown, the B-ultrasound locates the nasoenteric tube in the intestine. This range is the sonogram of the pylorus migrating to the duodenum, and the "double-track" sonogram appears in the area where it is imaged, which is the nasoenteric tube.

[0052] Specifically, the extracted multi-channel image features and multi-channel ultrasound features are used as inputs and fed into the ultrasound synthesis neural network. The ultrasound synthesis neural network performs waveform mask estimation based on the input features, and learns the feature patterns of the nasoenteric tube ultrasound signals through training the network.

[0053] The waveform mask estimation result is combined with the first ultrasound signal, and the waveform mask is applied to the ultrasound signal to highlight the signal of the nasoenteric tube and suppress other background signals. The processed ultrasound signal is fed into the inverse short-time Fourier transform network, and the inverse short-time Fourier transform converts the signal from the frequency domain back to the time domain, thereby restoring the double-track sonogram containing the nasoenteric tube, so that the doctor can judge the position, shape, orientation and whether there are abnormalities of the nasoenteric tube according to the information in the double-track sonogram.

[0054] In an optional manner, the ultrasound synthesis neural network adopts a generative adversarial network, and the generative adversarial network includes a generator and a discriminator;

[0055] Among them, the generator is used to extract features of different scales through a multi-scale feature fusion module and fuse them to generate a waveform mask estimation result.

[0056] The discriminator is used to input the first ultrasonic image into the discriminator, so that the discriminator evaluates the matching degree between the waveform mask estimation result output by the generator and the real nasoenteric tube sonogram.

[0057] In this embodiment, through the adversarial training of the generator and the discriminator, the generator continuously optimizes its output to make the matching degree between the waveform mask estimation result and the real nasoenteric tube sonogram higher.

[0058] Specifically, the generator adopts a multi-scale feature fusion module for extracting and fusing features of different scales. The output of the generator is the waveform mask estimation result. The discriminator uses a convolutional neural network as the discriminator. Its input is the first ultrasonic image and the waveform mask estimation result (or the real nasoenteric tube sonogram) output by the generator, and the output is the evaluation result (i.e., the matching degree).

[0059] The generator loss is used to measure the difference between the waveform mask estimation result and the real nasoenteric tube sonogram, and the discriminator loss is used to measure the discriminator's ability to distinguish between real and generated samples. An adversarial loss is used as part of the generator loss to encourage the generator to generate a waveform mask estimation result similar to the real nasoenteric tube sonogram.

[0060] During training, a training data set including the first ultrasonic image and the corresponding real nasoenteric tube sonogram is used to alternately train the discriminator first so that it can accurately distinguish between real and generated samples, and then train the generator so that it can generate a more realistic waveform mask estimation result. In the inference stage, a new first ultrasonic image is input into the trained generative adversarial network. The generator outputs a waveform mask estimation result, combines the waveform mask estimation result with the first ultrasonic signal, and outputs a dual-track sonogram containing the nasoenteric tube through inverse short-time Fourier transform.

[0061] In an optional manner, the inverse short-time Fourier transform network uses a convolutional neural network to perform fusion processing on the waveform mask estimation result and the first ultrasonic signal to obtain a fused feature;

[0062] The inverse short-time Fourier transform network converts the fused feature into a segmented time-domain signal;

[0063] The segmented time-domain signal is processed according to the Overlap-Add method to obtain a dual-track sonogram containing the nasoenteric tube.

[0064] In this embodiment, a convolutional neural network is used to fuse the waveform mask estimation result and the first ultrasonic signal, which can more accurately capture the correlation information between the two. The inverse short-time Fourier transform is implemented through the convolutional neural network, improving the quality of signal reconstruction. The Overlap-Add method is used to process the segmented time-domain signal, effectively reducing the distortion and artifacts in the signal reconstruction process and improving the clarity of the final output dual-track sonogram.

[0065] Among them, the Overlap-Add method is a block convolution method, which can effectively calculate the discrete convolution of a very long signal x[n] and a FIR filter h[n]. By dividing the long signal into smaller segments, calculating the convolution result of each segment respectively, and then overlapping and adding the results, the convolution of the entire signal is achieved. In this embodiment, the segmented time-domain signal is processed by overlapping and adding through the Overlap-Add method, reducing the distortion and artifacts in the signal reconstruction process. Specifically, when implemented, the adjacent segmented time-domain signals are weighted and averaged in the overlapping part, and the segmented time-domain signal will be closer to the real dual-track sonogram of the nasointestinal tube.

[0066] In an optional manner, the generator gradually generates the waveform mask from low resolution to high resolution, where the output of each resolution stage is used as the input of the next stage, and the generation result is gradually refined.

[0067] In this embodiment, by gradually increasing the resolution, the generator gradually refines the generation result, from a rough contour to fine details, gradually approaching the sonogram characteristics of the real nasointestinal tube, avoiding the problems of detail loss or artifacts that occur when directly generating high-resolution images. In the low-resolution stage, the amount of data processed by the generator is small, so the consumption of computing resources is relatively low. As the resolution gradually increases, although the amount of calculation gradually increases, since a relatively accurate contour has been generated in the previous stage, the calculation in the subsequent stage can be carried out more efficiently. Gradual generation enables the generator to fully utilize the output of the previous stage as prior information at each resolution stage, thereby guiding the generation process of the current stage and helping to generate a more realistic and delicate waveform mask.

[0068] In the low-resolution stage, the generator first generates a low-resolution waveform mask, which is a relatively rough contour but has captured the basic position and shape of the nasointestinal tube. In the medium-resolution stage, using the output of the low-resolution stage as the input, the generator further generates a medium-resolution waveform mask, which is richer in details and can capture more nasointestinal tube characteristics. In the high-resolution stage, using the output of the medium-resolution stage as the input, the generator generates a high-resolution waveform mask, which is very fine in details and can basically accurately reflect the sonogram characteristics of the nasointestinal tube.

[0069] In an alternative manner, the step of inputting the waveform mask estimation result and the first ultrasonic signal into the inverse short-time Fourier transform network further includes:

[0070] Obtaining a restored time-domain signal according to the waveform mask estimation result and the first ultrasonic signal;

[0071] Inputting the restored time-domain signal into the inverse short-time Fourier transform network.

[0072] In this embodiment, while retaining the important features of the original signal, the restored time-domain signal reduces unnecessary noise and artifacts, and can identify and emphasize the boundaries, shapes, etc. of the nasointestinal tube in the ultrasonic signal.

[0073] In an alternative manner, the calculation formula for the restored time-domain signal is:

[0074]

[0075] where s(t) is the restored time-domain signal; STFT -1 is the inverse short-time Fourier transform; X[k] is the k-th component in the frequency domain; k and l are frequency components; t is a point on the time axis.

[0076] In this embodiment, through the inverse short-time Fourier transform (ISTFT) and the calculation formula for the restored time-domain signal, the frequency-domain signals X[k] and X[l] are converted back to the time domain to obtain the restored time-domain signal s(t).

[0077] In an alternative manner, the loss function for each resolution stage is:

[0078]

[0079] where L (k) is the loss function for the k-th resolution stage; D KL is the KL divergence; is the data distribution generated in the k-th resolution stage; P data (x) is the true data distribution; D(x) is the output of the discriminator; λ is the regularization coefficient; is the expectation of the random variable z, where z follows the distribution P z (z); P z (z) is the Gaussian probability distribution of the random variable z.

[0080] In this embodiment, the Kullback-Leibler (KL) divergence is used to measure the difference between the generated data distribution and the real data distribution. The smaller the KL divergence, the closer the generated data distribution is to the real data distribution. By introducing the regularization coefficient λ and the expected value of the discriminator output D(x), overfitting of the discriminator can be prevented. The random variable z follows a Gaussian probability distribution, and the generator can produce diverse outputs, increasing the randomness and diversity of the generated data.

[0081] In an alternative approach, the generator adopts a pyramid network structure;

[0082] where each layer of the pyramid network includes a feature extraction module, a feature fusion module, and an upsampling module for extracting, fusing, and upsampling features; and, each layer of the pyramid network constrains the estimated result of the generated waveform mask through a loss function to ensure the similarity between the generated waveform mask and the real sonogram of the nasointestinal tube.

[0083] In this embodiment, the pyramid network structure extracts multi-scale features from low-level to high-level through different levels of convolutional layers, which helps the generator capture the fine structures and global context information in the nasointestinal tube sonogram, thereby improving the accuracy of the waveform mask. Each layer of the pyramid network includes a feature fusion module to fuse features at different levels to fully utilize information at different scales. The upsampling module enables the generator to gradually convert the low-resolution feature map into a high-resolution waveform mask. The process of gradual upsampling helps the generator generate a more refined and realistic waveform mask while reducing artifacts and noise. Each layer of the pyramid network constrains the estimated result of the generated waveform mask through a loss function to ensure the similarity between the generated waveform mask and the real sonogram of the nasointestinal tube.

[0084] According to the solution provided by the present invention, the first ultrasound image and the first ultrasound signal of the nasointestinal tube inserted into the digestive tract through the nasal cavity are obtained in real time; multi-channel images of the first ultrasound image are extracted through multiple groups of residual networks to obtain multi-channel image features; multi-channel ultrasounds of the first ultrasound signal are extracted by combining the scale-invariant feature transform network and the U-NET neural network to obtain multi-channel ultrasound features; the multi-channel image features and the multi-channel ultrasound features are input into the ultrasound synthesis neural network for waveform mask estimation, and the waveform mask estimation result is output; the waveform mask estimation result and the first ultrasound signal are input into the inverse short-time Fourier transform network to output a double-track sonogram including the nasointestinal tube. The present invention extracts features of images and ultrasound signals from multiple dimensions and by fusing the waveform mask estimation result and the original ultrasound signal, can more accurately display the double-track sonogram including the nasointestinal tube.

[0085] Figure 5 The framework schematic diagram of the nasointestinal tube positioning device based on artificial intelligence according to the embodiment of the present invention is shown. The nasointestinal tube positioning device based on artificial intelligence includes:

[0086] An ultrasonic acquisition module 510 is configured to obtain a first ultrasonic image and a first ultrasonic signal of a nasointestinal tube inserted into the digestive tract through the nasal cavity in real time.

[0087] A feature extraction module 520 is configured to extract multi-channel images of the first ultrasonic image through multiple groups of residual networks to obtain multi-channel image features; and extract multi-channel ultrasounds of the first ultrasonic signal by combining a scale-invariant feature transform network and a U-NET neural network to obtain multi-channel ultrasound features.

[0088] An acoustic image output module 530 is configured to input the multi-channel image features and the multi-channel ultrasound features into an ultrasonic synthesis neural network for waveform mask estimation, and output a waveform mask estimation result; and input the waveform mask estimation result and the first ultrasonic signal into an inverse short-time Fourier transform network to output a dual-track acoustic image including the nasointestinal tube.

[0089] Figure 6 The structural schematic diagram of an embodiment of the computing device according to the present invention is shown. The specific implementation of the present invention does not limit the specific implementation of the computing device.

[0090] As Figure 6 shown, the computing device may include: a processor 602, a communications interface 604, a memory 606, and a communication bus 608.

[0091] Wherein: the processor 602, the communications interface 604, and the memory 606 communicate with each other through the communication bus 608. The communications interface 604 is configured to communicate with network elements of other devices such as clients or other servers. The processor 602 is configured to execute a program 610, and specifically may execute relevant steps in the above-mentioned embodiment of the method for positioning a nasointestinal tube based on artificial intelligence.

[0092] Specifically, the program 610 may include program codes, and the program codes include computer operation instructions.

[0093] The processor 602 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the computing device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0094] A memory 606 for storing a program 610. The memory 606 may include high-speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

[0095] According to the solution provided by the present invention, a first ultrasonic image and a first ultrasonic signal of a nasoenteric tube inserted into the digestive tract through the nasal cavity are obtained in real time; multi-channel images of the first ultrasonic image are extracted through multiple groups of residual networks to obtain multi-channel image features; multi-channel ultrasounds of the first ultrasonic signal are extracted by combining a scale-invariant feature transform network and a U-NET neural network to obtain multi-channel ultrasound features; the multi-channel image features and the multi-channel ultrasound features are input into an ultrasonic synthesis neural network for waveform mask estimation, and a waveform mask estimation result is output; the waveform mask estimation result and the first ultrasonic signal are input into an inverse short-time Fourier transform network, and a double-track sonogram including the nasoenteric tube is output. The present invention extracts features of images and ultrasonic signals from multiple dimensions and by fusing the waveform mask estimation result and the original ultrasonic signal, can more accurately display the double-track sonogram including the nasoenteric tube.

[0096] Those skilled in the art can understand that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from this embodiment. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) can be replaced by an alternative feature providing the same, equivalent or similar purpose. In addition, those skilled in the art can understand that although some of the embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments means within the scope of the present invention and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination. The present invention can be implemented by means of hardware including several different elements and by means of a properly programmed computer. In the unit claims listing several devices, several of these devices can be embodied by the same hardware item. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the execution order.

Claims

1. A nasointestinal tube positioning method based on artificial intelligence, characterized in that: include: Acquire in real time a first ultrasound image and a first ultrasound signal of a nasogastric tube inserted into the digestive tract through the nasal cavity; Extracting a multi-channel image of the first ultrasound image through multiple groups of residual networks to obtain multi-channel image features; extracting multi-channel ultrasound of the first ultrasound signal by combining a scale-invariant feature transform network and a U-NET neural network to obtain multi-channel ultrasound features; Inputting the multi-channel image features and the multi-channel ultrasound features into an ultrasound synthesis neural network for waveform mask estimation, and outputting a waveform mask estimation result; The waveform mask estimation result and the first ultrasound signal are input into an inverse short-time Fourier transform network, and a dual-track sound image containing a nasointestinal tube is output.

2. The method for locating a nasointestinal tube based on artificial intelligence according to claim 1, characterized in that: The ultrasonic synthesis neural network adopts a generative adversarial network, which includes a generator and a discriminator; The generator is used to extract and fuse features of different scales through a multi-scale feature fusion module to generate a waveform mask estimation result. The discriminator is used to input the first ultrasound image into the discriminator, so that the discriminator evaluates the matching degree between the waveform mask estimation result output by the generator and the real nasointestinal tube sound image.

3. The method for locating a nasointestinal tube based on artificial intelligence according to claim 1, characterized in that: The inverse short-time Fourier transform network uses a convolutional neural network to fuse the waveform mask estimation result and the first ultrasonic signal to obtain a fusion feature; The fusion feature is converted into a time domain signal by the inverse short-time Fourier transform network; The segmented time domain signal is processed according to Overlap-Add method to obtain a double-track sound image including the nasointestinal tube.

4. The method for locating a nasointestinal tube based on artificial intelligence according to claim 2, characterized in that: The generator generates waveform masks step by step from low resolution to high resolution, wherein the output of each resolution stage is used as the input of the next stage, and the result is generated by stepwise refinement.

5. The method for locating a nasointestinal tube based on artificial intelligence according to claim 1, characterized in that: The step of inputting the waveform mask estimation result and the first ultrasonic signal into an inverse short-time Fourier transform network further comprises: Obtaining a restored time domain signal according to the waveform mask estimation result and the first ultrasonic signal; The restored time domain signal is input into the inverse short-time Fourier transform network.

6. The artificial intelligence-based nasointestinal tube positioning method according to claim 5, characterized in that: The calculation formula of the restored time domain signal is: Among them, s(t) is the restored time domain signal; STFT -1 is the inverse short-time Fourier transform; X[k] is the kth component in the frequency domain; k and l are frequency components; t is a point on the time axis.

7. The artificial intelligence-based nasointestinal tube positioning method according to claim 4, characterized in that: The loss function of each resolution stage is: Among them, L (k) is the loss function of the kth resolution stage; D KL is the KL divergence; The data distribution generated for the kth resolution stage; P data (x) is the real data distribution; D(x) is the output of the discriminator; λ is the regularization coefficient; is the expectation of the random variable z, z follows the distribution P z (z); P z (z) is the Gaussian probability distribution of the random variable z.

8. The artificial intelligence-based nasointestinal tube positioning method according to claim 2 or 4, characterized in that: The generator adopts a pyramid network structure; Among them, each layer of the pyramid network contains a feature extraction module, a feature fusion module and an upsampling module, which are used to extract, fuse and upsample features; and each layer of the pyramid network constrains the generated waveform mask estimation result through a loss function to ensure the similarity between the generated waveform mask and the real nasointestinal tube sound image.

9. A nasointestinal tube positioning device based on artificial intelligence, characterized in that: include: An ultrasound acquisition module, used for acquiring in real time a first ultrasound image and a first ultrasound signal of a nasogastric tube inserted into the digestive tract through the nasal cavity; A feature extraction module, configured to extract a multi-channel image of the first ultrasound image through multiple groups of residual networks to obtain multi-channel image features; extract multi-channel ultrasound of the first ultrasound signal by combining a scale-invariant feature transformation network and a U-NET neural network to obtain multi-channel ultrasound features; An audio-visual output module, used for inputting the multi-channel image features and the multi-channel ultrasound features into an ultrasound synthesis neural network for waveform mask estimation, and outputting a waveform mask estimation result; The waveform mask estimation result and the first ultrasound signal are input into an inverse short-time Fourier transform network, and a dual-track sound image containing a nasointestinal tube is output.

10. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned artificial intelligence-based nasoenteric tube positioning method.

Images