Intraoperative pulmonary nodule localization system based on laparoscopic B-ultrasound images

Abstract

The present invention discloses an intraoperative lung nodule positioning system based on laparoscopic B-ultrasound images, and relates to the technical field of lung nodule positioning. In order to solve the problem that the existing technology has inaccurate positioning, complications such as hemothorax and pneumothorax, which restricts its effect in practical applications; the intraoperative lung nodule positioning system based on laparoscopic B-ultrasound images includes a laparoscope unit, a lung nodule positioning unit and an AI display unit; the laparoscope unit is used to obtain the intraoperative lung B-ultrasound image, the lung nodule positioning unit is used to identify and analyze the B-ultrasound image, the patient's respiratory movement is monitored, and the B-ultrasound image is dynamically compensated, thereby eliminating the influence of respiratory movement on the positioning of lung nodules, improving the accuracy and stability of positioning, determining the precise position of lung nodules and safe cutting margins, providing navigation and surgical plan suggestions for doctors through the AI ​​display unit, providing real-time and accurate lung nodule positioning and surgical navigation, which helps to improve the surgical effect and reduce the occurrence of complications.

Description

Technical Field

[0001] The present invention relates to the technical field of lung nodule positioning, and in particular to an intraoperative lung nodule positioning system based on laparoscope B-ultrasound images. Background Art

[0002] Due to the advantages of less trauma and faster recovery, thoracoscopic surgery has become an important means of diagnosis and treatment of small lung nodules. However, how to quickly and accurately locate lung nodules during surgery, how to remove tumors as accurately as possible, and how to protect lung function to the maximum extent have always been challenges faced by thoracic surgeons. Regarding the positioning of lung nodules in thoracoscopic surgery, a Chinese patent with publication number: CN111821034B discloses a magnetic anchoring lung nodule positioning device for thoracoscopic surgery, including: two target magnets, used to clamp the target nodules on both sides of the target nodules; two coaxial puncture needles, both of which are hollow structures, and the hollow dimensions are respectively larger than the external dimensions of the two target magnets, so as to allow the two target magnets to pass through respectively; a positioning plate with multiple holes, and two coaxial puncture needles penetrate into the two holes to achieve the preliminary positioning of the target nodules; an anchoring magnet, used to attract the target magnet on the surface of the lung through magnetic force, and confirm the positioning range according to the magnitude of the magnetic force; the patent completes the positioning of the nodule lesions through magnetic attraction.

[0003] Although the above patent can locate nodule lesions, traditional preoperative small lung nodule auxiliary positioning technologies, such as CT-guided percutaneous puncture-assisted positioning, bronchoscopic puncture-assisted positioning, CT virtual 3D-assisted positioning, etc., although have certain effects, there are also problems such as inaccurate positioning and many complications; for example, these methods may cause complications such as hemothorax and pneumothorax, which restricts their effectiveness in practical applications. Summary of the invention

[0004] The purpose of the present invention is to provide an intraoperative lung nodule positioning system based on laparoscopic B-ultrasound images. The lung nodule positioning unit identifies and analyzes the B-ultrasound images to determine the precise location and safe resection margin of the lung nodules. The AI ​​display unit provides navigation and surgical plan suggestions for doctors, which helps to improve surgical results and reduce the occurrence of complications, so as to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The intraoperative lung nodule localization system based on laparoscopic B-ultrasound images includes:

[0007] The laparoscope unit is used to integrate the B-ultrasound probe into the laparoscope system, obtain the intraoperative lung B-ultrasound image in real time based on the B-ultrasound device, and pre-process the obtained B-ultrasound image to obtain dynamic image data of the patient's lung structure;

[0008] Lung Nodule Localization Unit for:

[0009] Determine the patient's respiratory motion characteristics based on the dynamic image data of the patient's lung structure, perform respiratory motion compensation on the B-ultrasound image based on the respiratory motion characteristics to obtain a compensated lung structure, and detect and identify lung nodules in the B-ultrasound image;

[0010] constructing a three-dimensional lung model of the patient based on the identified lung nodule information, determining the location, size and relationship of the lung nodule with surrounding tissues through the three-dimensional lung model, and calculating a safe resection margin based on the location and size of the lung nodule;

[0011] AI display unit for:

[0012] The patient's surgical data, B-ultrasound images and analysis results are displayed in real time. A surgical plan is generated based on the location, size and lung structure of the patient's lung nodules to determine the location of the nodules and the extent of resection.

[0013] At the same time, the surgical progress is monitored in real time during the operation, and real-time navigation instructions are provided to the doctor based on the current position of the resection device.

[0014] Furthermore, the laparoscope unit comprises:

[0015] B-ultrasound imaging module, used for receiving and processing the sound wave signal from the B-ultrasound probe in real time, and converting the sound wave signal into a visual B-ultrasound image;

[0016] An image processing module is used to perform smoothing and image enhancement processing on the original B-ultrasound image obtained from the B-ultrasound imaging module, and perform image segmentation operations on the processed B-ultrasound image;

[0017] The lung contour positioning module is used to extract features from the segmented B-ultrasound image, identify and mark the boundaries of the lungs in the image based on the extracted lung features, obtain the lung contour, and obtain dynamic lung image data by real-time monitoring of changes in the lung contour.

[0018] Furthermore, the image processing module includes:

[0019] A first pixel block extraction module, used for extracting each pixel block of the original B-ultrasound image as a main pixel block;

[0020] A second pixel block extraction module, used to extract a plurality of pixel blocks in contact with the primary pixel block as auxiliary pixel blocks;

[0021] A pixel block unit acquisition module, used to take each of the primary pixel blocks and its corresponding secondary pixel blocks as a pixel block unit;

[0022] A first grayscale value extraction module, used for extracting the grayscale value of each main pixel block of the original B-ultrasound image;

[0023] A second grayscale value extraction module, used for extracting grayscale values ​​of a plurality of contacting auxiliary pixel blocks corresponding to each of the primary pixel blocks;

[0024] A grayscale reference value acquisition module is used to obtain a grayscale reference value corresponding to each pixel value block by using the grayscale value of each main pixel block of the original B-ultrasound image and the grayscale values ​​of a plurality of contacting auxiliary pixel blocks corresponding to each main pixel block; wherein the grayscale reference value is obtained by:

[0025]

[0026] Among them, R c represents the grayscale reference value; n represents the number of auxiliary pixel blocks; R i Represents the grayscale value of the i-th auxiliary pixel block; R z Represents the gray value of the main pixel block; r 01 and r 02 Respectively represent the first compensation adjustment coefficient and the second compensation adjustment coefficient;

[0027]

[0028] Among them, R p Represents the average grayscale value of the secondary pixel block; R max Indicates the maximum grayscale value in the secondary pixel block;

[0029] A grayscale value adjustment module is used to adjust the grayscale value of the main pixel block by using the grayscale reference value.

[0030] Furthermore, the gray value adjustment module includes:

[0031] A grayscale reference value retrieving module, used for retrieving the grayscale reference value;

[0032] A first grayscale difference acquisition module, configured to acquire a grayscale difference between the grayscale reference value and the main pixel block by using the grayscale reference value and the grayscale value of the main pixel block as a first grayscale difference;

[0033] A second grayscale difference acquisition module, configured to acquire a grayscale difference between the grayscale reference value and each auxiliary pixel block by using the grayscale reference value and the grayscale value of each auxiliary pixel block as a second grayscale difference;

[0034] A target grayscale value acquisition module is used to acquire a target grayscale value of the main pixel block using the first grayscale difference and the second grayscale difference; wherein the target grayscale value is acquired by the following formula:

[0035]

[0036] Among them, R m Represents the target grayscale value of the main pixel block; R si represents the second grayscale value difference corresponding to the i-th auxiliary pixel block; R sz Indicates a first grayscale difference value corresponding to the main pixel block;

[0037] The adjustment execution module is used to adjust the grayscale value of the main pixel block according to the target grayscale value of the main pixel block.

[0038] Furthermore, the image processing module further includes:

[0039] A gray value difference acquisition module is used to extract the gray value difference between each main pixel block and each auxiliary pixel block after the gray value adjustment is completed for all pixel blocks of the original B-ultrasound image;

[0040] A grayscale value difference comparison module is used to compare the grayscale value difference between each primary pixel block and each secondary pixel block after grayscale value adjustment with a preset difference threshold;

[0041] A gray value data information extraction module is used to extract the gray value of the auxiliary pixel block corresponding to the gray value difference exceeding a preset difference threshold;

[0042] A grayscale value compensation coefficient acquisition module is used to acquire the grayscale value compensation coefficient of the grayscale value of the auxiliary pixel block through the grayscale value data information of the auxiliary pixel block corresponding to the grayscale value difference exceeding the preset difference threshold; wherein the grayscale value compensation coefficient is acquired by the following formula:

[0043]

[0044] Among them, R m Indicates the degree compensation coefficient; R fx and R fh Respectively represent the grayscale value of the auxiliary pixel block before and after grayscale adjustment; R e Represents the grayscale value difference between the auxiliary pixel block and the main pixel block; R cy Indicates the preset difference threshold;

[0045] The grayscale value compensation adjustment module is used to perform grayscale value compensation adjustment on the grayscale value of the auxiliary pixel block using the grayscale value compensation coefficient, wherein the grayscale value of the auxiliary pixel block after the grayscale value compensation adjustment is obtained by the following formula:

[0046]

[0047] Among them, R t Represents the gray value of the auxiliary pixel block after gray value compensation adjustment; R zhThey respectively represent the grayscale values ​​of the main pixel blocks after the grayscale values ​​are adjusted.

[0048] Furthermore, the lung contour positioning module is specifically:

[0049] Determine the shape features of the patient's lungs based on the B-ultrasound image, calculate the gradient strength and direction of the pixel points in the B-ultrasound image based on the shape features, and thus determine the discrete edge points of the patient's lung edges;

[0050] Based on the discrete edge points, identifying each end point of the patient's lung contour line, including a starting point, an end point, and a turning point of the contour;

[0051] Search for the pixels adjacent to each endpoint and continue searching along the direction of high gradient until returning to the starting point, thus forming a complete lung contour line;

[0052] Determining the distance between endpoints based on each endpoint of the contour line, extracting characteristic parameters of the contour line, and performing smoothing on the extracted characteristic parameters of the contour line to obtain a complete and smooth lung contour line;

[0053] The lung contours at different time points are compared, and the dynamic functional changes of the patient's lungs are analyzed based on the comparison results to obtain dynamic image data of the patient's lung structure.

[0054] Furthermore, the pulmonary nodule localization unit comprises:

[0055] A respiratory motion compensation module, used to determine the patient's respiratory motion characteristics based on the dynamic image data of the patient's lung structure, determine the patient's breathing pattern based on the patient's respiratory motion characteristics, and correct and compensate the B-ultrasound image based on the patient's breathing pattern;

[0056] The pulmonary nodule recognition module is used to identify and extract nodules in the image using image processing and analysis algorithms, analyze the extracted features of pixel values, textures, and shapes in the image, and mark the location and range of the pulmonary nodules;

[0057] The pulmonary nodule analysis module is used to convert continuous two-dimensional B-ultrasound images into a three-dimensional model based on three-dimensional reconstruction technology. The three-dimensional model includes the location, size and relationship of the pulmonary nodule with surrounding tissues;

[0058] The lung nodule positioning module is used to calculate the coordinates and size of the lung nodule based on the three-dimensional model and the analysis results of the lung nodule, and to evaluate the important structures and importance levels of the surrounding tissues of the lung nodule based on the relationship with the surrounding tissues.

[0059] Furthermore, the lung nodule recognition module is specifically:

[0060] The threshold of the B-ultrasound image is determined by using the grayscale histogram of the image, the peak position of the grayscale histogram is selected as the preset threshold, the image is segmented into foreground and background based on the preset threshold according to the grayscale characteristics of the image, and the possible nodule area is preliminarily detected;

[0061] Extract the shape features of nodules in possible nodule areas, analyze the texture features of nodules, measure the size of nodules, and compare them with the preset threshold;

[0062] The extracted features are classified and integrated to generate a lung nodule set, in which overlapping nodules are removed and adjacent nodules are merged, and true lung nodules and false positive results are obtained after processing;

[0063] The information on the location, size and shape of the detected true pulmonary nodules is output in the form of graphics and text.

[0064] Furthermore, the lung nodule positioning module is specifically:

[0065] Determine a coordinate system based on the three-dimensional model constructed by the pulmonary nodule analysis module, and calculate the coordinates of the recognition result of the pulmonary nodule recognition module in the three-dimensional image based on the determined coordinate system;

[0066] Determine the major axis, minor axis and minimum axis of each pulmonary nodule in the recognition result, and construct a local coordinate system based on the major axis, minor axis and minimum axis of each pulmonary nodule, wherein the center of the pulmonary nodule is the origin, and the major axis, minor axis and minimum axis correspond to the x, y and z axes respectively;

[0067] The size of each pulmonary nodule was determined based on its local coordinate system, the number of pixels occupied by the pulmonary nodule was calculated, and the volume of the pulmonary nodule was determined by multiplying the spatial resolution of each pixel.

[0068] Furthermore, the AI ​​display unit comprises:

[0069] A plan acquisition module is used to acquire historical medical images and physiological data of patients who have actively uploaded the data, and generate a surgical plan for the patient based on the acquired historical medical images and physiological data and the important structures and important levels of the relationship between the tissues surrounding the lung nodules;

[0070] A real-time positioning module is used to obtain the coordinate data of the lung nodules from the lung nodule positioning module, monitor the position and status of the surgical equipment in real time, and transmit the obtained positioning data to the real-time navigation module in real time;

[0071] A real-time navigation module, used to receive the positioning data of the surgical device from the real-time positioning module, calculate the difference between the positioning data of the surgical device and the coordinate data of the lung nodule, and generate real-time navigation instructions;

[0072] Display module, used to display the patient's surgical data in real time, as well as real-time images obtained through B-ultrasound;

[0073] The intraoperative data recording and analysis module is used to record detailed data during the operation in real time, including the location, size, and shape of the lung nodules, as well as the doctor's surgical operation data;

[0074] The remote collaboration module is used to transmit intraoperative B-ultrasound images and nodule location data to the remote doctor's terminal in real time based on the Internet of Things, thereby realizing real-time data interaction between the remote doctor and the on-site doctor.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] The laparoscope unit is used to obtain intraoperative B-ultrasound images of the lungs and process the images. The lung nodule positioning unit is used to identify and analyze the B-ultrasound images, monitor the patient's respiratory movements, and dynamically compensate for the B-ultrasound images, thereby eliminating the impact of respiratory movements on lung nodule positioning, improving positioning accuracy and stability, and determining the precise location of lung nodules and safe resection margins. Doctors can perform more accurate nodule positioning and analysis in the three-dimensional model to obtain the patient's lung structure information. The AI ​​display unit provides doctors with navigation and surgical plan recommendations, providing real-time and accurate lung nodule positioning and surgical navigation, which helps to improve surgical results and reduce the occurrence of complications. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 This is a module diagram of the intraoperative lung nodule positioning system based on laparoscopic B-ultrasound images of the present invention. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] In order to solve the technical problems in the prior art, such as inaccurate positioning and many complications, which restrict the effect of the technology in practical application, please refer to Figure 1 , this embodiment provides the following technical solutions:

[0080] The intraoperative lung nodule localization system based on laparoscopic B-ultrasound images includes:

[0081] The laparoscope unit is used to integrate the B-ultrasound probe into the laparoscope system, obtain the intraoperative lung B-ultrasound image in real time based on the B-ultrasound device, and pre-process the obtained B-ultrasound image to obtain dynamic image data of the patient's lung structure;

[0082] Lung Nodule Localization Unit for:

[0083] Determine the patient's respiratory motion characteristics based on the dynamic image data of the patient's lung structure, perform respiratory motion compensation on the B-ultrasound image based on the respiratory motion characteristics to obtain a compensated lung structure, and detect and identify lung nodules in the B-ultrasound image;

[0084] constructing a three-dimensional lung model of the patient based on the identified lung nodule information, determining the location, size and relationship of the lung nodule with surrounding tissues through the three-dimensional lung model, and calculating a safe resection margin based on the location and size of the lung nodule;

[0085] AI display unit for:

[0086] The patient's surgical data, B-ultrasound images and analysis results are displayed in real time. A surgical plan is generated based on the location, size and lung structure of the patient's lung nodules to determine the location of the nodules and the extent of resection.

[0087] At the same time, the surgical progress is monitored in real time during the operation, and real-time navigation instructions are provided to the doctor based on the current position of the resection device to ensure that the resection device operated by the doctor can accurately reach the location of the lung nodule.

[0088] In this embodiment, intraoperative B-ultrasound is radiation-free and integrated into the laparoscope system. It can locate the position of lung nodules in real time and reflect it on the display. The laparoscope unit is used to obtain the intraoperative B-ultrasound image of the lungs and process the image. The lung nodule positioning unit is used to identify and analyze the B-ultrasound image, monitor the patient's respiratory movement, and dynamically compensate for the B-ultrasound image, thereby eliminating the influence of respiratory movement on lung nodule positioning, improving the accuracy and stability of positioning, and determining the precise position of lung nodules and safe resection margins. Doctors can perform more accurate nodule positioning and analysis in the three-dimensional model to obtain the patient's lung structure information. The AI ​​display unit provides doctors with navigation and surgical plan suggestions, provides real-time and accurate lung nodule positioning and surgical navigation, improves the accuracy and efficiency of surgery, helps to improve surgical results, and reduce the occurrence of complications.

[0089] In this embodiment, the cavity mirror unit includes:

[0090] B-ultrasound imaging module, used for receiving and processing the sound wave signal from the B-ultrasound probe in real time, and converting the sound wave signal into a visual B-ultrasound image;

[0091] The image processing module is used to perform smoothing and image enhancement processing on the original B-ultrasound image obtained from the B-ultrasound imaging module, and perform image segmentation on the processed B-ultrasound image to separate different areas in the image, such as lung tissue, gas, liquid, etc. According to the characteristics of the B-ultrasound image, the segmentation algorithm based on gray value, texture or shape features is used for segmentation;

[0092] The lung contour positioning module is used to extract features from the segmented B-ultrasound image, including extracting edge features, texture features, shape features, etc. of the lungs, identifying and marking the boundaries of the lungs in the image based on the extracted lung features, obtaining the lung contour, and acquiring dynamic lung image data by real-time monitoring of changes in the lung contour;

[0093] Specifically, the image processing module includes:

[0094] A first pixel block extraction module, used for extracting each pixel block of the original B-ultrasound image as a main pixel block;

[0095] A second pixel block extraction module, used to extract a plurality of pixel blocks in contact with the primary pixel block as auxiliary pixel blocks;

[0096] A pixel block unit acquisition module, used to take each of the primary pixel blocks and its corresponding secondary pixel blocks as a pixel block unit;

[0097] A first grayscale value extraction module, used for extracting the grayscale value of each main pixel block of the original B-ultrasound image;

[0098] A second grayscale value extraction module, used for extracting grayscale values ​​of a plurality of contacting auxiliary pixel blocks corresponding to each of the primary pixel blocks;

[0099] A grayscale reference value acquisition module is used to obtain a grayscale reference value corresponding to each pixel value block by using the grayscale value of each main pixel block of the original B-ultrasound image and the grayscale values ​​of a plurality of contacting auxiliary pixel blocks corresponding to each main pixel block; wherein the grayscale reference value is obtained by:

[0100]

[0101] Among them, R c represents the grayscale reference value; n represents the number of auxiliary pixel blocks; R i represents the gray value of the ith auxiliary pixel block; R z Represents the gray value of the main pixel block; r 01 and r 02 Respectively represent the first compensation adjustment coefficient and the second compensation adjustment coefficient;

[0102]

[0103] Among them, R p Represents the average grayscale value of the secondary pixel block; R max Indicates the maximum grayscale value in the secondary pixel block;

[0104] A grayscale value adjustment module is used to adjust the grayscale value of the main pixel block by using the grayscale reference value.

[0105] The technical effect of the above technical solution is that by extracting the primary pixel block and the auxiliary pixel blocks around it and combining them into a pixel block unit, the image processing module can process the local area of ​​the image more finely. This processing method helps to enhance the details in the image, especially when there is a grayscale difference between the primary pixel block and the auxiliary pixel block.

[0106] The grayscale value adjustment module can adjust the grayscale value of the main pixel block by using the grayscale reference value calculated by the grayscale values ​​of the main pixel block and the auxiliary pixel block. This adjustment can make the image more uniform and reduce the grayscale value abnormality caused by noise or signal interference.

[0107] By introducing the first compensation adjustment coefficient (r01) and the second compensation adjustment coefficient (r02), the technical solution allows the user or the system to adjust the calculation method of the grayscale reference value according to specific needs. This flexibility enables the image processing module to adapt to different B-ultrasound images and processing requirements, thereby optimizing image quality.

[0108] The introduction of the average grayscale value (Rp) and the maximum grayscale value (Rmax) of the auxiliary pixel block helps to reduce the influence of individual abnormal pixels (such as noise points) on the calculation of grayscale reference values. This processing method makes the image processing module more robust and can provide more accurate grayscale reference values ​​in the presence of noise.

[0109] The contrast of the B-ultrasound image after grayscale value adjustment may be enhanced because the grayscale value adjustment module can make targeted adjustments to the main pixel blocks according to the grayscale reference value, thereby highlighting the key information in the image.

[0110] In summary, this technical solution can effectively improve the quality of B-ultrasound images by finely processing local areas of B-ultrasound images and adjusting grayscale values ​​using grayscale reference values ​​and compensation adjustment coefficients, including enhancing image details, reducing noise impact, and improving image contrast, etc. This is of great significance for medical diagnosis and disease analysis.

[0111] Specifically, the gray value adjustment module includes:

[0112] A grayscale reference value retrieving module, used for retrieving the grayscale reference value;

[0113] A first grayscale difference acquisition module, configured to acquire a grayscale difference between the grayscale reference value and the main pixel block by using the grayscale reference value and the grayscale value of the main pixel block as a first grayscale difference;

[0114] A second grayscale difference acquisition module, configured to acquire a grayscale difference between the grayscale reference value and each auxiliary pixel block by using the grayscale reference value and the grayscale value of each auxiliary pixel block as a second grayscale difference;

[0115] A target grayscale value acquisition module is used to acquire a target grayscale value of the main pixel block using the first grayscale difference and the second grayscale difference; wherein the target grayscale value is acquired by the following formula:

[0116]

[0117] Among them, R m Represents the target grayscale value of the main pixel block; R si represents the second grayscale value difference corresponding to the i-th auxiliary pixel block; R sz Indicates a first grayscale difference value corresponding to the main pixel block;

[0118] The adjustment execution module is used to adjust the grayscale value of the main pixel block according to the target grayscale value of the main pixel block.

[0119] The technical effect of the above technical solution is: by calculating the grayscale difference between the grayscale reference value and the main pixel block and its auxiliary pixel block, the module can accurately identify the grayscale difference between each main pixel block in the image and its surrounding environment. Then, these grayscale differences are used to calculate the target grayscale value of the main pixel block, thereby achieving accurate adjustment of the grayscale value of the main pixel block. This adjustment helps to improve the contrast of the image, reduce local brightness unevenness, and thus improve the overall quality of the image.

[0120] Since the grayscale value adjustment is based on the grayscale difference between the main pixel block and its auxiliary pixel block, this module can enhance the details in the image. By adjusting the grayscale value of the main pixel block to make it more coordinated with the surrounding environment, key information in the image, such as lesions and blood vessels, can be highlighted, thereby improving the doctor's diagnostic accuracy of the image.

[0121] When calculating the grayscale difference, the module takes into account the grayscale values ​​of the auxiliary pixel blocks, which helps reduce the impact of noise on the grayscale value adjustment of the primary pixel blocks. Since the auxiliary pixel blocks usually contain similar image information as the primary pixel blocks, their grayscale values ​​can provide useful reference information to help the module more accurately identify noise points and reduce their impact.

[0122] The entire grayscale value adjustment process is automated without human intervention, which greatly improves the efficiency and accuracy of image processing and reduces the impact of human factors on image processing results.

[0123] In summary, the grayscale value adjustment module can significantly improve the quality of B-ultrasound images, enhance image details, reduce noise impact, and has a high degree of flexibility and automation by calculating grayscale differences and adjusting the grayscale values ​​of the main pixel blocks based on these differences. These technical effects have important application value in medical diagnosis, image analysis and subsequent processing.

[0124] Specifically, the image processing module also includes:

[0125] A gray value difference acquisition module is used to extract the gray value difference between each main pixel block and each auxiliary pixel block after the gray value adjustment is completed for all pixel blocks of the original B-ultrasound image;

[0126] A grayscale value difference comparison module is used to compare the grayscale value difference between each primary pixel block and each secondary pixel block after grayscale value adjustment with a preset difference threshold;

[0127] A gray value data information extraction module is used to extract the gray value of the auxiliary pixel block corresponding to the gray value difference exceeding a preset difference threshold;

[0128] A grayscale value compensation coefficient acquisition module is used to acquire the grayscale value compensation coefficient of the grayscale value of the auxiliary pixel block through the grayscale value data information of the auxiliary pixel block corresponding to the grayscale value difference exceeding the preset difference threshold; wherein the grayscale value compensation coefficient is acquired by the following formula:

[0129]

[0130] Among them, R m Indicates the degree compensation coefficient; R fx and R fh Respectively represent the grayscale value of the auxiliary pixel block before and after grayscale adjustment; R e Represents the grayscale value difference between the auxiliary pixel block and the main pixel block; R cy Indicates the preset difference threshold;

[0131] The grayscale value compensation adjustment module is used to perform grayscale value compensation adjustment on the grayscale value of the auxiliary pixel block using the grayscale value compensation coefficient, wherein the grayscale value of the auxiliary pixel block after the grayscale value compensation adjustment is obtained by the following formula:

[0132]

[0133] Among them, R t Represents the gray value of the auxiliary pixel block after gray value compensation adjustment; R zh They respectively represent the grayscale values ​​of the main pixel blocks after the grayscale values ​​are adjusted.

[0134] The technical effect of the above technical solution is: through the gray value difference acquisition module and the gray value difference comparison module, the system can identify the significant difference in gray value between the main pixel block and the auxiliary pixel block. These differences may be caused by noise in the image acquisition process, equipment errors or the characteristics of the image itself. Through subsequent processing, the system can reduce these differences, thereby enhancing the consistency of gray values ​​in the image. The gray value data information extraction module and the gray value compensation coefficient acquisition module work together to identify the auxiliary pixel block that needs to be adjusted and calculate the appropriate gray value compensation coefficient for it. The purpose of this step is to improve the overall quality of the image by adjusting the gray value of the auxiliary pixel block to make it closer to the gray value of the main pixel block.

[0135] The entire processing flow is automated and does not require human intervention. This makes the image processing process more efficient and faster, and reduces errors introduced by human factors. By setting different difference thresholds, users can adjust the degree of processing of grayscale value differences according to actual needs. This provides flexibility and customizability, allowing the technical solution to adapt to different scenarios and application requirements. The grayscale value compensation adjustment module uses the calculated grayscale value compensation coefficient to adjust the grayscale value of the auxiliary pixel block to optimize the visual effect of the image. This adjustment can reduce unevenness and artifacts in the image, making the image clearer and easier to interpret.

[0136] In summary, this technical solution realizes the adjustment of the grayscale value difference between the main pixel block and the auxiliary pixel block in the B-ultrasound image through a series of modular processing flows, thereby enhancing the consistency of the image grayscale value, improving the image quality, optimizing the visual effect, and has the characteristics of automation, flexibility and customizability.

[0137] In this embodiment, the lung contour positioning module is specifically:

[0138] Determine the shape features of the patient's lungs based on the B-ultrasound image, calculate the gradient strength and direction of the pixel points in the B-ultrasound image based on the shape features, and thus determine the discrete edge points of the patient's lung edges;

[0139] Based on the discrete edge points, identifying each end point of the patient's lung contour line, including a starting point, an end point, and a turning point of the contour;

[0140] Search for the pixels adjacent to each endpoint and continue searching along the direction of high gradient until returning to the starting point, thus forming a complete lung contour line;

[0141] Determining the distance between endpoints based on each endpoint of the contour line, extracting characteristic parameters of the contour line, and performing smoothing on the extracted characteristic parameters of the contour line to obtain a complete and smooth lung contour line;

[0142] The lung contours at different time points are compared, and the dynamic functional changes of the patient's lungs are analyzed based on the comparison results to obtain dynamic image data of the patient's lung structure.

[0143] In this embodiment, by receiving the sound wave signal from the B-ultrasound probe in real time and converting it into a visualized B-ultrasound image, the original B-ultrasound image is smoothed and enhanced to improve the image quality, and a segmentation algorithm based on grayscale value, texture or shape features is used to separate different areas in the image. The doctor can see the structure and boundary of the lungs more clearly, and by obtaining a complete and smooth lung contour line, comparing the lung contour lines at different time points, the dynamic functional changes of the lungs are analyzed, and the dynamic functional changes of the lungs can be monitored in real time. The doctor can better avoid damaging important surrounding structures during the operation, thereby reducing the occurrence of complications.

[0144] In this embodiment, the pulmonary nodule positioning unit includes:

[0145] A respiratory motion compensation module is used to determine the patient's respiratory motion characteristics based on the dynamic image data of the patient's lung structure, determine the patient's respiratory pattern, such as respiratory frequency, respiratory depth, etc., based on the patient's respiratory motion characteristics, and correct and compensate the B-ultrasound image based on the patient's respiratory pattern to eliminate the impact of respiratory motion on image quality, which helps to ensure the accuracy and reliability of subsequent image processing;

[0146] The pulmonary nodule recognition module is used to identify and extract nodules in images using image processing and analysis algorithms, analyze the extracted features of pixel values, textures, and shapes in the images, and mark the location and range of pulmonary nodules. Specifically:

[0147] The threshold of the B-ultrasound image is determined by using the grayscale histogram of the image, and the peak position of the grayscale histogram is selected as the preset threshold. The image is segmented into foreground and background based on the preset threshold according to the grayscale characteristics of the image. The foreground is the possible nodule and the background is the normal lung tissue. The possible nodule area is preliminarily detected, and morphological operations are applied to remove small noise points while maintaining the shape characteristics of the nodule.

[0148] Extract the shape features of nodules in possible nodule areas, such as circularity, compactness, edge smoothness, etc., analyze the texture features of nodules, such as gray-level co-occurrence matrix, local binary pattern, etc., measure the size of nodules, such as diameter, volume, etc., and compare them with preset thresholds;

[0149] The extracted features are classified and integrated to generate a lung nodule set, in which overlapping nodules are removed and adjacent nodules are merged, and true lung nodules and false positive results are obtained after processing;

[0150] Output the information of the position, size and shape of the detected true lung nodules in the form of graphics and text;

[0151] The pulmonary nodule analysis module is used to convert continuous two-dimensional B-ultrasound images into a three-dimensional model based on three-dimensional reconstruction technology. The three-dimensional model includes the location, size and relationship of the pulmonary nodule with surrounding tissues;

[0152] The lung nodule positioning module is used to calculate the coordinates and size of the lung nodule based on the three-dimensional model and the analysis results of the lung nodule, and to evaluate the important structures and important levels of the surrounding tissues of the lung nodule, such as blood vessels and bronchi, based on the relationship with the surrounding tissues. Specifically:

[0153] Determine a coordinate system based on the three-dimensional model constructed by the pulmonary nodule analysis module, and calculate the coordinates of the recognition result of the pulmonary nodule recognition module in the three-dimensional image based on the determined coordinate system;

[0154] Determine the major axis, minor axis and minimum axis of each pulmonary nodule in the recognition result, and construct a local coordinate system based on the major axis, minor axis and minimum axis of each pulmonary nodule, wherein the center of the pulmonary nodule is the origin, and the major axis, minor axis and minimum axis correspond to the x, y and z axes respectively;

[0155] The size of each pulmonary nodule was determined based on its local coordinate system, the number of pixels occupied by the pulmonary nodule was calculated, and the volume of the pulmonary nodule was determined by multiplying the spatial resolution of each pixel.

[0156] In this embodiment, through respiratory motion compensation and advanced image processing algorithms, lung nodules can be identified and located more accurately, the possibility of misjudgment and missed diagnosis can be reduced, and the relationship between lung nodules and surrounding tissues can be accurately assessed. Doctors can be more careful to avoid damaging important structures during surgery, thereby reducing the risk of complications. Through the construction of a three-dimensional model and a local coordinate system, the lung nodule positioning module can more accurately calculate the coordinates and size of the lung nodules, improve the accuracy of positioning, calculate the volume of the lung nodules and the number of pixels occupied, and provide doctors with quantitative analysis support, which helps to better evaluate the growth rate and treatment effect of lung nodules, help doctors determine the best surgical path and strategy, and help to formulate more accurate surgical plans.

[0157] In this embodiment, the AI ​​display unit includes:

[0158] The plan acquisition module is used to obtain the historical medical images and physiological data of patients who have actively uploaded them. Based on the acquired historical medical images and physiological data and the important structures and important levels of the tissues around the lung nodules, the surgical plan for the patient is generated, providing doctors with precise surgical guidance based on the patient's specific conditions.

[0159] A real-time positioning module is used to obtain the coordinate data of the lung nodules from the lung nodule positioning module, monitor the position and status of the surgical equipment in real time, and transmit the obtained positioning data to the real-time navigation module in real time;

[0160] A real-time navigation module is used to receive the positioning data of the surgical equipment from the real-time positioning module, calculate the difference between the positioning data of the surgical equipment and the coordinate data of the lung nodules, and generate real-time navigation instructions to ensure the accuracy and safety of the operation;

[0161] The display module is used to display the patient's surgical data in real time, including various physiological parameters and surgical progress information, as well as real-time images obtained through B-ultrasound, providing doctors with a real-time view of the patient's internal conditions;

[0162] The intraoperative data recording and analysis module is used to record detailed data during the operation in real time, including the location, size, and shape of the lung nodules, as well as the doctor's surgical operation data. By analyzing these data, it can provide doctors with a basis for postoperative evaluation and improvement of surgical methods;

[0163] The remote collaboration module is used to transmit intraoperative B-ultrasound images and nodule location data to the remote doctor's terminal in real time based on the Internet of Things, so as to achieve real-time data interaction between the remote doctor and the on-site doctor. The remote doctor can provide remote guidance and suggestions to the on-site doctor to improve the safety and effectiveness of the operation.

[0164] In this embodiment, a personalized surgical plan is generated by integrating the patient's historical data and lung nodule characteristics, thereby improving the accuracy and effectiveness of the surgery. The surgical progress is monitored in real time and navigation instructions are provided. Doctors can reach the location of the lung nodules more quickly, shortening the operation time and improving the surgical efficiency. The recording and analysis of intraoperative data provide doctors with a basis for postoperative evaluation and improvement of surgical methods, which helps to improve the professional level of doctors. Especially in the handling of complex cases, real-time data interaction and remote guidance between on-site doctors and remote doctors are achieved through the remote collaboration module, which improves the safety and effectiveness of the surgery, provides all-round support and guarantee for the surgical process, improves the safety and effectiveness of the surgery, and also provides doctors with important postoperative evaluation and improvement tools.

[0165] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. The intraoperative pulmonary nodule localization system based on laparoscopic B-ultrasound images is characterized by: include: The laparoscope unit is used to integrate the B-ultrasound probe into the laparoscope system, obtain the intraoperative lung B-ultrasound image in real time based on the B-ultrasound device, and pre-process the obtained B-ultrasound image to obtain dynamic image data of the patient's lung structure; Lung Nodule Localization Unit for: Determine the patient's respiratory motion characteristics based on the dynamic image data of the patient's lung structure, perform respiratory motion compensation on the B-ultrasound image based on the respiratory motion characteristics to obtain a compensated lung structure, and detect and identify lung nodules in the B-ultrasound image; constructing a three-dimensional lung model of the patient based on the identified lung nodule information, determining the location, size and relationship of the lung nodule with surrounding tissues through the three-dimensional lung model, and calculating a safe resection margin based on the location and size of the lung nodule; AI display unit for: The patient's surgical data, B-ultrasound images and analysis results are displayed in real time. A surgical plan is generated based on the location, size and lung structure of the patient's lung nodules to determine the location of the nodules and the extent of resection. At the same time, the surgical process is monitored in real time during the operation, and real-time navigation instructions are provided to the doctor based on the current position of the resection device; Laparoscopic unit, comprising: B-ultrasound imaging module, used for receiving and processing the sound wave signal from the B-ultrasound probe in real time, and converting the sound wave signal into a visual B-ultrasound image; An image processing module is used to perform smoothing and image enhancement processing on the original B-ultrasound image obtained from the B-ultrasound imaging module, and perform image segmentation operations on the processed B-ultrasound image; The lung contour positioning module is used to extract features from the segmented B-ultrasound image, identify and mark the lung boundary in the image based on the extracted lung features, obtain the lung contour, and obtain dynamic lung image data by real-time monitoring of changes in the lung contour; Image processing module, including: A first pixel block extraction module, used for extracting each pixel block of the original B-ultrasound image as a main pixel block; A second pixel block extraction module, used to extract a plurality of pixel blocks in contact with the primary pixel block as auxiliary pixel blocks; A pixel block unit acquisition module, used to take each of the primary pixel blocks and its corresponding secondary pixel blocks as a pixel block unit; A first grayscale value extraction module, used for extracting the grayscale value of each main pixel block of the original B-ultrasound image; A second grayscale value extraction module, used for extracting grayscale values ​​of a plurality of contacting auxiliary pixel blocks corresponding to each of the primary pixel blocks; A grayscale reference value acquisition module is used to acquire a grayscale reference value corresponding to each pixel value block by using the grayscale value of each main pixel block of the original B-ultrasound image and the grayscale values ​​of a plurality of contacting auxiliary pixel blocks corresponding to each main pixel block; wherein the grayscale reference value is acquired by: Among them, R c represents the grayscale reference value; n represents the number of auxiliary pixel blocks; R i Represents the grayscale value of the i-th auxiliary pixel block; R z Represents the gray value of the main pixel block; r 01 and r 02 Respectively represent the first compensation adjustment coefficient and the second compensation adjustment coefficient; Among them, R p Represents the average grayscale value of the secondary pixel block; R max Indicates the maximum grayscale value in the secondary pixel block; A grayscale value adjustment module, used for adjusting the grayscale value of the main pixel block by using the grayscale reference value; Gray value adjustment module, including: A grayscale reference value retrieving module, used for retrieving the grayscale reference value; A first grayscale difference acquisition module, configured to acquire a grayscale difference between the grayscale reference value and the main pixel block by using the grayscale reference value and the grayscale value of the main pixel block as a first grayscale difference; A second grayscale difference acquisition module, configured to acquire a grayscale difference between the grayscale reference value and each auxiliary pixel block by using the grayscale reference value and the grayscale value of each auxiliary pixel block as a second grayscale difference; A target grayscale value acquisition module is used to acquire a target grayscale value of the main pixel block using the first grayscale difference and the second grayscale difference; wherein the target grayscale value is acquired by the following formula: Among them, R m Represents the target grayscale value of the main pixel block; R si represents the second grayscale value difference corresponding to the i-th auxiliary pixel block; R sz Indicates a first grayscale difference value corresponding to the main pixel block; An adjustment execution module, used for adjusting the grayscale value of the main pixel block according to the target grayscale value of the main pixel block; Lung contour positioning module, specifically: Determine the shape features of the patient's lungs based on the B-ultrasound image, calculate the gradient strength and direction of the pixel points in the B-ultrasound image based on the shape features, and thus determine the discrete edge points of the patient's lung edges; Based on the discrete edge points, identifying each end point of the patient's lung contour line, including a starting point, an end point, and a turning point of the contour; Search for the pixels adjacent to each endpoint and continue searching along the direction of high gradient until returning to the starting point, thus forming a complete lung contour line; Determining the distance between endpoints based on each endpoint of the contour line, extracting characteristic parameters of the contour line, and performing smoothing on the extracted characteristic parameters of the contour line to obtain a complete and smooth lung contour line; The lung contours at different time points are compared, and the dynamic functional changes of the patient's lungs are analyzed based on the comparison results to obtain dynamic image data of the patient's lung structure.

2. The intraoperative pulmonary nodule localization system based on laparoscopic B-ultrasound images according to claim 1, characterized in that: The image processing module also includes: A gray value difference acquisition module is used to extract the gray value difference between each main pixel block and each auxiliary pixel block after the gray value adjustment is completed for all pixel blocks of the original B-ultrasound image; A grayscale value difference comparison module is used to compare the grayscale value difference between each primary pixel block and each secondary pixel block after grayscale value adjustment with a preset difference threshold; A gray value data information extraction module is used to extract the gray value of the auxiliary pixel block corresponding to the gray value difference exceeding a preset difference threshold; A grayscale value compensation coefficient acquisition module is used to acquire the grayscale value compensation coefficient of the grayscale value of the auxiliary pixel block through the grayscale value data information of the auxiliary pixel block corresponding to the grayscale value difference exceeding the preset difference threshold; wherein the grayscale value compensation coefficient is acquired by the following formula: Among them, R m Indicates the degree compensation coefficient; R fx and R fh Respectively represent the grayscale value of the auxiliary pixel block before and after grayscale adjustment; R e Represents the grayscale value difference between the auxiliary pixel block and the main pixel block; R cy Indicates the preset difference threshold; The grayscale value compensation adjustment module is used to perform grayscale value compensation adjustment on the grayscale value of the auxiliary pixel block using the grayscale value compensation coefficient, wherein the grayscale value of the auxiliary pixel block after the grayscale value compensation adjustment is obtained by the following formula: Among them, R t Represents the gray value of the auxiliary pixel block after gray value compensation adjustment; R zh They respectively represent the grayscale values ​​of the main pixel blocks after the grayscale values ​​are adjusted.

3. The intraoperative pulmonary nodule localization system based on laparoscopic B-ultrasound images as claimed in claim 2, characterized in that: Pulmonary nodule localization unit, including: A respiratory motion compensation module, used to determine the patient's respiratory motion characteristics based on the dynamic image data of the patient's lung structure, determine the patient's breathing pattern based on the patient's respiratory motion characteristics, and correct and compensate the B-ultrasound image based on the patient's breathing pattern; The pulmonary nodule recognition module is used to identify and extract nodules in the image using image processing and analysis algorithms, analyze the extracted features of pixel values, textures, and shapes in the image, and mark the location and range of the pulmonary nodules; The pulmonary nodule analysis module is used to convert continuous two-dimensional B-ultrasound images into a three-dimensional model based on three-dimensional reconstruction technology. The three-dimensional model includes the location, size and relationship of the pulmonary nodule with surrounding tissues; The lung nodule positioning module is used to calculate the coordinates and size of the lung nodule based on the three-dimensional model and the analysis results of the lung nodule, and to evaluate the important structure and importance level of the surrounding tissue of the lung nodule based on the relationship with the surrounding tissue.

4. The intraoperative pulmonary nodule localization system based on laparoscopic B-ultrasound images as claimed in claim 3, characterized in that: Pulmonary nodule recognition module, specifically: The threshold of the B-ultrasound image is determined by using the grayscale histogram of the image, the peak position of the grayscale histogram is selected as the preset threshold, the image is segmented into foreground and background based on the preset threshold according to the grayscale characteristics of the image, and the possible nodule area is preliminarily detected; Extract the shape features of nodules in possible nodule areas, analyze the texture features of nodules, measure the size of nodules, and compare them with the preset threshold; The extracted features are classified and integrated to generate a lung nodule set, in which overlapping nodules are removed and adjacent nodules are merged, and true lung nodules and false positive results are obtained after processing; The information on the location, size and shape of the detected true pulmonary nodules is output in the form of graphics and text.

5. The intraoperative pulmonary nodule localization system based on laparoscopic B-ultrasound images as claimed in claim 3, characterized in that: Pulmonary nodule localization module, specifically: Determine a coordinate system based on the three-dimensional model constructed by the pulmonary nodule analysis module, and calculate the coordinates of the recognition result of the pulmonary nodule recognition module in the three-dimensional image based on the determined coordinate system; Determine the major axis, minor axis and minimum axis of each pulmonary nodule in the recognition result, and construct a local coordinate system based on the major axis, minor axis and minimum axis of each pulmonary nodule, where the center of the pulmonary nodule is the origin, and the major axis, minor axis and minimum axis correspond to the x, y and z axes respectively; The size of each pulmonary nodule was determined based on its local coordinate system, the number of pixels occupied by the pulmonary nodule was calculated, and the volume of the pulmonary nodule was determined by multiplying the spatial resolution of each pixel.

6. The intraoperative pulmonary nodule localization system based on laparoscopic B-ultrasound images according to claim 2, characterized in that: AI display unit, including: A plan acquisition module is used to acquire historical medical images and physiological data of patients who have actively uploaded the data, and generate a surgical plan for the patient based on the acquired historical medical images and physiological data and the important structures and important levels of the relationship between the tissues surrounding the lung nodules; A real-time positioning module is used to obtain the coordinate data of the lung nodules from the lung nodule positioning module, monitor the position and status of the surgical equipment in real time, and transmit the obtained positioning data to the real-time navigation module in real time; A real-time navigation module, used to receive the positioning data of the surgical device from the real-time positioning module, calculate the difference between the positioning data of the surgical device and the coordinate data of the lung nodule, and generate real-time navigation instructions; The display module is used to display the patient's surgical data in real time, as well as real-time images obtained through B-ultrasound.

7. The intraoperative pulmonary nodule localization system based on laparoscopic B-ultrasound images according to claim 6, characterized in that: The AI ​​display unit also includes: The intraoperative data recording and analysis module is used to record detailed data during the operation in real time, including the location, size, and shape of the lung nodules, as well as the doctor's surgical operation data; The remote collaboration module is used to transmit intraoperative B-ultrasound images and nodule location data to the remote doctor's terminal in real time based on the Internet of Things, thereby realizing real-time data interaction between the remote doctor and the on-site doctor.

Images