Sparse point cloud judgment method, device, equipment and medium based on fluctuation factor

Abstract

The present invention relates to the technical field of endoscopic image processing, and specifically discloses a method, device, equipment and medium for judging sparse point clouds based on fluctuation factors, wherein the method comprises the following steps: obtaining a point cloud data set corresponding to an endoscopic image, and obtaining a local point cloud set in a target area according to the point cloud data set; obtaining point cloud density information of the local point cloud set, and preliminarily judging whether the point cloud in the target area is dense according to the point cloud density information and a point cloud density threshold; when it is preliminarily judged to be dense, respectively calculating and obtaining fluctuation factors of the point cloud in the local point cloud set in the x, y and z directions, and finally judging whether the point cloud in the target area is dense according to the maximum fluctuation factor and the minimum fluctuation factor; the method introduces fluctuation factors to perform directional sparse analysis on the target area preliminarily judged to be dense, so as to more accurately judge the density of the endoscopic image point cloud, and provide more reliable basic data for subsequent three-dimensional reconstruction and surgical navigation.

Description

Technical Field

[0001] The present application relates to the technical field of endoscopic image processing, and in particular to a method, device, equipment and medium for determining a sparse point cloud based on a fluctuation factor. Background Art

[0002] Point cloud technology, as a 3D digital expression of physical objects, has shown great value in the field of endoscopic medicine. Millions of measurement points containing position and depth information provide unprecedented precise navigation and evaluation capabilities for endoscopic surgery. In endoscopic surgery, doctors can accurately plan the surgical path by combining point cloud models with real-time images, ensure the correct operation of surgical instruments, and improve the safety and success rate of surgery.

[0003] However, the sparsity of point clouds becomes a major challenge, especially when the endoscope field of view is limited, objects are blocked or reflected, etc., the reduction of sampling points leads to distortion of depth information. This requires us to effectively identify and process sparse areas when processing point cloud data to avoid misleading surgical operations. Traditional sparse area recognition methods are based on point cloud density and use fixed thresholds or average density judgments, but these methods are difficult to adapt to complex and changing scenes, and it is difficult to accurately distinguish between sparse and dense areas.

[0004] There is currently no effective technical solution to the above problems. Summary of the invention

[0005] The purpose of this application is to provide a sparse point cloud judgment method, device, equipment and medium based on fluctuation factors, so as to more accurately judge the density of the endoscopic image point cloud, give a final density assessment, accurately distinguish between sparse and dense areas, and provide more reliable basic data for subsequent three-dimensional reconstruction and surgical navigation.

[0006] In a first aspect, the present application provides a sparse point cloud judgment method based on a fluctuation factor, which is applied in point cloud reconstruction of an endoscopic image, and the method comprises the following steps:

[0007] Acquire a point cloud data set corresponding to the endoscopic image, and acquire a local point cloud set within the target area based on the point cloud data set;

[0008] Obtaining point cloud density information of the local point cloud set, and preliminarily determining whether the point cloud in the target area is dense based on the relationship between the point cloud density information and the point cloud density threshold;

[0009] When it is initially judged to be dense, the fluctuation factors of the point clouds in the local point cloud set in the x, y and z directions are calculated respectively, and the density of the point cloud in the target area is finally determined according to the proportional relationship between the largest fluctuation factor and the smallest fluctuation factor. The fluctuation factor is the mean-like variance of each point cloud in the target area and the central point cloud in the corresponding direction.

[0010] The sparse point cloud judgment method based on fluctuation factors of the present application performs a secondary sparsity analysis on the target area to ultimately determine whether the point cloud of the target area appears to be dense as a whole, analyzes the point cloud density information to quickly and preliminarily exclude areas that are obviously sparse, and introduces fluctuation factors to perform directional sparse analysis on the target area that is preliminarily judged to be dense, so as to more accurately judge the density of the endoscopic image point cloud, and give a final density assessment, which provides more reliable basic data for subsequent three-dimensional reconstruction and surgical navigation, and helps to improve the accuracy and safety of endoscopic surgery.

[0011] In the sparse point cloud judgment method based on fluctuation factors, the step of obtaining a point cloud data set corresponding to the endoscopic image comprises:

[0012] Acquire an endoscopic video frame, and correct the endoscopic video frame based on camera parameters to acquire an endoscopic image;

[0013] Performing stereoscopic imaging calculation on the endoscopic image to obtain a disparity map;

[0014] The point cloud data set is acquired according to the disparity map and the camera parameters.

[0015] In this example, the sparse point cloud judgment method based on fluctuation factors of the present application is combined with camera parameter correction and stereo imaging technology. This method can better handle these special cases, improve the accuracy and reliability of point cloud data, and provide high-quality data input for subsequent sparse point cloud judgment.

[0016] The method for determining sparse point cloud based on fluctuation factors, wherein the step of acquiring the point cloud data set according to the disparity map and the camera parameters comprises:

[0017] Acquire a mapping image according to the disparity map and a projection matrix, wherein the projection matrix is ​​acquired based on the camera parameters;

[0018] The map is reshaped and invalid data is deleted to obtain the point cloud dataset.

[0019] The sparse point cloud judgment method based on fluctuation factors of the present application obtains a point cloud data set based on the above steps, which not only improves the accuracy of the point cloud, but also reduces the computational burden of subsequent processing.

[0020] In the sparse point cloud judgment method based on fluctuation factors, the target area is a circular, rectangular or square area.

[0021] The sparse point cloud judgment method based on fluctuation factor, wherein the fluctuation factor is calculated based on the following formula:

[0022] ;

[0023] Among them, Ω is the target area mark, k is the direction mark, j is the center point cloud mark of the target area, and α j,k is the fluctuation factor about the center point cloud j in the k direction, Point i,k is the coordinate of the i-th point cloud in the target area in the k direction, Point j,k is the coordinate of the center point cloud j in the target area in the k direction, and N is the total number of point clouds or pixels in the target area.

[0024] In the method for determining sparse point clouds based on fluctuation factors, the step of finally determining whether the point cloud in the target area is dense based on the proportional relationship between the maximum fluctuation factor and the minimum fluctuation factor comprises:

[0025] Screen the largest and smallest fluctuation factors in the target area;

[0026] Calculate the volatility ratio between the maximum volatility factor and the minimum volatility factor;

[0027] The fluctuation ratio is compared with a preset ratio threshold to finally determine whether the point cloud in the target area is dense.

[0028] In the sparse point cloud judgment method based on fluctuation factors, the target area is determined based on the sliding window selection, or is generated based on the pixels corresponding to the central point cloud selected by the user and based on a preset area size.

[0029] In a second aspect, the present application further provides a sparse point cloud judgment device based on a fluctuation factor, which is applied in point cloud reconstruction of an endoscopic image, and the device comprises:

[0030] A point cloud acquisition module, used to acquire a point cloud data set corresponding to the endoscopic image, and to acquire a local point cloud set within a target area based on the point cloud data set;

[0031] A preliminary judgment module is used to obtain point cloud density information of a local point cloud set, and preliminarily judge whether the point cloud in the target area is dense according to the relationship between the point cloud density information and the point cloud density threshold;

[0032] The final judgment module is used to calculate the fluctuation factors of the point clouds in the local point cloud set in the x, y and z directions respectively when it is initially judged to be dense, and finally judge whether the point cloud in the target area is dense according to the proportional relationship between the largest fluctuation factor and the smallest fluctuation factor. The fluctuation factor is the mean-like variance of each point cloud in the target area and the central point cloud in the corresponding direction.

[0033] The sparse point cloud judgment device based on fluctuation factors of the present application performs a secondary sparsity analysis on the target area to ultimately determine whether the point cloud of the target area appears to be overall dense, analyzes the point cloud density information to quickly and preliminarily exclude areas that are obviously sparse, and introduces fluctuation factors to perform directional sparse analysis on the target area that is preliminarily judged to be dense, so as to more accurately judge the density of the endoscopic image point cloud, and give a final density assessment, which provides more reliable basic data for subsequent three-dimensional reconstruction and surgical navigation, and helps to improve the accuracy and safety of endoscopic surgery.

[0034] In a third aspect, the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are executed.

[0035] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps in the method provided in the first aspect are executed.

[0036] From the above, it can be seen that the present application provides a sparse point cloud judgment method, device, equipment and medium based on fluctuation factors, wherein the sparse point cloud judgment method based on fluctuation factors of the present application performs a secondary sparsity analysis on the target area to finally determine whether the point cloud of the target area appears to be overall dense, analyzes the point cloud density information to quickly and preliminarily exclude obviously sparse areas, and introduces fluctuation factors to perform directional sparse analysis on the target area preliminarily judged to be dense, so as to more accurately judge the density of the endoscopic image point cloud, and give a final density assessment, which provides more reliable basic data for subsequent three-dimensional reconstruction and surgical navigation, and helps to improve the accuracy and safety of endoscopic surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart of a sparse point cloud determination method based on fluctuation factors provided in an embodiment of the present application.

[0038] Figure 2 A schematic diagram of an endoscope image.

[0039] Figure 3 For the corresponding Figure 2 The disparity map of .

[0040] Figure 4 For the corresponding Figure 2 Schematic diagram of the 3D point cloud after visualization.

[0041] Figure 5 A schematic diagram of the structure of a sparse point cloud judgment device based on fluctuation factors provided in an embodiment of the present application.

[0042] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0043] Figure numerals: 201, point cloud acquisition module; 202, preliminary judgment module; 203, final judgment module; 301, processor; 302, memory; 303, communication bus. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0045] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0046] First, please refer to Figure 1-Figure 4 Some embodiments of the present application provide a sparse point cloud judgment method based on fluctuation factors, which is applied in point cloud reconstruction of endoscopic images. The method includes the following steps:

[0047] S1. Obtain a point cloud data set corresponding to the endoscopic image, and obtain a local point cloud set within the target area based on the point cloud data set;

[0048] S2, obtaining the point cloud density information of the local point cloud set, and preliminarily judging whether the point cloud in the target area is dense according to the relationship between the point cloud density information and the point cloud density threshold;

[0049] S3. When it is initially judged to be dense, the fluctuation factors of the point clouds in the local point cloud set in the x, y and z directions are calculated respectively. According to the proportional relationship between the largest fluctuation factor and the smallest fluctuation factor, it is finally judged whether the point cloud in the target area is dense. The fluctuation factor is the mean-like variance of each point cloud in the target area and the central point cloud in the corresponding direction.

[0050] Specifically, the sparse point cloud judgment method based on the fluctuation factor in the embodiment of the present application is mainly used in the three-dimensional reconstruction of 3D endoscopes. The three-dimensional reconstruction is generally reconstructed based on the three-dimensional point cloud. The sparse point cloud judgment method based on the fluctuation factor in the embodiment of the present application is used to analyze whether the point cloud data belongs to a sparse point cloud or to analyze whether there is a sparse point cloud in the target area, so as to avoid the inaccuracy of the depth information of the sparse point cloud affecting the three-dimensional reconstruction, and to avoid the key feature points of the three-dimensional reconstruction being located in the sparse point cloud.

[0051] More specifically, the endoscopic image is a two-dimensional image extracted from the video stream captured by a 3D endoscope. The 3D endoscope can also obtain the depth map corresponding to the two-dimensional image. Combining the two for stereo matching can obtain a three-dimensional point cloud that matches each pixel in the endoscopic image, thereby forming a point cloud data set.

[0052] More specifically, the target area is the area that needs to be analyzed whether there is a sparse point cloud. It can be an area selected by the user to be analyzed, or the system can automatically generate the selected area. The target area can be an area defined on the endoscopic image, or an area defined on the mapping diagram obtained below. After determining the target area, the corresponding three-dimensional point cloud can be extracted from the point cloud data set according to the matching relationship between the pixels in the target area and the point cloud to obtain a local point cloud set.

[0053] More specifically, the sparse point cloud judgment method based on the fluctuation factor of the embodiment of the present application divides the sparse judgment into two parts. The first part mainly relies on the traditional sparse judgment algorithm to perform an overall sparse characteristic analysis on the three-dimensional point cloud in the target area. The second part performs a directional sparse characteristic analysis on the three-dimensional point cloud in the target area after the overall analysis is passed. Among them, the first part uses point cloud density information for judgment. The point cloud density information can be calculated based on the k-nearest neighbor algorithm, or it can be determined based on the ratio of the number of point clouds in the target area and the size of the target area. It can directly reflect the density of the three-dimensional point cloud distribution in the target area, so it can be directly compared with the point cloud density threshold to perform a simple sparse characteristic analysis, that is, to preliminarily judge whether the three-dimensional point cloud in the target area has sparse characteristics from the overall distribution situation. Among them, if the point cloud density information is greater than or equal to the point cloud density threshold, it preliminarily represents that the point cloud in the target area is dense, and if the point cloud density information is less than the point cloud density threshold, it represents that the point cloud in the target area is sparse. Figure 1 As shown, it can be considered that the target area is sparse as a whole and cannot be directly used for 3D reconstruction of key feature points.

[0054] More specifically, the point cloud density threshold can be an empirical value preset by the user or a system-generated value, wherein the latter can be the overall density distribution information calculated based on the ratio of the number of point clouds in the point cloud data set and the size of the endoscopic image, or can be obtained by multiplying the overall density distribution information by a preset percentage value (such as 90%).

[0055] More specifically, the fluctuation factor is defined as the mean-like variance of each point cloud in the target area and the central point cloud in the corresponding direction. The sparse point cloud judgment method based on the fluctuation factor in the embodiment of the present application can effectively capture the spatial distribution characteristics of the point cloud by calculating the fluctuation factors in different directions.

[0056] More specifically, when it is determined that the target area is not overall sparse, it is necessary to analyze whether the target area has a local sparse problem. If the target area is further split and analyzed, the computational load will be greatly increased. Therefore, the sparse point cloud judgment method based on the fluctuation factor in the embodiment of the present application introduces the fluctuation factor for directional sparse analysis. Since the three-dimensional point cloud has three-dimensional coordinates, step S3 can analyze the fluctuation of the point cloud in the target area in three directions according to the point cloud in the local point cloud concentration. Since the fluctuation factor is the mean-like variance of each point cloud in the target area and the central point cloud in the corresponding direction, it can directly reflect the overall change of all point clouds in the target area in the corresponding direction. If there is an offset in the data of some point clouds in the corresponding direction, it will cause the part of the point cloud to show sparse characteristics relative to the nearby point clouds. These offsets are generally caused by inaccurate depth information of the corresponding pixels (manifested as a sharp change), resulting in the doubling of the distance between these point clouds and the central point cloud, and then the corresponding fluctuation factor is doubling.

[0057] It should be noted that since the endoscope collects images of the human body, the point cloud data set will not show planar characteristics, and the fluctuation factors in different directions will not be close to 0.

[0058] More specifically, when the point cloud in the target area shows directional sparse characteristics, the fluctuation factor in a certain direction is doubled, causing the fluctuation factor to appear to be much larger than the other fluctuation factors. Therefore, some embodiments can determine whether there is directional sparsity by judging the size of the maximum fluctuation factor through a preset size threshold. However, considering that the image pixels or target area sizes of the endoscope taken under different endoscopic parameters are different, directional sparse analysis directly based on the preset size threshold may not be able to accurately judge. Therefore, the sparse point cloud judgment method based on the fluctuation factor in the embodiment of the present application finally judges whether there is directional sparsity in the point cloud in the target area, that is, whether it is dense, based on the proportional relationship between the largest fluctuation factor and the smallest fluctuation factor, so as to improve the accuracy of the judgment algorithm and expand the scope of application; it is worth mentioning that the larger the proportional relationship, the more serious the directional sparsity.

[0059] It should be noted that the sparse point cloud judgment method based on fluctuation factors in the embodiment of the present application can be used to determine whether the target area belongs to a sparse area, can be used to determine whether there is a sparse point cloud in the target area, and can also be used to determine whether the central point cloud corresponds to a sparse feature point.

[0060] More specifically, the central point cloud is a point cloud at the center of the target area, that is, a point cloud corresponding to pixels at the center of the target area in the endoscopic image or mapping image.

[0061] The sparse point cloud judgment method based on fluctuation factors in the embodiment of the present application performs a secondary sparsity analysis on the target area to ultimately determine whether the point cloud of the target area appears to be overall dense, analyzes the point cloud density information to quickly and preliminarily exclude areas that are obviously sparse, and introduces fluctuation factors to perform directional sparse analysis on the target area that is preliminarily judged to be dense, so as to more accurately judge the density of the endoscopic image point cloud, and provide a final density assessment, which provides more reliable basic data for subsequent three-dimensional reconstruction and surgical navigation, and helps to improve the accuracy and safety of endoscopic surgery.

[0062] In some preferred embodiments, the step of obtaining a point cloud data set corresponding to the endoscopic image includes:

[0063] Acquire an endoscopic video frame, and correct the endoscopic video frame based on camera parameters to acquire an endoscopic image;

[0064] Perform stereo imaging calculation on the endoscopic image to obtain a disparity map;

[0065] Get the point cloud dataset based on the disparity map and camera parameters.

[0066] Specifically, the endoscope continuously captures and acquires a video stream, and the endoscope video frame is a frame image in the video stream that currently needs to be subjected to sparsity analysis.

[0067] More specifically, in the embodiments of the present application, the three steps of acquiring and correcting endoscopic images, stereoscopic imaging calculation, and point cloud data set generation together constitute a conversion process from 2D endoscopic images to 3D point cloud data, which can effectively extract point cloud data sets from endoscopic video frames and provide the necessary data basis for subsequent sparse point cloud judgment.

[0068] More specifically, camera parameters usually include intrinsic parameters (such as focal length, principal point coordinates, distortion coefficients, etc.) and extrinsic parameters (such as camera position and posture). The correction process can adopt a variety of algorithms, such as the Brown-Conrady model or the Kannala-Brandt model; the corrected image will eliminate lens distortion and provide more accurate geometric information for subsequent processing.

[0069] More specifically, the step of performing stereoscopic imaging calculation on the endoscopic image can adopt a variety of stereo matching algorithms, such as local-based block matching (Block Matching), semi-global matching (Semi-Global Matching, SGM) or deep learning-based methods.

[0070] More specifically, obtaining a point cloud dataset based on the disparity map and camera parameters is a key step in converting 2D information into a 3D point cloud. This process involves disparity-depth conversion and mapping of pixel coordinates to world coordinates. In specific implementation, the disparity value can be converted into a depth value using the triangulation principle combined with the baseline length and focal length of the camera. Then, using the internal and external parameters of the camera, each pixel is mapped to its actual position in the 3D space to generate a point cloud dataset.

[0071] More specifically, endoscopic images usually have a small field of view and special lighting conditions. The sparse point cloud judgment method based on fluctuation factors in the embodiment of the present application combines camera parameter correction and stereo imaging technology. This method can better handle these special situations, improve the accuracy and reliability of point cloud data, and provide high-quality data input for subsequent sparse point cloud judgment.

[0072] In some preferred embodiments, the step of acquiring a point cloud data set according to the disparity map and the camera parameters includes:

[0073] Obtain a mapping image based on the disparity map and the projection matrix, where the projection matrix is ​​obtained based on the camera parameters;

[0074] The map is reshaped and invalid data are removed to obtain a point cloud dataset.

[0075] Specifically, the projection matrix is ​​calculated based on the camera parameters. The sparse point cloud judgment method based on the fluctuation factor in the embodiment of the present application can convert the two-dimensional image information into three-dimensional space information by combining the disparity map with the projection matrix, thereby obtaining a mapping map. Each pixel in the mapping map has three channels, which respectively store the values ​​of the three-dimensional coordinates of the pixel in the camera coordinate system.

[0076] More specifically, the reshaping process may include adjusting the data structure or format so that the pixel data including the three-dimensional coordinates are converted into a three-dimensional point cloud so as to construct a point cloud dataset, and deleting invalid data is to remove inaccurate or meaningless points to improve the quality and reliability of the point cloud dataset.

[0077] More specifically, the process of obtaining the mapping can be achieved in a variety of ways. A common method is to use the intrinsic and extrinsic matrix of the depth camera to construct a projection matrix. The intrinsic matrix contains information such as focal length and principal point coordinates, and the extrinsic matrix describes the position and posture of the camera in the world coordinate system. Multiply these two matrices to obtain the complete projection matrix. Then, each pixel in the disparity map is converted to a three-dimensional space coordinate through the projection matrix to form a mapping.

[0078] More specifically, for the reshaping of the map, a variety of data structures can be used, such as octrees or KD trees, to facilitate subsequent point cloud processing and analysis; deleting invalid data is a key step in improving the quality of point clouds, and the process can set multiple screening conditions, such as depth thresholds, confidence thresholds, etc. For example, points with abnormal depth values ​​(such as too close or too far) can be removed, or points with unreliable disparity values ​​can be removed. In addition, statistical filters can be applied to remove outliers or noise points. The combination of these steps can significantly improve the accuracy and availability of point cloud data.

[0079] The sparse point cloud judgment method based on fluctuation factors in the embodiment of the present application obtains a point cloud data set based on the above steps, which not only improves the accuracy of the point cloud, but also reduces the computational burden of subsequent processing.

[0080] In some preferred embodiments, the target area is a circular, rectangular or square area.

[0081] Specifically, the sparse point cloud judgment method based on fluctuation factors in the embodiment of the present application provides a simple and effective way to select and analyze point cloud data by limiting the target area to a specific geometric shape, wherein a circular area may be suitable for certain specific endoscopic image features, such as circular organs or tissues; a rectangular or square area may be more suitable for analyzing a larger image area or a specific rectangular structure.

[0082] More specifically, in the embodiment of the present application, the target area is preferably a square, and its symmetry facilitates the collection and calculation of fluctuation factors in different directions to more comprehensively reflect the distribution of the point cloud in different directions.

[0083] More specifically, using these regular geometric shapes as target regions also simplifies the subsequent calculation process. When calculating the fluctuation factor, the center point cloud of the region can be more easily determined, which improves the efficiency and reliability of the algorithm.

[0084] In some preferred embodiments, the fluctuation factor is calculated based on the following formula:

[0085] ;

[0086] Among them, Ω is the target area mark, k is the direction mark, j is the center point cloud mark of the target area, and α j,k is the fluctuation factor about the center point cloud j in the k direction, Point i,k is the coordinate of the i-th point cloud in the target area in the k direction, Point j,k is the coordinate of the center point cloud j in the target area in the k direction, and N is the total number of point clouds or pixels in the target area.

[0087] Specifically, in this embodiment, N varies according to the size of the target area. For example, if the target area Ω is a square area with a side length of r pixels, then N=r 2 .

[0088] More specifically, in this embodiment, α j,k is the mean-like variance of all point clouds in the target area Ω and the center point cloud j in the k direction; Based on the above content, it can be seen that if there is an offset in the data of some point clouds in the corresponding direction in the target area, these offsets are generally caused by inaccurate depth information of the corresponding pixels (manifested as a sharp change). This part of the point cloud shows sparse characteristics relative to the nearby point clouds, which will cause the coordinate value of this part of the point cloud to change sharply in a certain direction (refer to Figure 4 The scattered point cloud in the upper left corner of the figure leads to the fluctuation factor α in the corresponding direction. j,k Double.

[0089] More specifically, k=x, y or z, that is, the fluctuation factors obtained by the sparse point cloud judgment method based on fluctuation factors in the embodiment of the present application include α j,x , α j,y and α j,z , Step S3 determines the largest fluctuation factor and the smallest fluctuation factor by comparing the size relationship of the three fluctuation factors, and then performs density analysis.

[0090] More specifically, the calculation of the fluctuation factor is closely related to the spatial distribution of the point cloud in the target area. By comparing the fluctuation factors in different directions, the uniformity of the point cloud distribution can be judged. Since the sparse point cloud will cause at least one type of fluctuation factor to multiply, the sparse point cloud judgment method based on the fluctuation factor in the embodiment of the present application can easily determine whether the point cloud in the target area is directionally sparse or dense by analyzing the proportional relationship between the largest fluctuation factor and the smallest fluctuation factor. The sparse point cloud judgment method based on the fluctuation factor in the embodiment of the present application effectively solves the problem of accurately judging whether the point cloud in the target area is dense by introducing the fluctuation factor as a quantitative indicator of the spatial distribution characteristics. It not only improves the accuracy of the point cloud density judgment, but also provides a more reliable foundation for subsequent point cloud processing and application, thereby improving the overall performance of endoscopic image point cloud reconstruction.

[0091] In some preferred embodiments, the step of finally judging whether the point cloud in the target area is dense according to the proportional relationship between the maximum fluctuation factor and the minimum fluctuation factor includes:

[0092] Screen the largest and smallest fluctuation factors in the target area;

[0093] Calculate the volatility ratio between the maximum volatility factor and the minimum volatility factor;

[0094] The fluctuation ratio is compared with the preset ratio threshold to finally determine whether the point cloud in the target area is dense.

[0095] Specifically, the maximum and minimum fluctuation factors in the target area can respectively reflect the extreme conditions of the point cloud distribution in different directions. The fluctuation ratio calculated based on the two can reflect the difference in the density of the point cloud distribution in different directions. The fluctuation ratio is compared with the ratio threshold to finally determine whether the point cloud in the target area is dense.

[0096] More specifically, in this embodiment, a larger fluctuation ratio indicates that the directional sparse characteristic of the point cloud in the target area is more serious, wherein a fluctuation ratio less than a ratio threshold indicates that the point cloud in the target area is dense.

[0097] More specifically, the preset ratio threshold can be a ratio set by the user based on experience, or a ratio set by the system based on the fluctuation ratio at the previous moment. The former can conveniently rely on manual experience to quickly determine whether there is directional sparsity, while the latter can combine the video stream to continuously pay attention to the changes in the fluctuation ratio to determine whether the current endoscopic image frame has directional sparsity, breaking away from the limitations of manual experience. Among them, the latter can be a ratio threshold set based on a preset proportional coefficient to magnify the qualified fluctuation ratio at the previous moment.

[0098] In some preferred embodiments, the target area is determined based on the sliding window selection, or is generated based on a preset area size based on pixels corresponding to a central point cloud selected by a user.

[0099] Specifically, these two methods provide flexibility and efficiency for determining the target area. The appropriate method can be selected according to different application requirements and analysis purposes, thereby improving the applicability and efficiency of sparse point cloud judgment. Among them, the first method can be applied to the automatic analysis process of the endoscope system, that is, the endoscope system analyzes the sparsity of the point clouds at different local positions in the entire endoscopic image based on a sliding window, and the second method is applied to the analysis process performed by the user on a specific area, that is, the endoscope system can perform sparsity analysis of the local area based on the position selected by the user.

[0100] More specifically, the sliding window method can achieve automated analysis by setting the window size and step size. For example, a 50x50 pixel square window can be set to slide across the entire point cloud dataset with a step size of 10 pixels. After each slide, the point cloud data within the window will be extracted as the target area for analysis. This method can systematically cover the entire point cloud dataset without missing any area, which is particularly suitable for situations where comprehensive analysis is required.

[0101] More specifically, the user selection method can be implemented through a graphical interface. The user can click the center of the area of ​​interest on the displayed point cloud image, and the system will then automatically generate a target area based on the preset area size (such as 100x100 pixels) with the pixel point as the center. This method allows users to analyze specific areas of interest, improving the pertinence and efficiency of the analysis.

[0102] More specifically, these two methods can produce significant results when used in conjunction with the aforementioned fluctuation factor analysis method. The sliding window method automates the systematic analysis of the entire point cloud dataset, while the user selection method allows for in-depth research on specific areas. This flexibility allows sparse point cloud judgment to better adapt to different analysis needs, whether a comprehensive analysis or analysis of a specific area is required.

[0103] Second, please refer to Figure 5 Some embodiments of the present application also provide a sparse point cloud judgment device based on fluctuation factors, which is applied in point cloud reconstruction of endoscopic images, and the device includes:

[0104] The point cloud acquisition module 201 is used to acquire a point cloud data set corresponding to the endoscopic image, and acquire a local point cloud set within the target area according to the point cloud data set;

[0105] A preliminary judgment module 202 is used to obtain point cloud density information of a local point cloud set, and preliminarily judge whether the point cloud in the target area is dense according to the relationship between the point cloud density information and the point cloud density threshold;

[0106] The final judgment module 203 is used to calculate the fluctuation factors of the point clouds in the local point cloud concentration in the x, y and z directions respectively when it is initially judged to be dense, and finally judge whether the point cloud in the target area is dense according to the proportional relationship between the maximum fluctuation factor and the minimum fluctuation factor. The fluctuation factor is the mean-like variance between each point cloud in the target area and the central point cloud in the corresponding direction.

[0107] The sparse point cloud judgment device based on the fluctuation factor of the embodiment of the present application performs a secondary sparsity analysis on the target area to finally determine whether the point cloud of the target area appears to be overall dense, analyzes the point cloud density information to quickly and preliminarily exclude areas that are obviously sparse, and introduces the fluctuation factor to perform a directional sparse analysis on the target area preliminarily judged to be dense, so as to more accurately judge the density of the endoscopic image point cloud, and give a final density assessment, which provides more reliable basic data for subsequent three-dimensional reconstruction and surgical navigation, and helps to improve the accuracy and safety of endoscopic surgery.

[0108] In some preferred embodiments, the sparse point cloud judgment device based on fluctuation factors of the embodiments of the present application is used to execute the sparse point cloud judgment method based on fluctuation factors provided in the first aspect above.

[0109] Third, please refer to Figure 6 Some embodiments of the present application also provide a structural diagram of an electronic device. The present application provides an electronic device, including: a processor 301 and a memory 302. The processor 301 and the memory 302 are interconnected and communicate with each other through a communication bus 303 and / or other forms of connection mechanisms (not marked). The memory 302 stores computer-readable instructions executable by the processor 301. When the electronic device is running, the processor 301 executes the computer-readable instructions to execute the method in any optional implementation of the above embodiments.

[0110] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method in any optional implementation of the above embodiment is executed. The computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0111] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0112] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0114] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0115] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A sparse point cloud judgment method based on fluctuation factors is applied in point cloud reconstruction of endoscopic images, characterized in that: The method comprises the following steps: Acquire a point cloud data set corresponding to the endoscopic image, and acquire a local point cloud set within the target area based on the point cloud data set; Obtaining point cloud density information of the local point cloud set, and preliminarily determining whether the point cloud in the target area is dense based on the relationship between the point cloud density information and the point cloud density threshold; When it is initially determined to be dense, the fluctuation factors of the point clouds in the local point cloud set in the x, y and z directions are calculated respectively, and whether the point cloud in the target area is dense is finally determined according to the proportional relationship between the largest fluctuation factor and the smallest fluctuation factor. The fluctuation factor is the mean-like variance of each point cloud in the target area and the central point cloud in the corresponding direction; The volatility factor is calculated based on the following formula: ; Among them, Ω is the target area mark, k is the direction mark, j is the center point cloud mark of the target area, and α j,k is the fluctuation factor about the center point cloud j in the k direction, Point i,k is the coordinate of the i-th point cloud in the target area in the k direction, Point j,k is the coordinate of the center point cloud j in the target area in the k direction, and N is the total number of point clouds or pixels in the target area; The step of finally judging whether the point cloud in the target area is dense according to the proportional relationship between the maximum fluctuation factor and the minimum fluctuation factor comprises: Screen the largest and smallest fluctuation factors in the target area; Calculate the volatility ratio between the maximum volatility factor and the minimum volatility factor; The fluctuation ratio is compared with a preset ratio threshold to finally determine whether the point cloud in the target area is dense.

2. The sparse point cloud judgment method based on fluctuation factor according to claim 1 is characterized in that: The step of obtaining a point cloud data set corresponding to the endoscopic image comprises: Acquire an endoscopic video frame, and correct the endoscopic video frame based on camera parameters to acquire an endoscopic image; Performing stereoscopic imaging calculation on the endoscopic image to obtain a disparity map; The point cloud data set is acquired according to the disparity map and the camera parameters.

3. The sparse point cloud judgment method based on fluctuation factor according to claim 2 is characterized in that: The step of acquiring the point cloud data set according to the disparity map and the camera parameters comprises: Acquire a mapping image according to the disparity map and a projection matrix, wherein the projection matrix is ​​acquired based on the camera parameters; The map is reshaped and invalid data is deleted to obtain the point cloud dataset.

4. The sparse point cloud judgment method based on fluctuation factor according to claim 1 is characterized in that: The target area is a circular, rectangular or square area.

5. The sparse point cloud judgment method based on fluctuation factor according to claim 1 is characterized in that: The target area is determined based on the sliding window selection, or is generated based on a preset area size based on pixels corresponding to a central point cloud selected by a user.

6. A sparse point cloud judgment device based on fluctuation factor, applied in point cloud reconstruction of endoscopic images, characterized in that: The device comprises: A point cloud acquisition module, used to acquire a point cloud data set corresponding to the endoscopic image, and to acquire a local point cloud set within a target area based on the point cloud data set; A preliminary judgment module is used to obtain point cloud density information of a local point cloud set, and preliminarily judge whether the point cloud in the target area is dense according to the relationship between the point cloud density information and the point cloud density threshold; The final judgment module is used to calculate and obtain the fluctuation factors of the point clouds in the local point cloud concentration in the x, y and z directions respectively when it is initially judged to be dense, and finally judge whether the point cloud in the target area is dense according to the proportional relationship between the maximum fluctuation factor and the minimum fluctuation factor. The fluctuation factor is the mean-like variance of each point cloud in the target area and the central point cloud in the corresponding direction; The volatility factor is calculated based on the following formula: ; Among them, Ω is the target area mark, k is the direction mark, j is the center point cloud mark of the target area, and α j,k is the fluctuation factor about the center point cloud j in the k direction, Point i,k is the coordinate of the i-th point cloud in the target area in the k direction, Point j,k is the coordinate of the center point cloud j in the target area in the k direction, and N is the total number of point clouds or pixels in the target area; The step of finally judging whether the point cloud in the target area is dense according to the proportional relationship between the maximum fluctuation factor and the minimum fluctuation factor comprises: Screen the largest and smallest fluctuation factors in the target area; Calculate the volatility ratio between the maximum volatility factor and the minimum volatility factor; The fluctuation ratio is compared with a preset ratio threshold to finally determine whether the point cloud in the target area is dense.

7. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 5 are executed.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are executed.

Images