Swallowing frame cloud picture synthesis method, device and equipment and storage medium

Abstract

The invention discloses a swallowing frame cloud picture synthesis method and device, equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: taking a plurality of swallowing frame cloud charts acquired by swallowing, zooming the swallowing frame cloud charts to a standard duration through mapping, and performing interpolation or three times of convolution processing to obtain processed swallowing frame cloud charts; determining a fusion factor corresponding to each pixel block in the processed swallowing frame cloud picture; the fusion factor is determined based on a first weight of a region type corresponding to the pixel block and a second weight of the processed swallowing frame cloud picture to which the pixel block belongs; the region type comprises a feature point, a target region and other regions; and carrying out cloud picture fusion according to the pixel value and the fusion factor corresponding to each pixel block in the processed swallowing frame cloud picture to obtain a fused swallowing frame cloud picture. A plurality of swallowing frame cloud pictures are fused to obtain a fused swallowing frame cloud picture capable of representing the comprehensive swallowing condition, so that the overall swallowing condition of a patient can be preliminarily evaluated.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for synthesizing swallowing box cloud images. Background Technology

[0002] Esophageal manometry is the gold standard for evaluating esophageal motility disorders, encompassing both conventional manometry and high-resolution manometry (HRM). Conventional manometry uses a 4- or 8-lead manometry catheter, employing a gradual traction manometry method after catheter insertion. Water-infused HRMs typically feature 21–36 pressure sensor channels, while solid-state catheter HRMs can integrate up to 33–36 or even more miniature pressure sensor channels, providing a more direct and accurate reflection of esophageal motility. During swallowing data monitoring, clinicians generally collect one resting frame contour map and multiple swallowing frame contour maps in various positions (e.g., supine, upright). These maps are then analyzed individually to obtain various technical parameters for diagnosis. However, the lack of a comprehensive overview before individual contour map analysis hinders the ability to assess the patient's overall condition and preliminary evaluation of overall swallowing effectiveness. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for synthesizing a swallowing frame contour map, which can obtain a fused swallowing frame contour map that can characterize the overall swallowing situation. The specific solution is as follows: In a first aspect, this application discloses a method for synthesizing swallowing box contour maps, including: Multiple swallowing box cloud maps obtained from swallowing acquisition are acquired. The swallowing box cloud maps are scaled to a standard duration by mapping and then interpolated or subjected to cubic convolution to obtain a processed swallowing box cloud map. The fusion factor corresponding to each pixel block in the processed swallowing bounding box cloud is determined; wherein, the fusion factor is determined based on a first weight of the region type corresponding to the pixel block and a second weight of the processed swallowing bounding box cloud to which the pixel block belongs; the region type includes feature points, target regions and other regions; Based on the pixel values ​​and fusion factors corresponding to each pixel block in the processed swallowing frame cloud map, cloud map fusion is performed to obtain a fused swallowing frame cloud map.

[0004] Optionally, before determining the corresponding fusion factor for each pixel block in the processed swallowing box cloud image, the method further includes: Based on the pressure threshold, the target region edge is detected in the processed swallowing box cloud map to determine the target region where the pressure is higher than the pressure threshold; The feature points in the processed swallowing frame cloud are determined; the feature points include the start position, end position and lowest point of UES relaxation, as well as the start position, end position and peak point of the swallowing peristaltic wave.

[0005] Optionally, the step of performing target region edge detection on the processed swallowing box cloud map based on a pressure threshold to determine the target region where the pressure is higher than the pressure threshold includes: The processed swallowing frame cloud map is divided into multiple pixel blocks by grid division; The pressure values ​​of the four vertices of each pixel block are compared with the pressure threshold, and target pixel blocks are selected based on the comparison results. The target pixel blocks are those that exclude pixel blocks where the pressure of all four vertices is less than the pressure threshold and pixel blocks where the pressure of all four vertices is greater than the pressure threshold. The target region is obtained by connecting two adjacent vertices in all the target pixel blocks that are above the pressure threshold.

[0006] Optionally, before connecting two adjacent vertices above the pressure threshold in all the target pixel blocks, the method further includes: Based on the pressure threshold, isobaric point interpolation is performed between two adjacent vertices in all target pixel blocks that are above the pressure threshold using linear interpolation. The target region is obtained by connecting the pressure points obtained from the interpolation.

[0007] Optionally, after comparing the numerical relationship between the pressure at each of the four vertices of each pixel block and the pressure threshold, the method further includes: Based on the comparison results, the vertex with a pressure value greater than or equal to the pressure threshold is designated as the first state point, and the vertex with a pressure value less than the pressure threshold is designated as the second state point. Determine the number of first state points corresponding to each of the processed swallowing box cloud maps, and the total number of first state points corresponding to all the processed swallowing box cloud maps; The second weight of the processed swallowing bounding box is determined based on the ratio of the number of the first state points to the total number of the first state points.

[0008] Optionally, the step of scaling the swallowing bounding box image to a standard duration through mapping and performing interpolation or cubic convolution to obtain a processed swallowing bounding box image includes: The swallowing frame contour map is scaled to a standard duration using center alignment mapping to obtain a scaled swallowing frame contour map; The pixel center points of the scaled swallowing box cloud are mapped back to the swallowing box cloud, and bilinear interpolation, cubic interpolation, nearest neighbor interpolation, or cubic convolution are performed to obtain the processed swallowing box cloud.

[0009] Optionally, determining the corresponding fusion factor for each pixel block in the processed swallowing bounding box includes: Traverse each pixel block in each processed swallowing box cloud map, and calculate the fusion factor of the current pixel block based on the first weight and the second weight corresponding to the current pixel block, and the weight coefficients corresponding to the first weight and the second weight respectively. The step of fusing the swallowing frame cloud map based on the pixel values ​​of each pixel block in the processed swallowing frame cloud map and the fusion factor corresponding to the pixel block to obtain a fused swallowing frame cloud map includes: Based on the pixel values ​​of the same position in the swallowing box cloud map after different processing, and the fusion factor corresponding to the pixel block, the fused pixel value of the pixel block at the current position is obtained by weighted summation; The fused swallowing box cloud map is obtained based on the pixel values ​​after fusion of all pixel blocks.

[0010] Secondly, this application discloses a swallowing frame cloud image synthesis device, comprising: The acquisition and processing module is used to acquire multiple swallowing box cloud maps obtained from swallowing data collection, scale the swallowing box cloud maps to a standard duration through mapping, and perform interpolation or cubic convolution processing to obtain processed swallowing box cloud maps. The fusion factor determination module is used to determine the corresponding fusion factor for each pixel block in the processed swallowing bounding box cloud map; wherein, the fusion factor is determined based on a first weight of the region type corresponding to the pixel block and a second weight of the processed swallowing bounding box cloud map to which the pixel block belongs; the region type includes feature points, target regions and other regions; The fusion module is used to perform cloud map fusion based on the pixel values ​​of each pixel block in the processed swallowing frame cloud map and the fusion factor corresponding to the pixel block, so as to obtain a fused swallowing frame cloud map.

[0011] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned swallowing box cloud image synthesis method.

[0012] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned swallowing box cloud image synthesis method.

[0013] In this application, multiple swallowing bounding box (BWT) images obtained from swallowing data are acquired. These BWT images are then scaled to a standard duration through mapping and subjected to interpolation or cubic convolution to obtain a processed BWT image. A fusion factor is determined for each pixel block in the processed BWT image. The fusion factor is determined based on a first weight of the region type corresponding to the pixel block and a second weight of the processed BWT image to which the pixel block belongs. The region type includes feature points, target regions, and other regions. BWT images are fused according to the pixel values ​​corresponding to each pixel block in the processed BWT image and the fusion factor to obtain a fused BWT image. The pixel values ​​are color values ​​converted from pressure values.

[0014] As can be seen above, preprocessing the swallowing frame cloud map by scaling and interpolation ensures the accuracy and effectiveness of subsequent fusion. Using pixel blocks as the fusion unit, the corresponding fusion factor for each pixel block is calculated, and cloud map fusion is performed based on pixel values ​​and fusion factors. Furthermore, the fusion factor of a pixel block is determined by comprehensively considering the region type corresponding to the pixel block and the processed swallowing frame cloud map to which the pixel block belongs, thereby improving the accuracy of fusion. By fusing multiple swallowing frame cloud maps, a fused swallowing frame cloud map that can characterize the overall swallowing situation is obtained, facilitating a preliminary assessment of the patient's overall swallowing condition. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 A flowchart of a swallowing frame cloud image synthesis method provided in this application; Figure 2 A flowchart of a specific swallowing frame cloud image synthesis method provided in this application; Figure 3 This application provides a schematic diagram of a swallowing frame cloud image synthesis device. Figure 4 This application provides a structural diagram of an electronic device. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In existing technologies, during swallowing data monitoring, clinicians typically collect one resting frame cloud map and multiple swallowing frame cloud maps in multiple body positions (such as supine and upright). These swallowing frame cloud maps can include various types such as wet swallowing, dry swallowing, solid swallowing, and multiple rapid swallows. Clinicians typically specify a primary body position (e.g., supine) and select a primary swallowing type (e.g., wet swallowing), collecting approximately 10 swallowing frames. Other swallowing types are selectively completed in smaller amounts. The standard duration for each swallowing frame is 30 seconds. The reporting physician analyzes each collected resting frame and swallowing frame icon to obtain various technical parameters for diagnosis. However, before conducting individual analyses, the reporting physician desires an overview of the patient's overall condition and a preliminary assessment of the overall swallowing effect to facilitate a preliminary diagnosis; hence, a comprehensive visualization is lacking. To overcome these technical problems, this application proposes a swallowing frame cloud map synthesis method that can generate an accurate and effective fused swallowing frame cloud map representing the overall swallowing situation, facilitating a preliminary assessment of the patient's overall swallowing condition.

[0019] This application discloses a method for synthesizing swallowing box contour maps. See also Figure 1 As shown, the method may include the following steps: Step S11: Obtain multiple swallowing box cloud maps obtained from swallowing acquisition, scale the swallowing box cloud maps to a standard duration through mapping, and perform interpolation or cubic convolution processing to obtain processed swallowing box cloud maps.

[0020] First, multiple swallowing frame contour maps are obtained from swallowing data collection. Specifically, a swallowing frame contour map refers to a valid swallowing frame identifier selected from the collected swallowing frame identifiers, which has undergone Chicago classification analysis and has a duration >20 seconds. It's understood that after swallowing frame data collection begins, there is an automatic 30-second countdown (the doctor can end the swallowing action early), which will not exceed 30 seconds. However, a swallowing duration of less than 20 seconds may indicate incomplete swallowing and cannot fully reflect a swallowing peristaltic waveform. After Chicago classification analysis, invalid swallowing frame identifiers are removed to avoid interference during the synthesis process. The swallowing frame contour map uses a heatmap format that displays pressure distribution; the pixel values ​​in the image are color values ​​converted from pressure values.

[0021] Because swallowing durations vary, the lengths of the swallowing bounding box contour maps also differ. To ensure effective subsequent fusion, all contour maps need to be converted to a unified standard first. Specifically, a base map is created first; the standard swallowing duration of 30 seconds is used as the X-axis, and the pixel height of the swallowing box marker is used as the Y-axis. Using 30 seconds as the standard duration minimizes interpolation calculations due to inconsistent durations during swallowing synthesis. Using the pixel height of the swallowing box marker as the Y-axis effectively utilizes existing, already calculated contour map data, improving the calculation speed of the synthesized image.

[0022] In some embodiments, scaling the swallowing box cloud to a standard duration through mapping and performing interpolation or cubic convolution to obtain a processed swallowing box cloud includes: scaling the swallowing box cloud to a standard duration through center-aligned mapping to obtain a scaled swallowing box cloud; mapping the pixel center points of the scaled swallowing box cloud back to the swallowing box cloud, and performing bilinear interpolation, cubic interpolation, nearest neighbor interpolation, or cubic convolution to obtain a processed swallowing box cloud.

[0023] Understandably, during the data collection process, due to patient tolerance or the patient ending swallowing prematurely, not all swallowing frame contour maps have a duration of 30 seconds. Therefore, it is necessary to first scale down the swallowing frame contour maps with non-standard durations. The specific mapping process is as follows: 1. Establish a matrix; First, select a swallowing frame contour map and treat its contour map data as a matrix A[m,n] with a width (X-axis) of m and a height (Y-axis) of n.

[0024] 2. Matrix Mapping: Consider the matrix as a coordinate axis consisting of x and y. For matrix A, its width ranges from [0, n] and its height ranges from [0, m]. Divide the cloud map into n columns and m rows to obtain multiple pixel blocks. In computer science, (0, 0) represents the top-left corner of the first pixel, and (m, n) represents the bottom-right corner of the last pixel. Therefore, the center coordinates of each pixel block in the cloud map are (0.5, 0.5), (0.5, 1.5), ..., (m-0.5, n-0.5). The new matrix B[M, N] obtained from the base map has a width range of [0, N] and a height range of [0, M]. The center coordinates of the pixels are (0.5, 0.5), (0.5, 1.5), ..., (M-0.5, N-0.5). The coordinates (iB, jB) of any pixel center point in matrix B (i.e., the center of the pixel block) correspond to the coordinates (src) in the coordinate system of the original matrix A. i src j The mapping formula is: src i =iB / M×m;src j = j B / N×n; Where iB represents the x-coordinate of the center of the pixel block, and src i It is the absolute x-coordinate mapped to the original image A.

[0025] If represented by row and column indices, the center coordinates of the pixel in the i-th row and j-th column of matrix B are (i+0.5, j+0.5), which can be mapped back to the coordinates (src) of matrix A. x src y )for: ; Where i represents the row index of the pixel block, and src x It is the relative x-coordinate mapped to the original image A; j represents the column index of the pixel block, src y It is the relative vertical coordinate mapped to the original image A.

[0026] To ensure pixel continuity in the scaled image, interpolation is performed, specifically bilinear interpolation. This aims to maintain both smoothness and computational speed in the contour image. The bilinear interpolation process is as follows: Find (src) x src y The nearest 2x2 pixels (Q11, Q12, Q21, Q22) are identified. Two linear interpolations are first performed horizontally to obtain R1 and R2. ; Then, perform a linear interpolation on R1 and R2 in the vertical direction to obtain the final result P: ; Where x1 = floor(src) x x2 = ceil(src) x ), y1 = floor(src y ), y2 = ceil(src y Similarly, floor(x) represents rounding x down, and ceil(x) represents rounding x up.

[0027] Step S12: Determine the fusion factor corresponding to each pixel block in the processed swallowing bounding box cloud map; wherein, the fusion factor is determined based on the first weight of the region type corresponding to the pixel block and the second weight of the processed swallowing bounding box cloud map to which the pixel block belongs; the region type includes feature points, target regions and other regions.

[0028] Before image fusion, a fusion factor for each pixel block is calculated. The fusion factor is determined by combining the first weight of the region type corresponding to the pixel block and the second weight of the processed swallowing box cloud map to which the pixel block belongs.

[0029] The calculation of the first weight requires first identifying the region types in the cloud map. In this embodiment, these are divided into feature points, target regions, and other regions, with different first weight values ​​W corresponding to each region. 1 different.

[0030] In some embodiments, before determining the corresponding fusion factor of each pixel block in the processed swallowing frame cloud, the method further includes: determining feature points in the processed swallowing frame cloud; the feature points include the start position, end position, and lowest point of UES relaxation, and the start position, end position, and peak point of swallowing peristaltic waves; performing target region edge detection on the processed swallowing frame cloud based on a pressure threshold to determine the target region where the pressure is higher than the pressure threshold. The feature points include specific positions of the swallowing frame identifier, such as the start, end, and lowest points of UES relaxation; and the start, end, and peak points of swallowing peristaltic waves corresponding to L3, L7, and L11 above the LES (3cm, 7cm, 11cm). The lowest point of UES relaxation is determined, and the start and end positions of UES relaxation are determined based on the intersection of the interpolation curve of the pressure value of the UES channel and the baseline; the highest point of the pressure wave on the target channel is determined, and the peak type, start and end positions of the swallowing peristaltic waves of the target channel are determined based on the intersection of the interpolation curve of the pressure value of the target channel and the corresponding baseline. This embodiment does not specifically limit the method of feature point recognition; any existing method is acceptable. Based on the pressure threshold, the processed swallowing frame contour map is used for target region edge detection to determine the region where the pressure is higher than the pressure threshold as the target region. For example… Figure 2 As shown, the high-pressure area obtained through edge detection is used as the target area.

[0031] In some embodiments, the step of performing target region edge detection on the processed swallowing bounding box cloud based on a pressure threshold to determine the target region where the pressure is higher than the pressure threshold includes: dividing the processed swallowing bounding box cloud into a grid to obtain multiple pixel blocks; comparing the numerical relationship between the pressure at the four vertices of each pixel block and the pressure threshold, and filtering out target pixel blocks based on the comparison results; the target pixel blocks are those that exclude pixel blocks where the pressure at all four vertices is less than the pressure threshold and pixel blocks where the pressure at all four vertices is greater than the pressure threshold; and connecting two adjacent vertices in all target pixel blocks that are higher than the pressure threshold to obtain the target region.

[0032] To accurately obtain the target area, the four vertices of each pixel block are analyzed in sequence, and the isobaric lines are marked in combination with the pressure threshold LT (such as 20 mmHg) to obtain the isobaric line with the pressure being the pressure threshold, which serves as the edge of the target area. The specific process of marking the isobaric line is as follows: Regarding the processed swallowing frame cloud map [M, N] as a discrete grid data of M×N, it can be represented by a function: F i,j =f(x i ,y j ); i = 0, 1, …, M - 1; j = 0, 1, …, N - 1; F i,j represents the pressure value at the coordinate (x i ,y j ). By finding all points that satisfy F≈LT in this discrete grid and connecting them into a continuous curve. Specifically, traverse all cells, a total of (M - 1)×(N - 1) cells, each cell consists of 4 adjacent grid points (the above vertices); for the 4 vertices of each cell, compare their relationships with LT, and exclude the pixel blocks where the pressures of all four vertices are less than the pressure threshold and the pixel blocks where the pressures of all four vertices are greater than the pressure threshold to obtain the target pixel blocks. Specifically, the target pixel blocks can be screened by forming a 4-bit binary number from the states of the 4 vertices. Each vertex has two states: 1, indicating F i,j >LT (higher than the threshold); 0, indicating F i,j <LT (lower than the threshold); then as shown in Table 1 below: Table 1 Schematic Diagram of Pixel Block Vertex States

[0033] That is, except for the pixel blocks of "0000" and "1111", the other pixel blocks are used as target pixel blocks; it can be understood that the pixel blocks corresponding to "0000" are in the low-pressure area, and the pixel blocks corresponding to "1111" are in the high-pressure area, and the purpose is to find the boundary between high and low pressures (here, high and low pressures are defined relative to the pressure threshold). The target pixel blocks are those where some vertices of the pixel block are less than the pressure threshold and some vertices have pressures greater than the pressure threshold. That is to say, the target pixel blocks contain the boundary between high and low pressures; therefore, by connecting the two adjacent vertices above the pressure threshold in all target pixel blocks, the enclosed area is the target area.

[0034] In some embodiments, before connecting the two adjacent vertices above the pressure threshold in all the target pixel blocks, it further includes: According to the pressure threshold, by linear interpolation, equal-pressure point interpolation is performed between the two adjacent vertices above the pressure threshold in all the target pixel blocks, and the target area is obtained by connecting the interpolated pressure points. Linear interpolation is performed on the edges passing through the isobaric lines, and finally, the two adjacent vertices above the pressure threshold in the target pixel blocks and the interpolation points are connected to form isobaric line segments, further ensuring the continuity of the edge of the target area.

[0035] Specifically, the isobaric point interpolation adopts a linear interpolation method. That is, for the edges crossing the isobars, the isobaric point positions are calculated through linear values. In a preferred embodiment, a bilinear interpolation method is adopted, and the specific interpolation process is as follows: For Pa(x i ,y j ) and Pb(x i+1 ,y j ), perform horizontal interpolation: ; Where, is the difference between x i+1 and x i , ; For Pa(x i ,y j ) and Pb(x i ,y j+1 ), perform vertical interpolation: Where, is the difference between y j+1 and y j , .

[0036] Meanwhile, it is also necessary to calculate the second weight of the pixel block. In some embodiments, after respectively comparing the numerical relationships between the pressures of the four vertices of each pixel block and the pressure threshold, it further includes: according to the comparison results, taking the vertices with pressure values greater than or equal to the pressure threshold as the first state points, and taking the vertices with pressure values less than the pressure threshold as the second state points; determining the number of first state points corresponding to each processed swallowing frame cloud map, and the total number of first state points corresponding to all the processed swallowing frame cloud maps; determining the second weight of the processed swallowing frame cloud map according to the ratio of the number of first state points to the total number of first state points.

[0037] It can be understood that the fusion factor needs to consider the proportion of the isobaric threshold LT as much as possible to ensure the fusion effect. Each vertex has two states: the first state point, represented by 1, indicating F i,j >LT (higher than the threshold); the second state point, represented by 0, indicating F i,j <LT (lower than the threshold). Therefore, after summing up the first state points (1), perform percentage allocation. For example, the vertices included in the first swallowing frame are a total of (M - 1) (N - 1) 4, where the total number of first state points (1), i.e., the number of first state points, is denoted as S1, and the number of first state points corresponding to the i-th swallowing box is denoted as Si. Then the number of first state points contained in all swallowing box cloud maps is ΣS=S1+S2+...+Sn. Then the second weight W corresponding to the i-th swallowing box cloud map is... i 2 =Si / (S1+S2...+Sn), where i∈(1,n).

[0038] By traversing each pixel block in the processed swallowing bounding box, and based on the first and second weights corresponding to the current pixel block, as well as the weight coefficients corresponding to the first and second weights respectively, the fusion factor of the current pixel block is calculated: α i =a W i 1 +b W i 2 .

[0039] Step S13: Perform cloud image fusion based on the pixel values ​​and fusion factors corresponding to each pixel block in the processed swallowing frame cloud image to obtain a fused swallowing frame cloud image; the pixel values ​​are color values ​​converted from pressure values.

[0040] Specifically, the swallowing frame cloud map is fused based on the pixel values ​​and fusion factors of each pixel block in the processed swallowing frame cloud map to obtain a fused swallowing frame cloud map. This includes: obtaining the fused pixel value of the pixel block at the current position by weighted summation based on the pixel values ​​of pixel blocks at the same position in different processed swallowing frame cloud maps and the fusion factors corresponding to the pixel blocks; and obtaining the fused swallowing frame cloud map based on the fused pixel values ​​of all pixel blocks.

[0041] Specifically, the linear gradient method can be used for image fusion, and the fusion formula is as follows: I_blend(x,y) = α1(x,y) I1(x,y)+ α2(x,y) I2(x,y)+... +α n (x,y) I n (x,y); Where α1, α2... α n For fusion factor; α1(x,y) represents the fusion factor corresponding to the pixel block with center coordinates (x,y) within the first swallowing bounding box contour map; I1(x,y) represents the pixel value corresponding to the pixel block with center coordinates (x,y) within the first swallowing bounding box contour map; α n(x,y) represents the fusion factor corresponding to the pixel block with center coordinates (x,y) within the nth swallowing box cloud map; I n (x, y) represents the pixel value corresponding to the pixel block with center coordinates (x, y) within the nth swallowing box contour map. The pixel value is the color value converted from the pressure value, that is, the color value converted by pressure, composed of R, G, and B, and is a 0xRGB hexadecimal number. For example, R=0x22, G=0x35, B=0x28, the corresponding pixel value is 0x223528, which is 2241832 in decimal.

[0042] By synthesizing swallowing frame cloud maps, an effective and accurate fused swallowing frame cloud map that can characterize the overall swallowing situation is obtained, reflecting the overall situation of the case. Before analyzing each cloud map individually, a quick overview of the case can be obtained, and the overall situation of the case can be assessed. The swallowing composite map comprehensively considers the UES relaxation and LES swallowing peristalsis waveforms, making it convenient to intuitively evaluate the patient's swallowing peristalsis contraction effect through the composite map.

[0043] As can be seen from the above, this embodiment acquires multiple swallowing bounding box (BWT) images obtained from swallowing data acquisition. These BWT images are then scaled to a standard duration through mapping and subjected to interpolation or cubic convolution to obtain a processed BWT image. A fusion factor is determined for each pixel block in the processed BWT image. The fusion factor is determined based on a first weight of the region type corresponding to the pixel block and a second weight of the processed BWT image to which the pixel block belongs. The region type includes feature points, target regions, and other regions. BWT images are fused according to the pixel values ​​corresponding to each pixel block in the processed BWT image and the fusion factor to obtain a fused BWT image. The pixel values ​​are color values ​​converted from pressure values. First, preprocessing the swallowing bounding box (BWT) cloud image, such as scaling and interpolation, yields a processed BWT cloud image, ensuring the accuracy and effectiveness of subsequent fusion. Using pixel blocks as the fusion unit, the corresponding fusion factor for each pixel block is calculated. BWT cloud images are then fused based on pixel values ​​and fusion factors to obtain a fused swallowing bounding box cloud image that can characterize the overall swallowing situation. Furthermore, the fusion factor for each pixel block is determined by comprehensively considering the region type corresponding to the pixel block and the processed swallowing bounding box cloud image to which the pixel block belongs, thereby improving the accuracy of fusion.

[0044] Accordingly, this application also discloses a swallowing frame cloud map synthesis device, see [link to relevant documentation]. Figure 3 As shown, the device includes: The acquisition and processing module 11 is used to acquire multiple swallowing frame cloud maps obtained from swallowing, scale the swallowing frame cloud maps to a standard duration through mapping, and perform interpolation or cubic convolution processing to obtain a processed swallowing frame cloud map. The fusion factor determination module 12 is used to determine the corresponding fusion factor for each pixel block in the processed swallowing box cloud map; wherein, the fusion factor is determined based on a first weight of the region type corresponding to the pixel block and a second weight of the processed swallowing box cloud map to which the pixel block belongs; the region type includes feature points, target regions and other regions; The fusion module 13 is used to perform cloud map fusion based on the pixel values ​​of each pixel block in the processed swallowing frame cloud map and the fusion factor corresponding to the pixel block to obtain a fused swallowing frame cloud map; the pixel values ​​are color values ​​converted from pressure values.

[0045] As can be seen from the above, this embodiment acquires multiple swallowing bounding box (BWT) images obtained from swallowing data acquisition. These BWT images are then scaled to a standard duration through mapping and subjected to interpolation or cubic convolution to obtain a processed BWT image. A fusion factor is determined for each pixel block in the processed BWT image. The fusion factor is determined based on a first weight of the region type corresponding to the pixel block and a second weight of the processed BWT image to which the pixel block belongs. The region type includes feature points, target regions, and other regions. BWT images are fused according to the pixel values ​​corresponding to each pixel block in the processed BWT image and the fusion factor to obtain a fused BWT image. The pixel values ​​are color values ​​converted from pressure values. As can be seen, by first performing preprocessing such as scaling and interpolation on the swallowing bounding box cloud map to obtain the processed swallowing bounding box cloud map, the accuracy and effectiveness of subsequent fusion are ensured. Using pixel blocks as the fusion unit, the corresponding fusion factor of each pixel block is calculated, and cloud map fusion is performed based on pixel values ​​and fusion factors to obtain a fused swallowing bounding box cloud map that can characterize the overall swallowing situation. Furthermore, the fusion factor of the pixel block is determined by comprehensively considering the region type corresponding to the pixel block and the processed swallowing bounding box cloud map to which the pixel block belongs, thereby improving the accuracy of fusion.

[0046] In some specific embodiments, the swallowing frame cloud synthesis device may specifically include: An edge detection unit is used to perform target region edge detection on the processed swallowing box cloud map according to a pressure threshold before determining the corresponding fusion factor of each pixel block in the processed swallowing box cloud map, and to determine the target region where the pressure is higher than the pressure threshold. The feature point determination unit is used to determine the feature points in the processed swallowing box cloud map; the feature points include the start position, end position and lowest point of UES relaxation, as well as the start position, end position and peak point of swallowing peristaltic wave.

[0047] In some specific embodiments, the edge detection unit may specifically include: A partitioning unit is used to divide the processed swallowing box cloud map into multiple pixel blocks using a grid. The target pixel block filtering unit is used to compare the numerical relationship between the pressure of the four vertices of each pixel block and the pressure threshold, and to filter out target pixel blocks based on the comparison results; the target pixel blocks are those that exclude pixel blocks where the pressure of all four vertices is less than the pressure threshold and pixel blocks where the pressure of all four vertices is greater than the pressure threshold. The target region determination unit is used to connect two adjacent vertices in all the target pixel blocks that are above the pressure threshold to obtain the target region.

[0048] In some specific embodiments, the edge detection unit may specifically include: An interpolation unit is used to perform isobaric point interpolation between two adjacent vertices above the pressure threshold in all the target pixel blocks before connecting them using linear interpolation based on the pressure threshold. The interpolated pressure points are then connected to obtain the target region.

[0049] In some specific embodiments, the fusion factor determination module 12 may specifically include: The vertex state determination unit is used to, after comparing the numerical relationship between the pressure and the pressure threshold of the four vertices of each pixel block, determine the first state point based on the comparison result, and the second state point based on the vertices whose pressure value is greater than or equal to the pressure threshold. The statistics unit is used to determine the number of first state points corresponding to each of the processed swallowing box cloud maps, and the total number of first state points corresponding to all the processed swallowing box cloud maps. The second weight determination unit is used to determine the second weight of the processed swallowing box cloud map based on the ratio of the number of the first state points to the total number of the first state points.

[0050] In some specific embodiments, the acquisition and processing module 11 may specifically include: The mapping unit is used to scale the swallowing frame cloud to a standard duration through center alignment mapping to obtain a scaled swallowing frame cloud. An interpolation unit is used to map the pixel center points of the scaled swallowing box cloud back to the swallowing box cloud, and perform bilinear interpolation, cubic interpolation, nearest neighbor interpolation, or cubic convolution to obtain the processed swallowing box cloud.

[0051] In some specific embodiments, the fusion factor determination module 12 may specifically include: The fusion factor calculation unit is used to traverse each pixel block in each processed swallowing box cloud map, and calculate the fusion factor of the current pixel block according to the first weight and the second weight corresponding to the current pixel block, and the weight coefficients corresponding to the first weight and the second weight respectively. Accordingly, the fusion module 13 may specifically include: The fused pixel value calculation unit is used to obtain the fused pixel value of the current position pixel block by weighted summation based on the pixel values ​​of pixel blocks at the same position in the swallowing box cloud map after different processing, and the fusion factor corresponding to the pixel block. The fused swallowing box cloud determination unit is used to obtain the fused swallowing box cloud based on the fused pixel values ​​of all pixel blocks.

[0052] Furthermore, this application also discloses an electronic device, see [link to relevant documentation]. Figure 4 As shown, the content in the figure should not be considered as any limitation on the scope of use of this application.

[0053] Figure 4 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the swallowing box cloud image synthesis method disclosed in any of the foregoing embodiments.

[0054] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0055] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include operating system 221, computer program 222 and data 223 including swallowing box cloud map, etc. The storage method can be temporary storage or permanent storage.

[0056] The operating system 221 manages and controls the various hardware devices on the electronic device 20 and the computer program 222 to enable the processor 21 to perform calculations and processing on the massive data 223 in the memory 22. The operating system 221 can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the swallowing box cloud image synthesis method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0057] Furthermore, this application also discloses a computer storage medium storing computer-executable instructions. When the computer-executable instructions are loaded and executed by a processor, they implement the swallowing box cloud map synthesis method steps disclosed in any of the foregoing embodiments.

[0058] Furthermore, this application also discloses a computer program product, including a computer program that, when executed by a processor, implements the swallowing box cloud image synthesis method steps disclosed in any of the foregoing embodiments.

[0059] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0060] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0061] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0062] The swallowing box cloud image synthesis method, apparatus, device, and storage medium provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for synthesizing swallowing box contour maps, characterized in that, include: Multiple swallowing bounding box images obtained from swallowing data acquisition are acquired. The swallowing bounding box images are scaled to a standard duration through mapping and then interpolated or subjected to cubic convolution to obtain the processed swallowing bounding box images. The fusion factor corresponding to each pixel block in the processed swallowing bounding box cloud is determined; wherein, the fusion factor is determined based on a first weight of the region type corresponding to the pixel block and a second weight of the processed swallowing bounding box cloud to which the pixel block belongs; the region type includes feature points, target regions and other regions; Based on the pixel values ​​and fusion factors corresponding to each pixel block in the processed swallowing frame cloud map, cloud map fusion is performed to obtain a fused swallowing frame cloud map.

2. The swallowing frame cloud image synthesis method according to claim 1, characterized in that, Before determining the corresponding fusion factor for each pixel block in the processed swallowing bounding box cloud map, the method further includes: Based on the pressure threshold, the target region edge is detected in the processed swallowing box cloud map to determine the target region where the pressure is higher than the pressure threshold; The feature points in the processed swallowing frame cloud are determined; the feature points include the start position, end position and lowest point of UES relaxation, as well as the start position, end position and peak point of the swallowing peristaltic wave.

3. The swallowing frame cloud synthesis method according to claim 2, characterized in that, The step of performing target region edge detection on the processed swallowing box cloud map based on a pressure threshold to determine the target region where the pressure is higher than the pressure threshold includes: The processed swallowing frame cloud map is divided into multiple pixel blocks by grid division; The pressure values ​​of the four vertices of each pixel block are compared with the pressure threshold, and target pixel blocks are selected based on the comparison results. The target pixel blocks are those that exclude pixel blocks where the pressure of all four vertices is less than the pressure threshold and pixel blocks where the pressure of all four vertices is greater than the pressure threshold. The target region is obtained by connecting two adjacent vertices in all the target pixel blocks that are above the pressure threshold.

4. The swallowing frame cloud image synthesis method according to claim 3, characterized in that, Before connecting two adjacent vertices above the pressure threshold in all the target pixel blocks, the method further includes: Based on the pressure threshold, isobaric point interpolation is performed between two adjacent vertices in all target pixel blocks that are above the pressure threshold using linear interpolation. The target region is obtained by connecting the pressure points obtained from the interpolation.

5. The swallowing frame cloud synthesis method according to claim 3, characterized in that, After comparing the numerical relationship between the pressure at each of the four vertices of each pixel block and the pressure threshold, the method further includes: Based on the comparison results, the vertex with a pressure value greater than or equal to the pressure threshold is designated as the first state point, and the vertex with a pressure value less than the pressure threshold is designated as the second state point. Determine the number of first state points corresponding to each of the processed swallowing box cloud maps, and the total number of first state points corresponding to all the processed swallowing box cloud maps; The second weight of the processed swallowing bounding box is determined based on the ratio of the number of the first state points to the total number of the first state points.

6. The swallowing frame cloud synthesis method according to claim 1, characterized in that, The process of scaling the swallowing bounding box to a standard duration through mapping and then performing interpolation or cubic convolution to obtain the processed swallowing bounding box includes: The swallowing frame contour map is scaled to a standard duration using center alignment mapping to obtain a scaled swallowing frame contour map; The pixel center points of the scaled swallowing box cloud are mapped back to the swallowing box cloud, and bilinear interpolation, cubic interpolation, nearest neighbor interpolation, or cubic convolution are performed to obtain the processed swallowing box cloud.

7. The method for synthesizing swallowing frame contour maps according to any one of claims 1 to 6, characterized in that, Determining the corresponding fusion factor for each pixel block in the processed swallowing bounding box includes: Traverse each pixel block in each processed swallowing box cloud map, and calculate the fusion factor of the current pixel block based on the first weight and the second weight corresponding to the current pixel block, and the weight coefficients corresponding to the first weight and the second weight respectively. The step of fusing the swallowing frame cloud map based on the pixel values ​​of each pixel block in the processed swallowing frame cloud map and the fusion factor corresponding to the pixel block to obtain a fused swallowing frame cloud map includes: Based on the pixel values ​​of the same position in the swallowing box cloud map after different processing, and the fusion factor corresponding to the pixel block, the fused pixel value of the pixel block at the current position is obtained by weighted summation; The fused swallowing box cloud map is obtained based on the pixel values ​​after fusion of all pixel blocks.

8. A swallowing frame cloud image synthesis device, characterized in that, include: The acquisition and processing module is used to acquire multiple swallowing box cloud maps obtained from swallowing data collection, scale the swallowing box cloud maps to a standard duration through mapping, and perform interpolation or cubic convolution processing to obtain processed swallowing box cloud maps. The fusion factor determination module is used to determine the corresponding fusion factor for each pixel block in the processed swallowing bounding box cloud map; wherein, the fusion factor is determined based on a first weight of the region type corresponding to the pixel block and a second weight of the processed swallowing bounding box cloud map to which the pixel block belongs; the region type includes feature points, target regions and other regions; The fusion module is used to perform cloud map fusion based on the pixel values ​​of each pixel block in the processed swallowing frame cloud map and the fusion factor corresponding to the pixel block, so as to obtain a fused swallowing frame cloud map.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the swallowing box cloud synthesis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein the computer program, when executed by a processor, implements the swallowing box cloud synthesis method as described in any one of claims 1 to 7.

Images