Ultrasonic stereo image reconstruction method, device and computer storage medium

Abstract

The present invention discloses a method, device, and computer storage medium for reconstructing ultrasonic stereo images. The method comprises: receiving two sets of sampled image sets; wherein the parts represented by the sampled image sets correspond to each other; using one of the two sampled image sets as a fixed image set and the other of the two sampled image sets as an active image set; performing translation transformation registration on the active image set; iteratively and sequentially modifying the depth parameters and plane parameters in the translation registration matrix to obtain a final calibration image set corresponding to the active image set; and performing matching fusion based on the final calibration image set and the fixed image set to obtain a fused stereo image. The ultrasonic stereo image reconstruction method provided by the present invention can simplify the operation logic and accelerate the registration speed and efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic imaging, and in particular to an ultrasonic stereoscopic image reconstruction method, device and computer storage medium. Background Art

[0002] Ultrasound imaging is widely used in industrial detection and even in technical fields such as medicine due to its advantages such as ease of operation, low cost, fast imaging speed, and low radiation exposure to the human body. Taking the medical field as an example, ultrasound imaging can be used to detect the condition of human organs such as the thyroid gland. Medical workers can hold the ultrasound probe and scan the patient's neck in a direction approximately perpendicular to the surface of the neck to obtain intermediate data for diagnosis, observation, or learning. While the above process requires medical workers to have a high level of experience in human anatomy, the ultrasound probe obtains a frame of image each time it passes through an anatomical location. Medical workers need to observe and determine the location of the lesion in real time, and cannot determine the global position of the lesion.

[0003] The prior art provides an ultrasound imaging solution that constructs three-dimensional data through volume reconstruction and then performs local adjustments and optimizations. However, this technical solution is prone to data accumulation and redundancy. Since the configuration idea of ​​global first and local later is adopted, it is easy to cause large errors and offsets in the final processed image, which damages the accuracy of the original data and makes it impossible to process and obtain a more accurate ultrasound image. Summary of the Invention

[0004] One of the purposes of the present invention is to provide an ultrasonic stereo image reconstruction method to solve the technical problems in the existing ultrasonic imaging solutions, such as the inability to take into account both the overall intuitiveness and the accuracy of the data, the complex processing steps and the low efficiency.

[0005] One of the objectives of the present invention is to provide an ultrasonic stereoscopic image reconstruction device.

[0006] One of the objects of the present invention is to provide a computer storage medium.

[0007] To achieve one of the above-mentioned objects of the invention, one embodiment of the present invention provides an ultrasound stereo image reconstruction method, comprising: receiving two groups of sampling image sets; wherein the parts represented by the sampling image sets correspond to each other; using one of the two groups of sampling image sets as a fixed image set, and using the other of the two groups of sampling image sets as an active image set, performing translation transformation alignment on the active image set, iteratively and sequentially modifying the depth parameters and plane parameters in the translation alignment matrix to obtain a final calibration image set corresponding to the active image set; performing matching fusion based on the final calibration image set and the fixed image set to obtain a fused stereo image.

[0008] As a further improvement of an embodiment of the present invention, the method specifically includes: performing translation transformation registration on the active image set, iterating and modifying the depth parameters in the translation registration matrix, iterating until the active image set meets the first preset condition, and obtaining a depth calibration image set corresponding to the active image set; performing translation transformation registration on each group of depth calibration images in the depth calibration image set, iterating and modifying the plane parameters in the translation registration matrix, iterating until the depth calibration image meets the second preset condition, and obtaining a final calibration image corresponding to the depth calibration image, thereby forming the final calibration image set.

[0009] As a further improvement of an embodiment of the present invention, the method specifically includes: calculating a first similarity parameter between the aligned active image set and the fixed image set, modifying the depth parameter according to the first similarity parameter, iterating until the first similarity parameter satisfies a preset first error condition, and obtaining a depth calibration image set corresponding to the active image set; calculating a second similarity parameter between the aligned depth calibration image and the fixed images in the corresponding fixed image set, modifying the plane parameter according to the second similarity parameter, iterating until the second similarity parameter satisfies a preset second error condition, and obtaining the final calibration image corresponding to the depth calibration image.

[0010] As a further improvement of an embodiment of the present invention, the method specifically includes: taking the fixed image set as a reference, performing a depth translation transformation on the active image set to obtain a depth transformation matrix; registering the active image set according to the depth transformation matrix to generate a depth registration image set, and performing resampling interpolation on the depth registration image set to obtain a depth interpolation image set; calculating the first similarity parameter according to the depth interpolation image set and the fixed image set; iteratively optimizing the depth parameter according to the first similarity parameter until the first similarity parameter converges to a first error interval to obtain a depth registration matrix corresponding to the translation registration matrix; calibrating the active image set according to the depth registration matrix to obtain the depth calibrated image set.

[0011] As a further improvement of an embodiment of the present invention, the first similarity parameter is a normalized cross-correlation coefficient, and the resampling interpolation includes linear interpolation.

[0012] As a further improvement of an embodiment of the present invention, the fixed image set and the depth interpolation image set respectively include a first fixed pixel and a first depth pixel corresponding to each other, and a second fixed pixel and a second depth pixel; the method specifically includes: traversing and calculating the fixed average grayscale value of the fixed image set and the depth average grayscale value of the depth interpolation image set; calculating the difference between the grayscale value of the first fixed pixel and the fixed average grayscale value, and the first product value of the difference between the grayscale value of the first depth pixel and the depth average grayscale value, and the second product value of the difference between the grayscale value of the second fixed pixel and the fixed average grayscale value, and the second product value of the difference between the grayscale value of the second depth pixel and the depth average grayscale value, and calculating the difference between the grayscale value of the second fixed pixel and the fixed average grayscale value and the second product value of the difference between the grayscale value of the second depth pixel and the depth average grayscale value according to the first pixel; The method further comprises: calculating a first similarity parameter of the first similarity parameter by calculating a sum of the product value and the second product value; calculating a first sum value of the square of the difference between the grayscale value of the first fixed pixel and the fixed average grayscale value and the square of the difference between the grayscale value of the second fixed pixel and the fixed average grayscale value, and a second sum value of the square of the difference between the grayscale value of the first depth pixel and the depth average grayscale value and the square of the difference between the grayscale value of the second depth pixel and the depth average grayscale value, and calculating a second similarity parameter of the first similarity parameter according to the product of the arithmetic square root of the first sum value and the arithmetic square root of the second sum value; and calculating the first similarity parameter according to the quotient of the first similarity parameter and the second similarity parameter.

[0013] As a further improvement of an embodiment of the present invention, the method specifically includes: using at least one of a gradient descent method and a Newton gradient method to iteratively optimize the depth parameter according to the first similarity parameter.

[0014] As a further improvement of an embodiment of the present invention, the method specifically includes: taking the fixed image in the fixed image set as a reference, performing a plane translation transformation on the corresponding depth calibration image in the depth calibration image set to obtain a plane transformation matrix; aligning the depth calibration image according to the plane transformation matrix to generate a plane registration image, and performing resampling interpolation on the plane registration image to obtain a plane interpolation image; calculating the second similarity parameter based on the plane interpolation image and the fixed image; iteratively optimizing the plane parameter according to the second similarity parameter until the second similarity parameter converges to a second error interval to obtain a plane registration matrix corresponding to the translation registration matrix; calibrating each group of depth calibration images in the depth calibration image set according to the plane registration matrix to obtain the final calibration image and form the final calibration image set.

[0015] As a further improvement of an embodiment of the present invention, the second similarity parameter is a grayscale gradient difference coefficient, and the resampling interpolation includes linear interpolation.

[0016] As a further improvement of an embodiment of the present invention, the method specifically includes: calculating the gradient difference between the planar interpolation image and the fixed image in a first direction and a second direction, respectively, to obtain gradient difference image data; wherein the gradient difference image data includes a first gradient difference grayscale value corresponding to the first direction, and a second gradient difference grayscale value corresponding to the second direction; and calculating the second similarity parameter based on the first gradient difference grayscale value and the second gradient difference grayscale value.

[0017] As a further improvement of an embodiment of the present invention, the fixed image and the plane interpolation image respectively include a first fixed pixel and a first interpolation pixel, and the first fixed pixel and the first interpolation pixel correspond to each other and correspond to the first gradient pixel in the gradient difference image data; the method specifically includes: calculating and performing a difference operation based on the partial differential parameter of the grayscale value of the first fixed pixel in the first direction and the partial differential parameter of the grayscale value of the first interpolation pixel in the first direction to obtain the first gradient difference grayscale value; calculating and performing a difference operation based on the partial differential parameter of the grayscale value of the first fixed pixel in the second direction and the partial differential parameter of the grayscale value of the first interpolation pixel in the second direction to obtain the second gradient difference grayscale value; traversing the gradient difference image The first gradient variance and the second gradient variance are obtained by respectively calculating the variances of all first gradient difference grayscale values ​​and all second gradient difference grayscale values ​​in the data; the first gradient parameter corresponding to the first gradient pixel is calculated using the first gradient variance as the dividend and the sum of the first gradient variance and the square of the first gradient difference grayscale value as the divisor; the second gradient parameter corresponding to the first gradient pixel is calculated using the second gradient variance as the dividend and the sum of the second gradient variance and the square of the second gradient difference grayscale value as the divisor; all gradient pixels in the gradient difference image data are traversed to calculate all first gradient parameters and all second gradient parameters, and the second similarity parameter is calculated based on the sum of all first gradient parameters and the sum of all second gradient parameters.

[0018] As a further improvement of an embodiment of the present invention, the method specifically includes: performing ultrasonic symmetrical scanning on the part to be measured to obtain two sets of original image sets; and resampling the original image sets to obtain two sets of sampled image sets.

[0019] As a further improvement of an embodiment of the present invention, the method specifically includes: resampling the original image set according to size parameters of the original planar images in the original image set, and correspondingly obtaining two groups of sampled image sets.

[0020] As a further improvement of an embodiment of the present invention, the method specifically includes: calculating the target total number of frames based on the inverse of the pixel height value of a single pixel in the original plane image and the ultrasonic scanning distance in the depth direction; calling a preset rounding operation function to calculate the image insertion order data based on the actual total number of frames of the original plane image and the target total number of frames; calculating the resampled depth coordinate value corresponding to the image insertion order data based on the ultrasonic scanning distance and the target total number of frames; calculating the uplink weight parameter and the downlink weight parameter based on the resampled depth coordinate value and the image insertion order data, and combining the original grayscale value of the pixel in the original plane image to calculate the resampled grayscale value corresponding to each pixel to form a resampled plane image; rearranging the corresponding resampled plane images according to the resampled depth coordinate value to form the sampling image set.

[0021] As a further improvement of an embodiment of the present invention, the image insertion order data includes a first insertion index value and a second insertion index value, wherein the difference between the second insertion index value and the first insertion index value is equal to 1; the method specifically includes: taking the quotient of the difference between the actual total number of frames of the original plane image and 1 and the difference between the target total number of frames and 1, and the product value of the preset order index value as the independent variable of the rounding operation function, to calculate the first insertion index value.

[0022] As a further improvement of an embodiment of the present invention, the original grayscale value includes a first original grayscale value corresponding to the first insertion index value, and a second original grayscale value corresponding to the second insertion index value; the method specifically includes: calculating the image resampling position data according to the order index value and the resampling depth coordinate value; calculating the uplink weight parameter according to the image resampling position data and the difference between the product of the first insertion index value and the quotient of the ultrasound scanning distance and the actual total number of frames minus 1 and the second insertion index value, and the difference between the image resampling position data; taking the product of the downlink weight parameter and the first original grayscale value and the sum of the product of the uplink weight parameter and the second original grayscale value as the first resampling parameter, taking the sum of the uplink weight parameter and the downlink weight parameter as the second resampling parameter, and calculating the resampled grayscale value according to the quotient of the first resampling parameter and the second resampling parameter to form the resampled planar image.

[0023] As a further improvement of one embodiment of the present invention, the translation registration matrix corresponds to the fixed registration matrix and the active registration matrix of the fixed image and the active image respectively, the fixed registration matrix includes a fixed depth parameter and a fixed plane parameter, the active registration matrix includes an active depth parameter and an active plane parameter, the sum of the fixed depth parameter and the active depth parameter is equal to 1, and the sum of the fixed plane parameter and the active plane parameter is equal to 1; the method specifically includes: performing corresponding translation transformation registration on the fixed image set and the active image set, iteratively and sequentially modifying the fixed depth parameter and the fixed plane parameter in the fixed registration matrix, as well as the active depth parameter and the active plane parameter in the active registration matrix, to obtain a final matching image set corresponding to the fixed image set, and a final calibration image set corresponding to the active image set; performing matching fusion according to the final calibration image set and the final matching image set to obtain the fused stereo image.

[0024] To achieve one of the above-mentioned objectives of the invention, one embodiment of the present invention provides an ultrasonic stereoscopic image reconstruction device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the steps of the ultrasonic stereoscopic image reconstruction method described in any of the above-mentioned technical solutions when executing the program stored in the memory.

[0025] To achieve one of the above-mentioned objectives of the invention, an embodiment of the present invention provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the ultrasound stereo image reconstruction method as described in the above-mentioned technical solution are implemented.

[0026] Compared with the existing technology, the ultrasound stereo image reconstruction method provided by the present invention processes two sets of sampled images in comparison with each other, and performs translation transformation alignment on at least one set of the sampled image sets, and performs alignment operations between the two sampled image sets in sequence from the depth and plane dimensions, thereby obtaining image set pairs with a higher degree of matching. In addition, due to the translation transformation alignment operation, the operation logic can be simplified, the calculation speed and efficiency can be accelerated, and the content of the original data in the sampled image set can be retained to the greatest extent. After the final matching fusion is performed, a stereo image can be reconstructed to assist medical workers or other field workers in knowing the specific location of the area of ​​interest. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 2 is a schematic structural diagram of an ultrasonic stereoscopic image reconstruction device in one embodiment of the present invention.

[0028] Figure 2 1 is a schematic diagram of the steps of an ultrasonic stereo image reconstruction method according to one embodiment of the present invention.

[0029] Figure 3 It is a schematic diagram of a fixed image set, a moving image set and a fused stereo image in a process of executing an ultrasound stereo image reconstruction method in one embodiment of the present invention.

[0030] Figure 4 1 is a schematic diagram of the steps of an ultrasound stereo image reconstruction method in another embodiment of the present invention.

[0031] Figure 5 FIG. 1 is a schematic diagram of some steps of a first embodiment of an ultrasound stereo image reconstruction method in another embodiment of the present invention.

[0032] Figure 6 It is a schematic diagram of some steps of a specific example of the first embodiment of the ultrasound stereo image reconstruction method in another embodiment of the present invention.

[0033] Figure 7 Schematic diagram of a sampling image set and its orientation during execution of an ultrasound stereo image reconstruction method in another embodiment of the present invention.

[0034] Figure 8 FIG. 1 is a schematic diagram of some steps of a second embodiment of an ultrasound stereo image reconstruction method in another embodiment of the present invention.

[0035] Figure 9 It is a schematic diagram of some steps of a specific example of a second embodiment of an ultrasound stereo image reconstruction method in another embodiment of the present invention.

[0036] Figure 10 FIG. 4 is a schematic diagram of a fixed image and a moving image during an ultrasonic stereo image reconstruction method according to another embodiment of the present invention.

[0037] Figure 11 FIG. 1 is a schematic diagram of the steps of an ultrasonic stereo image reconstruction method in yet another embodiment of the present invention.

[0038] Figure 12 FIG. 1 is a schematic diagram of some steps of an embodiment of an ultrasound stereo image reconstruction method in yet another embodiment of the present invention.

[0039] Figure 13 It is a schematic diagram of some steps of a specific example of an embodiment of an ultrasound stereo image reconstruction method in yet another embodiment of the present invention.

[0040] Figure 14 3 is a schematic diagram of the steps of an ultrasound stereo image reconstruction method in another embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0042] It should be noted that the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In addition, the terms "first," "second," etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance.

[0043] One embodiment of the present invention is to solve the problem that the ultrasound imaging solution in the prior art cannot take into account both the overall intuitiveness and the accuracy of the data, and the processing steps are complex and inefficient, as well as other technical problems derived from this problem, and provide a computer storage medium. The computer storage medium can be set in a computer and store a computer program. The computer storage medium can be any available medium that can be accessed by the computer, or it can be a storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium such as a floppy disk, a hard disk, a tape, etc., or an optical medium such as a DVD (Digital Video Disc, high-density digital video disc), or a semiconductor medium such as an SSD (Solid State Disk, solid-state drive). When the computer program is executed by any processor in the computer, an ultrasound stereo image reconstruction method is implemented to perform the reception of a sampled image set, the modification of the depth parameters and plane parameters in the translation registration matrix, the generation of a final calibration image set, and the generation of a fused stereo image. Thus, the above technical problems are solved.

[0044] An embodiment of the present invention further provides an ultrasonic stereo image reconstruction device 100, such as Figure 1 The ultrasound stereo image reconstruction device 100 includes a processor 11, a communication interface 12, a memory 13, and a communication bus 14. The processor 11, the communication interface 12, and the memory 13 communicate with each other via the communication bus 14.

[0045] The memory 13 is used to store computer programs; the processor 11 is used to execute the program stored in the memory. The program may be the computer program stored on a computer-readable storage medium as described above. When executing this program, the processor 11 can implement the steps of a method for reconstructing ultrasound stereo images, including receiving a set of sampled images, modifying depth parameters and plane parameters in a translation registration matrix, generating a final set of calibrated images, and generating a fused stereo image. This solves the aforementioned technical problem.

[0046] Continue as Figure 2 As shown, one embodiment of the present invention provides a method for reconstructing stereoscopic ultrasound images. The corresponding program or instructions of this method can be installed in the aforementioned computer storage medium and / or the stereoscopic ultrasound image reconstruction device 100 to achieve the technical effect of stereoscopic ultrasound image reconstruction. In this embodiment, the method specifically includes the following steps.

[0047] Step 21: Receive two sets of sampled images.

[0048] In step 22, one of the two sampled image sets is used as the fixed image set, and the other of the two sampled image sets is used as the active image set. A translation transformation registration is performed on the active image set, and the depth parameters and plane parameters in the translation registration matrix are modified iteratively and in sequence to obtain the final calibrated image set corresponding to the active image set.

[0049] Step 23: performing matching fusion based on the final calibration image set and the fixed image set to obtain a fused stereo image.

[0050] Among them, the parts represented by the sampling image sets correspond to each other. In this way, the two sets of sampling image sets can be made to correspond to each other and form a reference, and one of them can be registered with the other one, thereby forming a fused stereo image of the "represented part", assisting medical workers to understand the overall tissue structure of the part, and based on this, observe the position and morphology of the region of interest in the corresponding tissue structure. When the region of interest is a lesion such as a tumor, its relative position can be accurately located. Since the above technical solution completes the calibration and modification of the depth parameters and the plane parameters in sequence, it can quickly determine the mutually matching sampling images, thereby adjusting the registration relationship between the corresponding sampling images, and improving the registration success rate, preventing the constructed fused stereo image from having display errors such as jaggedness and dislocation, which affects the accuracy of the position and morphology judgment of the region of interest.

[0051] Each set of sampled image sets contains multiple sampled images, and each sampled image can be a two-dimensional image data that records the cross-sectional conditions of the tissue structure of the corresponding part. The sampled images are combined to form the sampled image set, which also constitutes the three-dimensional volume data of the tissue structure of the corresponding part. After the sampled image set is registered at the depth parameter level, an order matching relationship between different active images and different fixed images in the active image set and the fixed image set can be established; after the sampled image set is registered at the plane parameter level, a display matching relationship between each active image and each fixed image in the active image set and the fixed image set can be established. After completing the calibration of the two sets of volume data, matching fusion is performed, and a fused stereoscopic image that records the complete tissue structure characteristics of the corresponding part can be finally generated. The technical solution provided in this embodiment can be specifically applied to the detection of the human thyroid part. If the human thyroid part or its internal lesions are used as the above-mentioned region of interest, after the above-mentioned steps are performed, a complete three-dimensional model of the thyroid gland and its surrounding tissues can be finally generated.

[0052] Specifically, if Figure 3 As shown, one of the two groups of sampling image sets is defined as a fixed image set 61, and the other is defined as a moving image set 62. The two groups respectively form two groups of volume data representing different directions, angles or observation sides of the corresponding parts. Through the above-mentioned technical solution provided in this embodiment, the two groups can be aligned to finally form a fused stereo image 63 to represent the complete tissue structure of the part.

[0053] In another embodiment, the present invention provides an ultrasound stereo image reconstruction method based on the above embodiment, such as Figure 4 In this other embodiment, the method may specifically include the following steps.

[0054] Step 21: Receive two sets of sampled images.

[0055] Step 22: Using one of the two sampled image sets as the fixed image set and the other of the two sampled image sets as the active image set, perform translation transformation registration on the active image set, iteratively and sequentially modify the depth parameters and plane parameters in the translation registration matrix to obtain the final calibration image set corresponding to the active image set. Step 22 specifically includes:

[0056] Step 221 , performing translation transformation registration on the active image set, iterating and modifying the depth parameters in the translation registration matrix, iterating until the active image set meets a first preset condition, and obtaining a depth calibration image set corresponding to the active image set;

[0057] Step 222: Perform translation transformation registration on each group of depth calibration images in the depth calibration image set, iterate and modify the plane parameters in the translation registration matrix, iterate until the depth calibration image meets the second preset condition, obtain the final calibration image corresponding to the depth calibration image, and form a final calibration image set.

[0058] Step 23: performing matching fusion based on the final calibration image set and the fixed image set to obtain a fused stereo image.

[0059] In this way, the dependence on external training data input is reduced, and there is no need for human intervention to make adjustments. The entire registration process can be completed simply by relying on the sampled image set; each iteration only modifies at most one parameter in the translation registration matrix, which can reduce the amount of calculation and time consumption, and assist the iterative process to proceed quickly; the registration process uses a fixed image set as a benchmark to prevent the increase in calculation amount caused by algorithms such as bilateral approximation, and the resulting increase in errors.

[0060] Other parameters in the translation registration matrix can be set according to actual needs to meet the needs of image translation transformation. In one embodiment, the elements on the main diagonal of the translation registration matrix can be set to 1, and the last column of elements in the translation registration matrix can be arranged from top to bottom with directional parameters such as plane parameters and depth parameters. Other elements of the translation registration matrix can be set to 0, thereby simplifying the calculation process. Specifically, the plane parameter is defined as t x , the degree parameter is t y , the depth parameter is tz, then the translation registration matrix M can at least satisfy:

[0061]

[0062] It is worth noting that, on the one hand, although the above expression gives a 4×4 matrix form, the present invention is not limited to this matrix form, and the translation registration matrix can also be configured as a higher-order matrix or a lower-order matrix; on the other hand, for the plane parameter t x , degree parameter t y And the definition of depth parameter tz can be further referred to Figure 7 shown.

[0063] The sampling image set represents at least part of the volume data of the corresponding part, and a spatial rectangular coordinate system (or Cartesian coordinate system) is established with a vertex in the sampling image set as the coordinate origin O. In the context of thyroid detection, the technical solution provided in this embodiment is applied to the environment of thyroid detection. Correspondingly, the third coordinate axis Z can represent the extension direction of the neck, that is, extending from the head to the shoulder or in the opposite direction thereof, the first coordinate axis X can represent the front-to-back extension direction of the neck, that is, extending from the nuchal ligament to the thyroid cartilage or in the opposite direction thereof, and the second coordinate Y can represent the inner and outer extension direction of the neck, that is, extending from the external jugular vein to the thyroid cartilage or in the opposite direction thereof. It can be understood that the first coordinate axis X, the second coordinate axis Y and the third coordinate axis Z are arranged perpendicular to each other.

[0064] Based on this, the plane parameter t x Characterizes the registration deviation between the active image (set) and the fixed image (set) in the X direction of the first coordinate axis, that is, whether the position pointed by the active image and the position pointed by the fixed image are offset in the front-back direction of the neck and the degree of offset; the degree parameter t y Characterizes the registration deviation of the active image (set) and the fixed image (set) in the Y direction of the second coordinate axis, that is, the degree to which the medical worker presses the neck or other parts to be tested. It can be assumed that the pressing degree of the active image and the fixed image is equal. In this case, there is no need to adjust the degree parameter t y For registration, we can set the degree parameter t y =0; the depth parameter tz represents the registration deviation between the active image (set) and the fixed image (set) in the direction of the third coordinate axis Z, that is, whether the position pointed by the active image and the position pointed by the fixed image are staggered in the length extension direction of the neck part and the degree of staggered.

[0065] For the first preset condition and the second preset condition, in a preferred embodiment, the similarity between the active image and the fixed image after translation transformation alignment can be specifically judged until the similarity falls within the preset error allowable range, and the depth calibration image set and the final calibration image set can be iterated and finally formed.

[0066] Specifically, the iterative modification of the depth parameters provided in step 221 may specifically include the steps of calculating a first similarity parameter between the registered active image set and the fixed image set, modifying the depth parameter according to the first similarity parameter, iterating until the first similarity parameter satisfies a preset first error condition, and obtaining a depth calibration image set corresponding to the active image set.

[0067] The iterative modification of the plane parameters provided in step 222 may specifically include the steps of calculating a second similarity parameter between the aligned depth calibration image and a fixed image in the corresponding fixed image set, modifying the plane parameters according to the second similarity parameter, iterating until the second similarity parameter satisfies a preset second error condition, and obtaining the final calibration image corresponding to the depth calibration image.

[0068] Based on the first embodiment of the above-mentioned implementation scheme, the present invention provides a method for reconstructing ultrasonic stereo images, which refines the process of iteratively modifying the depth parameters in the translation registration matrix, that is, it provides a technical solution for aligning the deviation of the active image (set) and the fixed image (set) in the direction of the extension of the neck length. This first embodiment includes Figure 4 Step 21, step 22, and step 23 in the embodiment of the present invention are performed, and step 22 specifically includes step 221 and step 222, which will not be repeated here. For step 221 in the first embodiment, it can specifically include Figure 5 Follow the steps below.

[0069] Step 2211 , based on the fixed image set, perform depth translation transformation on the moving image set to obtain a depth transformation matrix.

[0070] Step 2212 : registering the active image set according to the depth transformation matrix to generate a depth-registered image set, and performing resampling interpolation on the depth-registered image set to obtain a depth-interpolated image set.

[0071] Step 2213: Calculate a first similarity parameter based on the depth interpolation image set and the fixed image set.

[0072] Step 2214 , iteratively optimize the depth parameter according to the first similarity parameter until the first similarity parameter converges to a first error interval, thereby obtaining a depth registration matrix corresponding to the translation registration matrix.

[0073] Step 2215: calibrate the active image set according to the depth registration matrix to obtain a depth calibrated image set.

[0074] In this way, through rigid registration, two sets of images representing corresponding body parts can be used as input to initially determine appropriate depth parameters. The algorithm, which combines resampling interpolation with similarity parameter calculation, better adapts to the discrete nature of image data and quantifies the registration data, facilitating evaluation and exiting the iterative process.

[0075] Specifically, the depth transformation matrix can be the plane parameter t of the translation transformation matrix x , degree parameter t yAmong the depth parameters tz, the other two parameters except the depth parameter tz are assigned to zero, thereby isolating the depth parameter tz for iterative calculation. Alternatively, after calculating the above three parameters, only the data of the depth parameter tz is retained, and the plane parameter t generated in the process is discarded. x and degree parameter t y Regarding the calculation process of the depth interpolation image set, in addition to the steps of alignment and resampling interpolation based on the active image set, in an embodiment where the active image set and the fixed image set are detected from two different sides of the neck region, the characteristic that the tissue of the neck region is approximately symmetrical can be utilized to perform mirror flipping processing on the active image set so that its general contour direction is consistent with that of the fixed image, thereby processing and obtaining the corresponding depth interpolation image set.

[0076] Based on this, after iteration, a depth parameter tz that meets the requirements of the first error interval can be obtained, that is, the depth parameter tz in the depth transformation matrix is ​​determined, and the corresponding depth registration matrix is ​​obtained, and the active image set is calibrated accordingly. Of course, if a mirror flipping process is used in the previous step to improve the efficiency of similarity judgment, then at step 2215, the calibrated active image set can also be mirror flipped again to obtain the depth calibration image set. Of course, for the depth parameter tz, it is also feasible to perform a mirror flipping process before calibrating the active image set.

[0077] In a preferred embodiment, the first similarity parameter is a normalized cross-correlation coefficient, which can better determine the similarity between the depth interpolation image set and the fixed image set. The resampling interpolation can preferably include linear interpolation. This is because the pixels in the image are discrete, the grayscale values ​​of the pixels of the active image are known in the initial state, and during the registration operation, each time the active image needs to be transformed according to a certain step size, the pixels moved to non-integer coordinates (based on the spatial rectangular coordinate system or Cartesian coordinate system described above) must be interpolated. Using simple linear interpolation can greatly speed up the operation and simplify the operation steps.

[0078] Preferably, the first error interval is not necessarily pre-set, so the loop exit condition provided in step 2214 can be interpreted as the first similarity parameter converges. However, in a specific scenario, the first error interval can be 0.01.

[0079] In a specific example based on the first embodiment above, Figure 5The step 2213 of calculating the similarity measure can be specifically refined as traversing the grayscale value of each pixel at each coordinate in the depth interpolation image set and the fixed image set, and calculating the first similarity parameter based on the grayscale value of the pixel of the depth interpolation image set and the fixed image set as a whole. For example, the fixed image set and the depth interpolation image set respectively include a first fixed pixel and a first depth pixel corresponding to each other, and a second fixed pixel and a second depth pixel. Then the specific example provided by the present invention can specifically include Figure 6 The following steps are shown.

[0080] Step 31 : traverse and calculate the fixed average grayscale value of the fixed image set and the depth average grayscale value of the first depth image set.

[0081] Step 32: Calculate a first product value of a difference between a grayscale value of a first fixed pixel and a fixed average grayscale value, and a first product value of a difference between a grayscale value of a first depth pixel and a depth average grayscale value, and a second product value of a difference between a grayscale value of a second fixed pixel and a fixed average grayscale value, and a first similarity parameter of a first similarity parameter based on the sum of the first product value and the second product value.

[0082] Step 33: Calculate a first sum of the square of the difference between the grayscale value of the first fixed pixel and the fixed average grayscale value and the square of the difference between the grayscale value of the second fixed pixel and the fixed average grayscale value, and a second sum of the square of the difference between the grayscale value of the first depth pixel and the depth average grayscale value and the square of the difference between the grayscale value of the second depth pixel and the depth average grayscale value, and calculate a second similarity parameter of the first similarity parameter based on the product of the arithmetic square root of the first sum and the arithmetic square root of the second sum.

[0083] Step 34: Calculate a first similarity parameter based on the quotient of the first similarity parameter and the second similarity parameter.

[0084] In the above-mentioned spatial rectangular coordinate system, the grayscale value of the first depth pixel with the coordinate position (i, j, k) on the depth interpolation image set is defined as I M1 (i, j, k), the gray value of the first fixed pixel corresponding to the coordinate position (i, j, k) on the fixed image set is I F1 (i, j, k), where the three elements in (i, j, k) correspond to the first coordinate axis X, the second coordinate axis Y, and the third coordinate axis Z. The depth average grayscale value of the depth interpolation image set is The fixed average grayscale value on a fixed image set is Then the first similarity parameter S NNC(1) Can be configured to at least meet:

[0085]

[0086] The second similarity parameter S NNC(2) Can be configured to at least meet:

[0087]

[0088] Based on this, the first similarity parameter may be configured to at least satisfy:

[0089]

[0090] In this way, the (i, j, k) coordinate value can be used as the pixel index to traverse all depth pixels in the depth interpolation image set and all fixed pixels in the fixed image set, and finally calculate the first similarity parameter to evaluate the alignment performance of the current depth transformation matrix until the first similarity parameter meets the preset conditions.

[0091] Preferably, the iterative optimization process in step 2214 in any of the above technical solutions can be specifically configured as at least one of the gradient descent method and the Newton gradient method, that is, the step 2214 preferably includes: using at least one of the gradient descent method and the Newton gradient method to iteratively optimize the depth parameter according to the first similarity parameter.

[0092] The present invention provides a method for reconstructing an ultrasonic stereo image based on the second embodiment of the above-mentioned implementation mode, which refines the process of iteratively modifying the plane parameters in the translation registration matrix, that is, it provides a technical solution for registering the deviation in the front and back extension direction of the neck. In the present invention, preferably, Figure 4 The depth calibration image set generated in step 221 is used in conjunction with the fixed image set as input to improve the accuracy of registration. Of course, in other implementations, there may be a technical solution that uses a moving image set or a depth interpolation image in conjunction with the fixed image set as input. This second embodiment includes Figure 4 Step 21, step 22, and step 23 in the embodiment of the present invention are performed, and step 22 specifically includes step 221 and step 222, which will not be repeated here. For step 222 in the second embodiment, it can specifically include Figure 8 Follow the steps below.

[0093] Step 2221 : Taking a fixed image in the fixed image set as a reference, perform a plane translation transformation on a corresponding depth calibration image in the depth calibration image set to obtain a plane transformation matrix.

[0094] Step 2222: register the depth calibration image according to the plane transformation matrix to generate a plane registration image, and perform resampling interpolation on the plane registration image to obtain a plane interpolation image.

[0095] Step 2223: Calculate a second similarity parameter based on the plane interpolation image and the fixed image.

[0096] Step 2224 , iteratively optimize the plane parameters according to the second similarity parameter until the second similarity parameter converges to the second error interval, thereby obtaining a plane registration matrix corresponding to the translation registration matrix.

[0097] Step 2225 , calibrate each set of depth calibration images in the depth calibration image set according to the planar registration matrix to obtain a final calibration image and form a final calibration image set.

[0098] In this way, through rigid registration, a set of depth-calibrated images with modified depth parameters and a corresponding set of fixed images can be used as input to initially determine appropriate plane parameters. By combining resampling interpolation with similarity parameter calculation, this computational path better adapts to the image's inherent characteristics, facilitating parameter evaluation and exiting the loop.

[0099] Specifically, the plane transformation matrix preferably includes the depth parameter tz calculated in the previous step, but based on the default pressing degree is equal, the degree parameter t y Assign it to zero, so as to iteratively calculate the plane parameter t x , you can also calculate the degree parameter t y and plane parameter t x , but the degree parameter t is discarded when the plane registration matrix is ​​finally generated y In the calculation process of the final calibration image set, in addition to the steps of alignment and resampling interpolation based on the depth calibration image set, in an embodiment in which the active image set corresponding to the depth calibration image set and the fixed image set are detected from two different sides of the neck, the characteristic that the tissue of the neck is approximately symmetrical can be utilized to perform mirror flipping processing on the depth calibration image set so that its general contour direction is consistent with that of the fixed image, thereby obtaining the corresponding final calibration image set.

[0100] Based on this, the plane parameter t that meets the requirements of the second error interval can be obtained through iteration. x , that is, the plane parameter t in the plane transformation matrix is ​​determined x, obtain the corresponding plane registration matrix, and calibrate the depth calibration image (set) accordingly. Of course, if the mirror flipping process is used in the previous step to improve the efficiency of similarity judgment, then at step 2225, the calibrated depth calibration image can also be mirror flipped again to obtain the depth calibration image set. Of course, there is a technical solution that performs mirror flipping before calibrating the depth calibration image (set). In this case, the plane parameter t needs to be adjusted accordingly. x The numerical value (which may be positive or negative) is used to prevent the registration movement direction from being wrong. It can be seen that the present invention allows the adjustment of the sequence between steps.

[0101] In a preferred embodiment, the second similarity parameter is a grayscale gradient difference coefficient, thereby enabling better determination of the similarity between the planar interpolated image (set) and the fixed image (set). The resampling interpolation can preferably include linear interpolation to accommodate discrete pixel conditions in the image and the problem of coordinate deviation from integers caused by step size transformation, thereby significantly accelerating the computation and simplifying the computational steps.

[0102] Preferably, the second error interval is not necessarily pre-set, so the loop exit condition provided in step 2224 can be interpreted as the second similarity parameter converges. However, in a specific scenario, the second error interval can be 0.1.

[0103] In a specific example based on the first embodiment above, Figure 8 The step 2223 of calculating the similarity measure can be specifically refined to generate gradient difference image data representing the gradient difference based on the plane interpolation image and the fixed image, and then analyze the gradient difference image data to calculate the second similarity parameter. Therefore, the specific example provided by the present invention can specifically include Figure 9 The following steps are shown.

[0104] Step 41 : Calculate the gradient differences between the plane interpolation image and the fixed image in the first direction and the second direction respectively to obtain gradient difference image data.

[0105] Step 42: Calculate a second similarity parameter based on the first gradient difference grayscale value and the second gradient difference grayscale value.

[0106] The gradient difference image data includes a first gradient difference grayscale value corresponding to the first direction and a second gradient difference grayscale value corresponding to the second direction. This allows the similarity between the interpolated image and the fixed image to be calculated using the gradient differences in at least two directions. Because the object being processed is not volumetric data representing the corresponding part, but rather two-dimensional data representing a cross section, this technical solution allows for more accurate and rapid determination of the second similarity parameter, optimizing the computational process.

[0107] Furthermore, for example, the fixed image and the plane interpolation image respectively include a first fixed pixel and a first interpolation pixel, the first fixed pixel and the first interpolation pixel correspond to each other and correspond to the first gradient pixel in the gradient difference image data. Based on the specific example of the second embodiment described above, the following steps can be specifically provided to form a preferred technical solution, such as Figure 9 shown.

[0108] The step 41 may specifically include:

[0109] Step 411, calculating and performing a difference operation based on the partial differential parameter of the grayscale value of the first fixed pixel in the first direction and the partial differential parameter of the grayscale value of the first interpolation pixel in the first direction to obtain a first gradient difference grayscale value;

[0110] Step 412 , calculate and perform a difference operation based on the partial differential parameter of the grayscale value of the first fixed pixel in the second direction and the partial differential parameter of the grayscale value of the first interpolated pixel in the second direction to obtain a second gradient difference grayscale value.

[0111] A plane rectangular coordinate system is established with the same endpoint on the fixed image and the plane interpolation image as the origin (preferably the upper left corner of the image), and the grayscale value of the first fixed pixel with the coordinate position (u, v) on the fixed image is defined as I F1 (u, v), the gray value of the first interpolation pixel corresponding to the coordinate position (u, v) on the plane interpolation image is I M1 (u, v). It should be explained that the first interpolation pixel mentioned above refers only to the pixel on the plane interpolation image, and does not necessarily refer to the pixel obtained by interpolation. Define the direction u corresponding to the pixel row arrangement as the first direction, and define the direction v corresponding to the pixel column arrangement as the second direction. Then the first gradient difference grayscale value I DiffV (u,v) can at least satisfy:

[0112]

[0113] The second gradient difference grayscale value I DiffH (u,v) can at least satisfy:

[0114]

[0115] In this way, the gradient difference image data can be constructed, or at least the first gradient difference grayscale value and the second gradient difference grayscale value can be calculated. V represents the vertical direction, and H represents the horizontal direction. That is, performing a partial differential in the first direction u can obtain the gradient change in the vertical direction V, and performing a partial differential in the second direction v can obtain the gradient change in the horizontal direction H. a represents a coefficient factor for the gradient change, with an initial value of 0. Each iteration increases by a preset step size until the second similarity parameter reaches its maximum value or converges to the second error range, at which point the increment stops.

[0116] In addition, the step 42 may specifically include:

[0117] Step 421, traversing all first gradient difference grayscale values ​​and all second gradient difference grayscale values ​​in the gradient difference image data, respectively calculating variances to obtain first gradient variance and second gradient variance;

[0118] Step 422: Using the first gradient variance as a dividend and the sum of the first gradient variance and the square of the first gradient difference grayscale value as a divisor, calculate a first gradient parameter corresponding to the first gradient pixel;

[0119] Step 423: Using the second gradient variance as the dividend and the sum of the squares of the second gradient variance and the second gradient difference grayscale value as the divisor, calculate a second gradient parameter corresponding to the first gradient pixel.

[0120] Step 424 , traverse all gradient pixels in the gradient difference image data, calculate all first gradient parameters and all second gradient parameters, and calculate a second similarity parameter based on the sum of all first gradient parameters and the sum of all second gradient parameters.

[0121] In the XOY plane rectangular coordinate system corresponding to the above-mentioned space rectangular coordinate system, the gray value I of the first fixed pixel is used. F1 (u, v), the gray value of the first interpolated pixel I M1 The definition of (u, v) is as follows: DiffH The definition of (u, v), and the above-mentioned second gradient difference gray value I DiffV (u,v) and further define the first gradient variance as A h , define the second gradient variance as A v , then the first gradient parameter S GDiff(1) Can at least meet:

[0122]

[0123] The second gradient parameter S GDiff(2) Can at least meet:

[0124]

[0125] Thus, all gradient pixels in the gradient difference image (data) can be traversed to calculate the corresponding first gradient parameter and second gradient parameter. Further, combining the sum of the two, the second similarity parameter S can be calculated. GDiff , wherein the second similarity parameter S GDiff At least it can meet:

[0126]

[0127] Preferably, the iterative optimization process in step 2224 in any of the above technical solutions can be specifically configured as at least one of the gradient descent method and the Newton gradient method, that is, the step 2224 preferably includes: using at least one of the gradient descent method and the Newton gradient method to iteratively optimize the plane parameters according to the second similarity parameter.

[0128] In the process of executing the above steps, the image is reflected on the side of the corresponding image, and the image can be reflected as follows: Figure 10 As shown, the depth interpolated image 610 originates from the active image set, with edges extending along the first coordinate axis X and the second coordinate axis Y, respectively. After mirror flipping and resampling and interpolation, a plane interpolated image 610' is formed. The plane interpolated image can then be registered with a fixed image 620' in the fixed image set and similarity parameters calculated. The final calibration image corresponding to the plane interpolated image 610' is then matched and fused with the fixed image 620', thereby forming a portion of the fused stereo image, namely, the plane fused image 630 shown in the figure.

[0129] In another embodiment of the present invention, a method for reconstructing an ultrasonic stereo image is provided. Figure 11 In yet another embodiment, the method may specifically include the following steps.

[0130] Step 51: Perform ultrasonic symmetrical scanning on the part to be measured to obtain two sets of original image sets.

[0131] Step 52: resample the original image set to obtain two sets of sampled image sets.

[0132] In this way, we can take advantage of the nearly symmetrical structure of the neck and cervical region by acquiring ultrasound images from both sides, thereby generating a set of original images. Considering that the display quality of the original image set depends on uncertain factors such as the detection environment, resampling it can improve the image quality to a certain extent, thereby generating a set of sampled images for subsequent registration processing.

[0133] The set of original images obtained by performing ultrasonic symmetric scanning, each of which represents the cross-sectional tissue conditions of the corresponding part and has physical coordinate information. In this way, the physical coordinate information can be used to complete the subsequent resampling step. The physical coordinate information can include the interval between adjacent pixels, the physical width and height of the entire image (determined by the ultrasonic probe), and preferably also includes the pixel size of the image, that is, the number of pixels in the width and height directions of the image, in pixels.

[0134] In a preferred embodiment, step 52 may further include resampling the original image set according to size parameters of the original planar images in the original image set, thereby obtaining two corresponding sets of sampled images. The size parameters may be included in the physical coordinate information or be interpreted equivalently to the physical coordinate information.

[0135] Based on the above-mentioned embodiment, the present invention provides an embodiment to specifically and better implement the generation process of the above-mentioned sampling image set, such as Figure 11 and Figure 12 As shown, this embodiment may include Figure 11 All steps from step 51 to step 23 are not repeated here.

[0136] In this embodiment, step 52 may specifically include the following steps:

[0137] Step 521 : Calculate the target total number of frames according to the inverse of the pixel height value of a single pixel in the original planar image and the ultrasonic scanning distance in the depth direction.

[0138] Step 522 : Based on the actual total number of frames of the original planar image and the target total number of frames, a preset rounding operation function is called to calculate image insertion order data.

[0139] Step 523 : Calculate the resampled depth coordinate value corresponding to the image insertion order data according to the ultrasound scanning distance and the target total number of frames.

[0140] Step 524, based on the resampled depth coordinate value and the image insertion order data, calculate the upstream weight parameter and the downstream weight parameter, and combine the original grayscale value of the pixel in the original plane image to calculate the resampled grayscale value corresponding to each pixel to form a resampled plane image.

[0141] Step 525 : rearrange the corresponding resampled planar images according to the resampled depth coordinate values ​​to form the sampled image set.

[0142] For step 521, the target total number of frames can be defined as equal to the product of the pixel height value and the ultrasonic scanning distance in the depth direction (which can be the third coordinate axis Z direction). The reciprocal of the pixel height value can be calculated based on the physical coordinate information and can be specifically the quotient of the height pixel size of the original planar image and the physical height value of the original planar image. Based on this, the height pixel size is defined as height, the physical height value is defined as H (in millimeters), and the ultrasonic scanning distance is defined as scanDist. Then, the target total number of frames dstImageNum can at least meet the following requirements:

[0143]

[0144] For step 521, the image insertion order data represents the relative order of the position where the image generated after resampling needs to be inserted in the original plane image set. In a preferred embodiment, the image insertion order data includes a first insertion index value and a second insertion index value, wherein the difference between the second insertion index value and the first insertion index value is equal to 1. Based on this, step 522 can further specifically include the steps of: taking the quotient of the difference between the actual total number of frames of the original plane image and 1 and the difference between the target total number of frames and 1, and the product value of the preset order index value as the independent variable of the rounding operation function, and calculating the first insertion index value. The first insertion index value and the second insertion index value represent the position order of the image frames adjacent to each other on both sides of the resampled plane image to be inserted.

[0145] Specifically, the first insertion index value is defined as m1, the second insertion index value is defined as m2, the actual total number of frames is defined as srcImageNum, the preset order index value is defined as z, and the rounding operation function is defined as floor(). Then, the first insertion index value m1 and the second insertion index value m2 can at least satisfy the following conditions:

[0146]

[0147] The order index value z is an increasing integer with a value range of [0, dstImage-2]. During a loop iteration (preferably a for loop), the order index value z increases by 1 each time through the loop, correspondingly updating the first insertion index value m1 and the second insertion index value m2. Thus, the first insertion index value m1 has a value range of [0, srcImageNum-2], and the second insertion index value m2 has a value range of [0, srcImageNum-1]. Preferably, the rounding operation function floor() is a floor operator.

[0148] For step 523, the physical coordinate information may include plane coordinate values ​​spacingX and spacingY defined according to the first coordinate axis X, the second coordinate axis Y, and the third coordinate axis Z, respectively, in the spatial three-dimensional coordinate system defined above, and a resampled depth coordinate value spacingZ (in millimeters / pixel). The plane coordinate values ​​can be directly read or analyzed from the original plane image, and the resampled depth coordinate value spacingZ can be calculated based on the ultrasonic scanning distance and the target total number of frames. That is, using the above definition, the resampled depth coordinate value spacingZ can at least satisfy:

[0149]

[0150] Based on this, step 525 can be executed to insert or rearrange the calculated resampled plane images to form resampled three-dimensional volume data. The above-mentioned resampled plane images, which correspond to step 524, in a specific example of this embodiment, can specifically include the following: Figures 11 to 13 The steps shown in FIG. In this specific example, Figure 11 Steps 51 to 23 shown in FIG. Figure 12 Steps 521 to 525 are shown in FIG. 1 , which are not described in detail here. Step 524 may further include: Figure 13 The following steps are shown.

[0151] Step 5241, calculate the image resampling position data according to the order index value and the resampling depth coordinate value.

[0152] Step 5242 : Calculate the uplink weight parameter based on the difference between the image resampling position data and the product of the first insertion index value and the quotient of the ultrasound scanning distance and the actual total number of frames minus 1.

[0153] Step 5243 : Calculate the downlink weight parameter based on the product of the quotient of the ultrasound scanning distance and the actual total number of frames minus 1, the second insertion index value, and the difference between the image resampling position data.

[0154] In step 5244, the product of the downlink weight parameter and the first original grayscale value and the sum of the product of the uplink weight parameter and the second original grayscale value are used as the first resampling parameter, and the sum of the uplink weight parameter and the downlink weight parameter are used as the second resampling parameter. The resampled grayscale value is calculated based on the quotient of the first resampling parameter and the second resampling parameter to form a resampled planar image.

[0155] The original grayscale values ​​include first original grayscale values ​​corresponding to the first insertion index values ​​and second original grayscale values ​​corresponding to the second insertion index values.

[0156] Continuing with the above definitions of the order index value z and the resampled depth coordinate value spacingZ, the image resampled position data currentDist represents the physical distance of the inserted new resampled plane image of the zth frame relative to the scanning starting point, and can at least satisfy:

[0157] currentDist=spacingZ*z.

[0158] Based on this, following the above definition of the first insertion index value m1, the uplink weight parameter d1 represents the physical distance in the depth direction between the m1th frame image, which is adjacent to the new resampled planar image of the zth frame and closer to the scanning starting point, and the scanning starting point. Compared with the difference in the depth direction between the physical distance between the resampled planar image and the scanning starting point, that is, the distance between the m1th frame image and the new resampled planar image of the zth frame, can at least satisfy:

[0159]

[0160] Continuing with the above definition of the second insertion index value m2, the downstream weight parameter d2 represents the physical distance in the depth direction between the m2th frame image, which is adjacent to the new resampled planar image of the zth frame and farther away from the scanning starting point, and the scanning starting point. Compared with the difference in the depth direction between the resampled planar image and the scanning starting point, that is, the distance between the m2th frame image and the new resampled planar image of the zth frame, can at least satisfy:

[0161]

[0162] Thus, the uplink and downlink weight parameters that characterize the spacing situation can be used as grayscale weights to fuse the adjacent m1-th frame image and m2-th frame image, calculate the resampled grayscale value, and form a resampled plane image. Following the above definition of the uplink weight parameter d1 and the downlink weight parameter d2, and defining the grayscale values ​​of the pixels corresponding to each other on the m1-th frame image and the m2-th frame image, that is, the first original grayscale value and the second original grayscale value are grayValueM1 and grayValueM2 respectively, traverse all the pixel grayscale values ​​on the m1-th frame image and the m2-th frame image to perform operations, thereby obtaining all the resampled grayscale values. Based on this, the resampled grayscale value dstGrayValue can at least satisfy:

[0163]

[0164] In another embodiment of the present invention, a method for reconstructing an ultrasonic stereo image is provided. Figure 14 In another embodiment, the method may specifically include the following steps.

[0165] Step 21: Receive two sets of sampled images.

[0166] Step 22', performing corresponding translation transformation registration on the fixed image set and the active image set, iteratively and sequentially modifying the fixed depth parameters and fixed plane parameters in the fixed registration matrix, as well as the active depth parameters and active plane parameters in the active registration matrix, to obtain a final matching image set corresponding to the fixed image set, and a final calibration image set corresponding to the active image set.

[0167] Step 23 ′: performing matching fusion based on the final calibration image set and the final matching image set to obtain a fused stereo image.

[0168] The translation registration matrix corresponds to the fixed registration matrix and the active registration matrix of the fixed image and the active image respectively, the fixed registration matrix includes a fixed depth parameter and a fixed plane parameter, the active registration matrix includes an active depth parameter and an active plane parameter, the sum of the fixed depth parameter and the active depth parameter is equal to 1, and the sum of the fixed plane parameter and the active plane parameter is equal to 1.

[0169] In this way, the fusion effect between the image sets representing the three-dimensional data can be further improved. The translation amount in the direction of the first coordinate axis X (corresponding to the plane parameter) and / or the translation amount in the direction of the third coordinate axis Z (corresponding to the depth parameter) can be distributed to the active image set (and its derived image set) and the fixed image set, so that both are resampled, which can also improve the quality and display quality of the fused stereo image.

[0170] Taking the apportionment of the translation amount in the X direction of the first coordinate axis as an example, the fixed registration matrix is ​​defined as M1, the movable registration matrix is ​​defined as M2, and the movable plane parameter is defined as αt x , define the above activity depth parameter as αt z , then the fixed depth parameter is 0, and the fixed plane parameter is (1-α)t x , so that the fixed registration matrix M1 and the movable registration matrix M2 can at least satisfy:

[0171]

[0172]

[0173] Where α is a constant weight factor. Preferably, when t x <0, we can set α = 0.7 to increase the translation ratio in the registration process of the active image set and its derived image sets. x When α > 0, we can set α = 0.3 to increase the proportion of translation in the registration process of a fixed image set.

[0174] In summary, the ultrasound stereo image reconstruction method provided by the present invention processes two groups of sampled images in comparison with each other, and performs translation transformation alignment on at least one group of the sampled image sets, and performs alignment operations between the two sampled image sets in sequence from the depth and plane dimensions, so as to obtain image set pairs with higher matching degree. Moreover, due to the translation transformation alignment operation, the operation logic can be simplified, the calculation speed and efficiency can be accelerated, and the content of the original data in the sampled image set can be retained to the greatest extent. After the final matching fusion is performed, a stereo image can be reconstructed to assist medical workers or other field workers in knowing the specific location of the area of ​​interest.

[0175] It should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation method can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0176] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for reconstructing an ultrasonic stereo image, characterized in that: include: receiving two sets of sampled image sets, wherein the parts represented by the sampled image sets correspond to each other; Using one of the two sampled image sets as a fixed image set and the other of the two sampled image sets as an active image set, performing translation transformation registration on the active image set, calculating a first similarity parameter between the registered active image set and the fixed image set, iterating and modifying a depth parameter in a translation registration matrix according to the first similarity parameter, iterating until the first similarity parameter satisfies a preset first error condition, and obtaining a depth calibration image set corresponding to the active image set; the first similarity parameter is a normalized cross-correlation coefficient; performing translation transformation registration on each group of depth calibration images in the depth calibration image set, calculating a second similarity parameter between the registered depth calibration image and a corresponding fixed image in the fixed image set, iterating and modifying plane parameters in the translation registration matrix according to the second similarity parameter, iterating until the second similarity parameter satisfies a preset second error condition, obtaining a final calibration image corresponding to the depth calibration image, and forming a final calibration image set; wherein the second similarity parameter is a grayscale gradient difference coefficient; Matching fusion is performed based on the final calibration image set and the fixed image set to obtain a fused stereo image.

2. The ultrasonic stereo image reconstruction method according to claim 1, characterized in that: The method specifically includes: Taking the fixed image set as a reference, performing a depth translation transformation on the moving image set to obtain a depth transformation matrix; Registering the active image set according to the depth transformation matrix to generate a depth-registered image set, and performing resampling interpolation on the depth-registered image set to obtain a depth-interpolated image set; Calculating the first similarity parameter according to the depth interpolation image set and the fixed image set; Iteratively optimizing the depth parameter according to the first similarity parameter until the first similarity parameter converges to a first error interval, thereby obtaining a depth registration matrix corresponding to the translation registration matrix; The active image set is calibrated according to the depth registration matrix to obtain the depth calibrated image set.

3. The ultrasonic stereo image reconstruction method according to claim 2, wherein: The resampling interpolation includes linear interpolation.

4. The ultrasonic stereo image reconstruction method according to claim 3, characterized in that: The fixed image set and the depth interpolation image set respectively include first fixed pixels and first depth pixels, and second fixed pixels and second depth pixels corresponding to each other; the method specifically includes: Traversing and calculating the fixed average grayscale value of the fixed image set and the depth average grayscale value of the depth interpolation image set; Calculating a first product value of a difference between a grayscale value of the first fixed pixel and the fixed average grayscale value and a difference between a grayscale value of the first depth pixel and the depth average grayscale value, and a second product value of a difference between a grayscale value of the second fixed pixel and the fixed average grayscale value and a difference between a grayscale value of the second depth pixel and the depth average grayscale value, and calculating a first similarity parameter of the first similarity parameter based on a sum of the first product value and the second product value; Calculating a first sum of a square of a difference between a grayscale value of the first fixed pixel and the fixed average grayscale value and a square of a difference between a grayscale value of the second fixed pixel and the fixed average grayscale value, and a second sum of a square of a difference between a grayscale value of the first depth pixel and the depth average grayscale value and a square of a difference between a grayscale value of the second depth pixel and the depth average grayscale value, and calculating a second similarity parameter of the first similarity parameter based on a product of a square root of the first sum and a square root of the second sum; The first similarity parameter is calculated according to a quotient of the first similarity parameter and the second similarity parameter.

5. The ultrasonic stereo image reconstruction method according to claim 2, wherein: The method specifically includes: At least one of a gradient descent method and a Newton gradient method is used to iteratively optimize the depth parameter according to the first similarity parameter.

6. The ultrasonic stereo image reconstruction method according to claim 1, characterized in that: The method specifically includes: Taking a fixed image in the fixed image set as a reference, performing a plane translation transformation on a corresponding depth calibration image in the depth calibration image set to obtain a plane transformation matrix; Registering the depth calibration image according to the plane transformation matrix to generate a plane registration image, and performing resampling interpolation on the plane registration image to obtain a plane interpolation image; Calculating the second similarity parameter according to the plane interpolation image and the fixed image; Iteratively optimizing the plane parameters according to the second similarity parameter until the second similarity parameter converges to a second error interval, thereby obtaining a plane registration matrix corresponding to the translation registration matrix; Each group of depth calibration images in the depth calibration image set is calibrated according to the planar registration matrix to obtain the final calibration image and form the final calibration image set.

7. The ultrasonic stereoscopic image reconstruction method according to claim 6, characterized in that: The resampling interpolation includes linear interpolation.

8. The ultrasonic stereoscopic image reconstruction method according to claim 7, characterized in that: The method specifically includes: Calculating gradient differences between the planar interpolation image and the fixed image in a first direction and a second direction, respectively, to obtain gradient difference image data; wherein the gradient difference image data includes a first gradient difference grayscale value corresponding to the first direction and a second gradient difference grayscale value corresponding to the second direction; The second similarity parameter is calculated according to the first gradient difference grayscale value and the second gradient difference grayscale value.

9. The ultrasonic stereo image reconstruction method according to claim 8, characterized in that: The fixed image and the planar interpolation image respectively include a first fixed pixel and a first interpolation pixel, the first fixed pixel and the first interpolation pixel correspond to each other and correspond to a first gradient pixel in the gradient difference image data; the method specifically includes: Calculating and performing a difference operation based on the partial differential parameter of the grayscale value of the first fixed pixel in the first direction and the partial differential parameter of the grayscale value of the first interpolation pixel in the first direction to obtain the first gradient difference grayscale value; Calculating and performing a difference operation based on the partial differential parameter of the grayscale value of the first fixed pixel in the second direction and the partial differential parameter of the grayscale value of the first interpolation pixel in the second direction to obtain the second gradient difference grayscale value; Traversing all first gradient difference grayscale values ​​and all second gradient difference grayscale values ​​in the gradient difference image data, respectively calculating variances to obtain first gradient variances and second gradient variances; Taking the first gradient variance as a dividend and the sum of the first gradient variance and the square of the first gradient difference grayscale value as a divisor, calculate a first gradient parameter corresponding to the first gradient pixel; Using the second gradient variance as a dividend and the sum of the second gradient variance and the square of the second gradient difference grayscale value as a divisor, a second gradient parameter corresponding to the first gradient pixel is calculated; All gradient pixels in the gradient difference image data are traversed to obtain all first gradient parameters and all second gradient parameters, and the second similarity parameter is obtained by calculating the sum of all first gradient parameters and the sum of all second gradient parameters.

10. The ultrasonic stereo image reconstruction method according to claim 1, wherein: The method specifically includes: Perform ultrasonic symmetrical scanning on the part to be measured to obtain two sets of original image sets; The original image set is resampled to obtain two sets of sampled image sets.

11. The ultrasonic stereo image reconstruction method according to claim 10, characterized in that: The method specifically includes: The original image set is resampled according to the size parameters of the original planar images in the original image set, and two groups of sampled image sets are correspondingly obtained.

12. The ultrasonic stereo image reconstruction method according to claim 11, characterized in that: The method specifically includes: Calculating a target total number of frames according to the reciprocal of a pixel height value of a single pixel in the original planar image and the ultrasonic scanning distance in the depth direction; According to the actual total number of frames of the original planar image and the target total number of frames, calling a preset rounding operation function to calculate image insertion order data; Calculating a resampled depth coordinate value corresponding to the image insertion order data according to the ultrasonic scanning distance and the target total number of frames; Calculating an upstream weight parameter and a downstream weight parameter according to the resampled depth coordinate value and the image insertion order data, and calculating a resampled grayscale value corresponding to each pixel in combination with the original grayscale value of the pixel in the original planar image to form a resampled planar image; According to the resampled depth coordinate values, the corresponding resampled planar images are rearranged to form the sampled image set.

13. The ultrasonic stereo image reconstruction method according to claim 12, characterized in that: The image insertion order data includes a first insertion index value and a second insertion index value, wherein a difference between the second insertion index value and the first insertion index value is equal to 1; the method specifically includes: The quotient of the difference between the actual total number of frames of the original plane image and 1 and the difference between the target total number of frames and 1, and the product value of the preset order index value are used as the independent variables of the rounding operation function to calculate the first insertion index value.

14. The ultrasonic stereo image reconstruction method according to claim 13, characterized in that: The original grayscale values ​​include first original grayscale values ​​corresponding to the first insertion index values ​​and second original grayscale values ​​corresponding to the second insertion index values; the method specifically includes: Calculating image resampling position data according to the order index value and the resampling depth coordinate value; Calculating the uplink weight parameter according to the difference between the image resampling position data and the product of the first insertion index value and the quotient of the ultrasound scanning distance and the actual total number of frames minus 1; Calculating the downlink weight parameter according to the difference between the product of the quotient of the ultrasound scanning distance and the actual total number of frames minus 1 and the second insertion index value and the image resampling position data; The product of the downlink weight parameter and the first original grayscale value and the sum of the product of the uplink weight parameter and the second original grayscale value are used as the first resampling parameter, and the sum of the uplink weight parameter and the downlink weight parameter are used as the second resampling parameter. The resampled grayscale value is calculated based on the quotient of the first resampling parameter and the second resampling parameter to form the resampled planar image.

15. The ultrasound stereo image reconstruction method according to claim 1, wherein: The translation registration matrix corresponds to a fixed registration matrix and a moving registration matrix of the fixed image and the moving image, respectively, the fixed registration matrix includes a fixed depth parameter and a fixed plane parameter, the moving registration matrix includes a moving depth parameter and a moving plane parameter, the sum of the fixed depth parameter and the moving depth parameter is equal to 1, and the sum of the fixed plane parameter and the moving plane parameter is equal to 1; The method specifically includes: performing corresponding translation transformation registration on the fixed image set and the active image set, iteratively and sequentially modifying the fixed depth parameters and the fixed plane parameters in the fixed registration matrix, and the active depth parameters and the active plane parameters in the active registration matrix, to obtain a final matching image set corresponding to the fixed image set and a final calibration image set corresponding to the active image set; Matching fusion is performed according to the final calibration image set and the final matching image set to obtain the fused stereo image.

16. An ultrasonic stereo image reconstruction device, characterized in that: comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is configured to implement the steps of the ultrasonic stereoscopic image reconstruction method according to any one of claims 1 to 15 when executing the program stored in the memory.

17. A computer storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the ultrasonic stereoscopic image reconstruction method according to any one of claims 1 to 15 are implemented.

Images