An ultrasonic wide-field imaging method, apparatus and storage medium

Abstract

This application relates to an ultrasound wide-view imaging method, apparatus, and storage medium. The method includes: acquiring probe attitude information; determining the scanning plane and emission plane of the probe based on the attitude information, and determining a scanning path line and a scanning normal vector based on the scanning plane and the emission plane; receiving a stitching command, determining a target scanning path based on the scanning path line and the scanning normal vector, and acquiring multiple frames of ultrasound images along the target scanning path; and stitching adjacent frames of ultrasound images together to generate an ultrasound wide-view image. This method only acquires and stitches ultrasound images along the target scanning path, filtering out ultrasound images acquired along unplanned paths, ensuring the quality of the input image, and thus improving the quality of the ultrasound wide-view image.

Description

Technical Field

[0001] This application relates to the field of medical imaging technology, and in particular to an ultrasound wide-view imaging method, device and storage medium. Background Technology

[0002] Wide-field ultrasound imaging is a technique that stitches together multiple ultrasound images to generate a large field of view, providing more comprehensive lesion information in medical diagnosis and is widely used. However, the quality of wide-field ultrasound imaging is limited by the operator's difficulty in maintaining the correct posture and position of the probe during scanning, resulting in a lack of effective overlap between adjacent images, leading to poor image stitching and ultimately reducing the clinical value of wide-field imaging. Currently, various solutions have been proposed to improve the quality of wide-field ultrasound imaging, including integrating scanning target localization modules, coordinate transformation modules, posture localization modules, pose correction modules, force feedback correction modules, as well as robotic arms and matching fixtures to achieve precise positioning and posture control of the ultrasound probe. While this approach can improve scanning accuracy, it requires multiple sensor modules and complex hardware systems, resulting in high costs and operational complexity, making it difficult to popularize in ordinary medical institutions. Existing technologies also include magnetic navigation-based wide-field ultrasound imaging methods, which use magnetic sensors to bind the ultrasound probe and monitor its spatial position and posture in real time. While this method can improve the accuracy of image stitching to some extent, it requires additional magnetic navigation equipment, increasing system complexity and cost, and is susceptible to magnetic field interference, leading to measurement errors. Therefore, there is an urgent need for an efficient, low-cost, and accurate ultrasound wide-view imaging method that can simplify procedures in practical clinical applications and meet the needs of medical diagnosis. Summary of the Invention

[0003] Therefore, it is necessary to provide an ultrasonic wide-view imaging method, device, and storage medium to address the aforementioned technical problems.

[0004] An ultrasound wide-view imaging method, the method comprising: Acquire the probe's attitude information; The scanning surface and the emitting surface of the probe are determined based on the attitude information, and the scanning path line and the scanning normal vector are determined based on the scanning surface and the emitting surface. The system receives a stitching instruction, determines a target scanning path based on the scanning path line and the scanning normal vector, acquires multiple ultrasound images along the target scanning path, and stitches together adjacent ultrasound images to generate a wide-view ultrasound image.

[0005] An ultrasonic wide-view imaging device, the device comprising: The first acquisition unit is used to acquire the probe's attitude information; The first determining unit determines the scanning surface and the emitting surface of the probe based on the attitude information, determines the scanning path line and the scanning normal vector based on the scanning surface and the emitting surface, and determines the target scanning path based on the scanning path line and the scanning normal vector. The second acquisition unit is used to acquire ultrasound images on the target scanning path based on the received splicing instructions; An image generation unit is used to stitch together the ultrasound images to generate a wide-view ultrasound image.

[0006] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described ultrasonic wide-view imaging method.

[0007] This invention acquires the probe's attitude information, determines the target scanning path based on the probe's attitude information, and stitches together ultrasound images only along the target scanning path, filtering out ultrasound images along unplanned paths, thus ensuring the quality of the input image and improving the quality of the wide-view ultrasound image. At the same time, the target scanning path line provides users with clear scanning guidance, making the scanning process more efficient. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating an ultrasound wide-view imaging method in one embodiment; Figure 2 This is a flowchart illustrating step S106 in one embodiment; Figure 3 This is another flowchart illustrating step S106 in one embodiment; Figure 4 This is another schematic diagram of the ultrasound wide-view imaging method in one embodiment; Figure 5 This is a schematic diagram of a 3D image of a scanned scene in one embodiment; Figure 6 This is a schematic diagram of the structure of an ultrasonic wide-view imaging device in one embodiment; Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0010] Example 1 This embodiment provides an ultrasound wide-view imaging method, such as Figure 1 As shown, the method includes the following steps: S102 acquires the probe's attitude information.

[0011] In this embodiment, the probe's attitude information includes its position and orientation in three-dimensional space. Specifically, the probe's attitude information can be acquired using devices such as depth cameras, sensors, gyroscopes, and accelerometers. These devices can monitor the probe's position coordinates and angle information in real time. The probe's attitude information can be represented by a six-dimensional vector, including three position coordinates (x, y, z) and three rotation angles (α, β, γ), representing the probe's position in three-dimensional space and its rotation angles around the three coordinate axes, respectively.

[0012] S104 determines the scanning surface and emission surface of the probe based on the probe's attitude information, and determines the scanning path line and scanning normal vector based on the scanning surface and emission surface.

[0013] In this embodiment, based on the probe's scanning direction (usually determined by the probe's axis direction), the scanning surface is defined as the plane pointed to by the probe's axis direction, which can be calculated using the position coordinates and rotation angle from the probe's attitude information. Specifically, the scanning surface can be represented by a plane equation: ax + by + cz + d = 0, where (x, y, z) represent the coordinates of any point on the scanning surface in three-dimensional space, a, b, and c are the normal vectors of the scanning surface, and d is a constant term in the plane equation. The rotation angle (α0, β0, γ0) is used to calculate the probe axis direction vector n. Specifically, the rotation angle (α0, β0, γ0) is converted into the direction vector n = (a, b, c) using a rotation matrix or Euler angle formula.

[0014] In this embodiment, the emitting surface of the probe is the plane on the probe that contacts the area to be scanned and is used to emit ultrasonic waves (usually perpendicular to the probe axis). The direction of the probe axis is vector n. The emitting surface can be represented as a plane passing through a point P on the emitting surface of the probe and with normal vector n, where m is perpendicular to n. The emitting surface can be represented by another plane equation: ex + fy + gz + h = 0, where (x, y, z) represents the coordinates of any point on the emitting surface in three-dimensional space, e, f, and g are the normal vectors of the emitting surface, and h is the constant term of the plane equation.

[0015] In this embodiment, the specific method for determining the scanning path line based on the scanning surface and the emitting surface includes: determining the line intersecting the scanning surface and the emitting surface as the scanning path line. Both the scanning surface and the emitting surface are planes in three-dimensional space, and their intersection can be a straight line, which is the scanning path line. The straight line intersecting the scanning surface and the emitting surface is determined as the scanning path line. This straight line can be represented as a system of parametric equations: x = x0 + at, y = y0 + bt, z = z0 + ct, where (x0, y0, z0) are the intersection points of the scanning surface and the emitting surface, i.e., the initial position; (a, b, c) is the cross product of the scanning surface normal vector and the emitting surface normal vector, i.e., n × m, and t is a parameter.

[0016] The scan normal vector is a unit vector perpendicular to the emission surface and can be calculated using the plane equation of the emission surface. Specifically, if the plane equation of the emission surface is ex + fy + gz + h = 0, then the scan normal vector is... .

[0017] The scanning path can be represented as a series of pose information {(x0, y0, z0, a,b,c,α}}. i , β i , γ i )}, where (x0, y0, z0a,b,c) are the parameters of the equation of the line corresponding to the scan path, (α i , β i , γ i ) is the scan normal vector, that is, the scan path includes the scan path line and the scan normal vector.

[0018] S106 receives the stitching command, determines the target scanning path based on the scanning path line and scanning normal vector, acquires multiple ultrasound images on the target scanning path, stitches two adjacent ultrasound images together, and generates a wide-view ultrasound image.

[0019] In this embodiment, the target scanning path refers to the path that the probe should follow during the scanning process. It includes not only the probe's position trajectory (i.e., the scanning path line) but also the probe's orientation at each position (i.e., the scanning normal vector). In practical applications, before receiving the stitching command, the probe's attitude information is acquired in real time to generate the corresponding scanning path. Simultaneously, the probe emits and receives ultrasound waves to acquire the current scanning image. To obtain a high-quality scanning image, the user needs to continuously move and rotate the probe. During this process, the probe's attitude information will change, and the corresponding scanning path will also change continuously. When the user determines that the ultrasound image obtained at a certain time meets the requirements, the user will issue a stitching command. The scanning path at this time is the target scanning path, where the scanning path line at this time is the target scanning path line, and the scanning normal vector at this time is the target normal vector. That is, the scanning path line and scanning normal vector determined by the scanning surface and the emitting surface when receiving the stitching command are used as the target scanning path.

[0020] In this embodiment, upon receiving the user's stitching command, the ultrasound imaging device begins acquiring ultrasound images along the target scanning path. Specifically, the probe moves along the target scanning path while continuously emitting and receiving ultrasound waves, thus obtaining multiple frames of ultrasound images. These ultrasound images are then stitched together to generate a complete wide-view ultrasound image; the specific method of stitching will be described in detail later.

[0021] In this embodiment, as Figure 2 As shown, step S106 may include: After receiving the stitching command, S11 obtains the scanning path corresponding to each frame of ultrasound image.

[0022] S12 If the scanning path corresponding to the acquired ultrasound image matches the target scanning path, then the ultrasound image corresponding to that frame is the ultrasound image on the target scanning path.

[0023] S13 stitches together the ultrasound images on two adjacent target scanning paths to generate a wide-view ultrasound image.

[0024] Specifically, the ultrasound imaging equipment checks whether each frame of ultrasound image was acquired along the target scanning path; only those ultrasound images acquired along the target scanning path are used for stitching. This ensures the quality and accuracy of the generated wide-view ultrasound image. The stitching command can be issued by the user.

[0025] In actual scanning, the current scan image may not meet the requirements. At this time, the user may return to rescan. This method can handle the situation where the probe partially returns to rescan during the scanning process. When the probe moves along the target scanning path in the direction that has been scanned, it may acquire multiple ultrasound images at the same position. In order to avoid repeated stitching and to select the images obtained from the rescan for stitching, it is necessary to delete the previous repeated parts of the image.

[0026] To address the aforementioned problems, in this embodiment, as follows: Figure 3 As shown, step S106 further includes: S21 acquires ultrasound images along the target scanning path in real time, and stitches together adjacent ultrasound images to generate a pre-stitched image.

[0027] Specifically, ultrasound images along the target scanning path are acquired in real time. Feature points (such as edges and key points) are extracted from two adjacent frames for matching. Through feature point matching, the transformation matrix (such as translation, rotation, and scaling) between the two frames is calculated. The two adjacent frames are aligned according to the transformation matrix and fused. The images are then stitched together in real time to generate a pre-stitched image.

[0028] S22 acquires the probe's attitude information in real time and compares the distance between the intersection points of the current frame and the previous frame.

[0029] Specifically, during stitching, the probe's attitude information, including position (x, y, z) and orientation (rotation angle or orientation vector), is acquired in real time. Based on the attitude information, the spatial equation of the probe centerline (i.e., the probe's central axis) is calculated. Based on the probe centerline corresponding to the target scanning path line, the intersection point of the target scanning path line and the probe centerline is determined and used as a reference point, i.e., the initial intersection point. According to the probe's current attitude, the intersection point of the probe centerline and the target scanning path line corresponding to the current frame is calculated.

[0030] Calculate the distance between the current frame intersection point and the initial intersection point, denoted as d. N Calculate the distance between the intersection point of the previous frame and the initial intersection point, denoted as d. N-1 Compare d N and d N-1 If d N >d N-1 This indicates that the probe is moving away from the initial point; if d N <d N-1 This indicates that the probe is moving towards the initial point.

[0031] S23 If the intersection distance of the current frame is less than the intersection distance of the previous frame, the intersection distance of the current frame is compared with the intersection distance of the previous frames in turn until the intersection distance of the current frame is greater than the intersection distance of a certain frame, and then the comparison stops.

[0032] In the process of comparing the intersection distance of the current frame with the intersection distance of the previous frame, if the ultrasound imaging device detects an intersection distance d in the current frame... N Greater than the intersection distance d of the previous frame N-1 This indicates that the probe is moving away from the initial point; if the intersection distance d of the current frame is detected... N The intersection distance d of the previous frame is less than the previous frame N-1 This indicates that the probe is moving towards the initial point, and there is a possibility of it retracing and rescanning. This may be because the user is not satisfied with the current scanned image and wants to scan again. In this case, the intersection distances (e.g., d) of the previous frames will be read sequentially from the pre-stitched image. N-2 d N-3 d N-1 …d1), and the distance d from the intersection point with the current frame. N Compare the points. The distance d between the intersection points of the current frames. N Greater than the intersection distance d of a previous frame N-M If the distance between the intersection points of the current frames is d, then stop the comparison and record the position of that frame. N The intersection distance d of a previous frame is less than N-M Then continue comparing the intersection distance d in the next frame. N-M-1 The process continues until a frame that meets the conditions is found or all previous frames have been traversed, where N and M are both integers, and N > M. This process determines the position where the probe will rescan along the target scanning path, thus avoiding the repeated stitching of multiple frames at the same location.

[0033] S24 Delete the ultrasound images corresponding to frames whose intersection distance in the pre-stitched image is greater than the intersection distance of the current frame.

[0034] By comparing the intersection distance of the current frame with the intersection distances of all frames in the pre-stitched image, frames with intersection distances greater than the current frame's are identified. The ultrasound images corresponding to these frames are then removed from the pre-stitched image to avoid duplicate stitching or unsatisfactory images in the pre-stitched image. The process of stitching adjacent ultrasound images continues, updating the pre-stitched image.

[0035] S25 obtains a wide-view ultrasound image based on pre-stitched images.

[0036] Continue the above process, updating the pre-stitched image until stitching is complete, resulting in a wide-view ultrasound image.

[0037] In this embodiment, the stitching process in step S106 includes performing feature matching on two adjacent ultrasound images to determine matching points, determining an affine transformation matrix based on the matching points, using the affine transformation matrix to project the ultrasound images on the target scanning path onto the same coordinate system, and performing fusion through linear fusion.

[0038] Feature matching refers to finding corresponding feature points in two adjacent ultrasound images. These feature points are usually salient features in the image, such as edges and corners. Specifically, algorithms such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) can be used to extract image feature points, and then the RANSAC (Random Sample Consensus) algorithm is used for feature matching to find matching points in the two images. Determining the affine transformation matrix based on the matching points means calculating a transformation matrix based on the matching points in the two images. This matrix can transform one image to the coordinate system of the other image. The affine transformation matrix can be a 3×3 matrix and can represent transformations such as translation, rotation, and scaling. Specifically, assuming n pairs of matching points {(x... i , y i ),(x i ',y i Given ')}, i=1,2,...,n, where n is an integer, the affine transformation matrix A can be obtained by solving the following system of equations: [x i ' y i '1] T = A*[x i y i 1] T i=1,2,...,n An affine transformation matrix is ​​used to project ultrasound images along the target scanning path into the same coordinate system. Specifically, the calculated affine transformation matrix is ​​used to transform each frame of ultrasound image into the same coordinate system (i.e., the wide-view image coordinate system). In this way, all ultrasound images can be stitched together within the same coordinate system.

[0039] Fusion can be achieved through linear fusion, specifically by weighting the pixel values ​​of adjacent frames in the stitching region according to certain weights, thus achieving a smooth transition. Specifically, the pixel values ​​in the stitching region can be calculated using the following formula: I(x, y) = w1* I1(x, y) + w2* I2(x, y) Where I(x, y) is the pixel value at point (x, y) in the stitched image, I1(x, y) and I2(x, y) are the pixel values ​​at point (x, y) in the two adjacent original images, respectively, and w1 and w2 are weights, satisfying w1 + w2 = 1. The weights can be determined based on the position of point (x, y) in the stitching region, such that at the edge of the stitching region, the weight of one image is 1 and the weight of the other image is 0; in the middle of the stitching region, the weights of both images are 0.5.

[0040] After stitching together to generate a wide-view ultrasound image, as shown Figure 4 As shown, this embodiment also includes step S108, which optimizes the ultrasonic wide-view image using the beam adjustment method.

[0041] Bundle adjustment is a method for optimizing 3D reconstruction results. It can optimize the position and orientation of each frame of ultrasound image, making the stitched wide-view image more accurate.

[0042] Specifically, the objective function of the bundle adjustment method is to minimize the sum of squares of the reprojection errors: min Σ||X ij - P(X i C j )||² Among them, X ij X is the observed position of the i-th feature point in the j-th frame of the image. i C is the three-dimensional coordinate of the i-th feature point in the wide-view image. j is the probe pose information (including position and orientation) of the j-th frame image, and P is the projection function that projects the three-dimensional points onto the image plane.

[0043] After receiving the splicing instruction, such as Figure 4 As shown, this embodiment also includes the following steps: S107 compares the probe's current scanning path with the target scanning path and generates a probe adjustment prompt; wherein, the probe's current scanning path is determined by the probe's current attitude information.

[0044] In this embodiment, the current scanning path of the probe is determined by the probe's current attitude information. Specifically, the current scanning path of the probe is calculated based on the real-time acquired current attitude information and compared with the previously determined target scanning path. If a difference exists, the system generates a probe adjustment prompt, guiding the user to adjust the probe's position and orientation to align with the target scanning path.

[0045] Step S107 specifically includes the following: If the current scanning path matches the target scanning path, no probe adjustment prompt will be given, or the current scanning path line will be displayed in the first preset color.

[0046] If the current scanning path does not match the target scanning path, the current scanning path line will be displayed in the second preset color.

[0047] Specifically, when determining the current scanning path and current scanning normal vector based on the probe's current attitude information, the method is the same as that used to determine the scanning path and scanning normal vector, and will not be repeated here. If the current scanning path coincides with the target scanning path, and the current scanning normal vector is parallel to or coincides with the target scanning normal vector, it indicates that the current scanning path matches the target scanning path, meaning that the scanning direction and posture are correct. In this case, the probe adjustment prompt is not required, or the current scanning path is displayed in a first preset color. For example, in this embodiment, the first preset color is green to indicate that the current scanning path matches the target scanning path, thus indicating that the current scanning is correct. If the current scanning path does not coincide with the target scanning path, or if the current scanning normal vector is not parallel to the target scanning normal vector, it indicates that the current scanning path does not match the target scanning path, meaning that the scanning direction or posture needs to be adjusted. The current scanning path is displayed in a second preset color; wherein, the first preset color and the second preset color are different colors; for example, in this embodiment, the second preset color is red to indicate that the current scanning path does not match the target scanning path, so as to prompt that the current scanning needs to be adjusted.

[0048] Furthermore, the interface displays the target scanning path line and the current scanning path line. Specifically, before receiving the stitching command, the target scanning path line is displayed in a first preset form, such as a white dashed line; after receiving the stitching command, the target scanning path line is displayed in a second preset form, such as a blue solid line, and the current scanning path line is displayed in a third or fourth preset form. The third preset form can be a dashed line of the first preset color, and the fourth preset form can be a dashed line of the second preset color. This allows users to distinguish between the target scanning path line and the current scanning path line, and to quickly understand whether the current scanning status is correct.

[0049] In this embodiment, as Figure 4 As shown, before obtaining the probe's attitude information, the following steps are also included: Step S101: Acquire infrared and depth images of the scanning scene, and determine the three-dimensional image of the scanning scene and the probe's pose information based on the infrared and depth images.

[0050] Specifically, using an infrared camera to capture infrared images of the scanned scene allows for the clear display of object outlines, including at least the outlines of the probe and the area to be inspected, even in low-light conditions. Simultaneously, using an infrared camera to acquire depth images of the scanned scene, recording the distance from each point in the scene to the camera, provides three-dimensional information about the scene. Based on the infrared and depth images, a three-dimensional image of the scanned scene can be obtained through 3D reconstruction techniques, such as... Figure 5As shown, the 3D image displays the pose information of the probe and the area to be scanned. By using image processing and pattern recognition techniques, the probe can be located in the 3D image, and its position and orientation can be calculated to obtain the probe's pose information.

[0051] Preferably, a marker is placed on the probe. An active infrared structured light depth camera acquires infrared and depth images of the scanned scene. Based on these images, the pose information of the marker is obtained. The pose information of the probe is then determined based on the marker's pose information. The active infrared structured light depth camera includes an infrared floodlight, an infrared speckle projector, and an infrared camera for receiving light information. The floodlight and camera acquire infrared images of the scanned scene, while the speckle projector and camera acquire depth images. The marker can be a special pattern affixed to the probe or a specially shaped object attached to it, making it easily identifiable and trackable in the infrared and depth images. By identifying and tracking the marker, and based on its position in the infrared and depth images, combined with the intrinsic and extrinsic parameters of the depth camera, the three-dimensional coordinates of the marker can be determined. Then, based on the known relationship between the marker and the probe, the pose information of the probe can be calculated.

[0052] In practical applications, a marker can be a planar pattern containing multiple feature points, such as a black square, black dot, QR code, ArUco marker, or AprilTag marker. These markers can be accurately identified and located in infrared and depth images. By analyzing the marker's position and shape in the image, the marker's three-dimensional position and orientation can be calculated. Then, the PnP (Perspective-n-Point) algorithm is used to solve for the probe's attitude information (including position and rotation angle).

[0053] For example, assuming the marker is a black square, its 3D coordinates are calculated based on its position in the infrared and depth images, combined with the intrinsic and extrinsic parameters of the depth camera. The coordinates of its four corner points in 3D space are {(X1,Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), (X4, Y4, Z4)}, and these coordinates are relative to the marker's own coordinate system. In the infrared and depth images, the positions of these four corner points {(u1, v1), (u2, v2), (u3, v3), (u4, v4)} and their depth values ​​{d1, d2, d3, d4} can be found. By solving the Perspective-n-Point (PnP) problem, the marker's position and orientation relative to the camera can be calculated. Then, based on the known relationship between the marker and each point on the probe, the probe's position and orientation relative to the camera, i.e., the probe's attitude information, can be calculated. Assuming there are three black circles marked on the probe handle and three black circles marked on the transducer housing, the position and outline of the black circles can be detected in the infrared image through circle detection, template matching, or feature detection algorithms (such as SIFT or ORB). Then, based on the position of the black circles in the infrared and depth images, combined with the intrinsic and extrinsic parameters of the depth camera, the three-dimensional coordinates of the black circles can be calculated.

[0054] In this embodiment, after receiving the splicing instruction, the process further includes: Step S109 Displays the target scanning path line corresponding to the target scanning path and the current scanning path line corresponding to the current scanning path in different forms on the three-dimensional image, wherein the current scanning path is determined by the current attitude information of the probe.

[0055] Specifically, such as Figure 5 As shown, the 3D image can intuitively display the three-dimensional structure of the scanning scene, including the scanning surface A and the emitting surface B. The target scanning path line L1 and the current scanning path line L2 are also displayed in different forms on the 3D image, helping users better understand and adjust the probe's position and orientation. For example, after receiving a stitching command, the target scanning path line L1 on the 3D image changes from a white dashed line to a blue solid line, making it easier for users to clearly identify the target scanning path line L1. The position of this line does not change, which can be understood as the acceptance of the stitching command indicating that the current scanning path line is the target scanning path line L1 desired by the user, serving as a reference for subsequent probe scanning and adjustment. Simultaneously, the real-time acquired current scanning path line L2 is displayed as a black dashed line to distinguish it from the target scanning path line L1. When the current scanning path matches the target scanning path, the current scanning path line L2 changes from black to green. The specific color and line style can be selected as needed.

[0056] The high efficiency, accuracy, and ease of operation demonstrated in this embodiment provide a low-cost, high-quality solution for wide-view ultrasound imaging. It should be understood that, although Figure 1-4 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1-4 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0057] Example 2 This embodiment provides an ultrasonic wide-view imaging device, such as... Figure 6 As shown, it includes: a first acquisition unit, a first determination unit, a second acquisition unit, and an image generation unit, wherein: The first acquisition unit is used to acquire the probe's attitude information.

[0058] The first determining unit determines the scanning surface and emission surface of the probe based on the probe's attitude information, determines the scanning path line and scanning normal vector based on the scanning surface and emission surface, and determines the target scanning path based on the scanning path line and normal vector.

[0059] The second acquisition unit is used to acquire ultrasound images along the target scanning path based on splicing instructions.

[0060] The image generation unit is used to stitch together ultrasound images along the target scanning path to generate a wide-view ultrasound image.

[0061] In this embodiment, the first determining unit is used to determine the target scanning path based on the scanning path line and the normal vector. Specifically, it includes using the scanning path line and scanning normal vector of the probe when receiving the stitching command as the target scanning path. Further, the line where the scanning surface intersects the transmitting surface can be determined as the scanning path line.

[0062] In this embodiment, the first acquisition unit is further configured to acquire the current posture information of the probe, and the first determination unit is further configured to determine the current scanning path based on the current posture information.

[0063] In this embodiment, the ultrasonic wide-view imaging device further includes a second determining unit. This second determining unit compares the current scanning path of the probe with the target scanning path and generates a probe adjustment prompt. Specifically, if the current scanning path matches the target scanning path, no probe adjustment prompt is given, or the current scanning path line is displayed in a first preset color; if the current scanning path does not match the target scanning path, the current scanning path line is displayed in a second preset color. The first preset color and the second preset color are different colors. The criteria for determining matching and non-matching have been described in previous embodiments and will not be repeated here.

[0064] In this embodiment, the second acquisition unit acquires ultrasound images on the target scanning path, specifically including: acquiring the scanning path corresponding to each frame of ultrasound image; if the scanning path matches the target scanning path, then the ultrasound image corresponding to that frame is the ultrasound image on the target scanning path.

[0065] In this embodiment, the image generation unit performs stitching processing on the ultrasound images along the target scanning path to generate a wide-view ultrasound image. This may include: stitching together ultrasound images from two adjacent frames along the target scanning path to generate a wide-view ultrasound image. It may also include: acquiring ultrasound images along the target scanning path in real time, stitching together adjacent frames to generate a pre-stitched image (the stitching process can be referred to in the previous embodiment); simultaneously, acquiring probe attitude information in real time, comparing the intersection distance between the current frame and the previous frame; if the intersection distance of the current frame is less than the intersection distance of the previous frame, triggering a sequential comparison of the intersection distance of the current frame with the intersection distances of previous frames until the intersection distance of the current frame is greater than the intersection distance of a certain frame, stopping the comparison; deleting the ultrasound images corresponding to frames in the pre-stitched image whose intersection distance is greater than the intersection distance of the current frame; updating the pre-stitched image, thereby obtaining the wide-view ultrasound image.

[0066] In this embodiment, the image generation unit can further optimize the ultrasonic wide-view image using the bundle adjustment method.

[0067] Specific limitations regarding the ultrasound wide-view imaging device can be found in the limitations of the ultrasound wide-view imaging method above, and will not be repeated here. Each module in the aforementioned ultrasound wide-view imaging device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0068] Example 3 A computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an ultrasonic wide-view imaging method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0069] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0070] The computer device of this embodiment includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described ultrasonic wide-view imaging method embodiment.

[0071] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps described in the above-described ultrasonic wide-view imaging method embodiment.

[0072] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0073] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0074] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An ultrasonic wide-view imaging method, characterized in that, The method includes: Acquire the probe's attitude information; The scanning surface and the emitting surface of the probe are determined based on the attitude information, and the scanning path line and the scanning normal vector are determined based on the scanning surface and the emitting surface; the scanning normal vector is a unit vector perpendicular to the emitting surface. Upon receiving the stitching instruction, the target scanning path is determined based on the scanning path line and the scanning normal vector, and multiple ultrasound images on the target scanning path are acquired. The adjacent two ultrasound images are stitched together to generate a wide-view ultrasound image. Determining the scanning path line based on the scanning surface and the emitting surface includes: The line where the scanning surface intersects the emitting surface is determined as the scanning path line; The step of accepting the stitching command and determining the target scanning path based on the scanning path and the scanning normal vector includes: using the scanning path line and the scanning normal vector determined by the scanning surface and the emitting surface when the stitching command is accepted as the target scanning path.

2. The method according to claim 1, characterized in that, After receiving the splicing instruction, the process also includes: The current scanning path of the probe is compared with the target scanning path to generate a probe adjustment prompt; wherein the current scanning path is determined by the current attitude information of the probe.

3. The method according to claim 1, characterized in that, The step of comparing the current scanning path of the probe with the target scanning path and generating a probe adjustment prompt includes: If the current scanning path matches the target scanning path, then the probe adjustment prompt will not be given, or the current scanning path line corresponding to the current scanning path will be displayed in a first preset color; If the current scanning path does not match the target scanning path, the current scanning path line is displayed in a second preset color.

4. The method according to claim 1, characterized in that, The step of acquiring multiple frames of ultrasound images along the target scanning path and stitching together adjacent frames of ultrasound images to generate a wide-view ultrasound image includes: Obtain the scanning path corresponding to each frame of the ultrasound image. If the scanning path matches the target scanning path, then the ultrasound image corresponding to that frame is the ultrasound image on the target scanning path. Stitch together the ultrasound images on the target scanning path of two adjacent frames to generate the ultrasound wide-view image.

5. The method according to claim 1, characterized in that, The step of acquiring multiple frames of ultrasound images along the target scanning path and stitching adjacent two frames of ultrasound images to generate a wide-view ultrasound image further includes: The ultrasound images along the target scanning path are acquired in real time, and adjacent frames of the ultrasound images are stitched together to generate a pre-stitched image. The attitude information of the probe is acquired in real time, and the distance between the intersection points of the current frame and the previous frame is compared. If the intersection distance of the current frame is less than the intersection distance of the previous frame, the intersection distance of the current frame is compared with the intersection distance of the previous frames in turn until the intersection distance of the current frame is greater than the intersection distance of a certain frame and the comparison stops. Delete the ultrasound images corresponding to frames whose intersection distance in the pre-stitched image is greater than the intersection distance of the current frame; The ultrasonic wide-view image is obtained based on the pre-stitched image.

6. The method according to any one of claims 1 to 5, characterized in that, The stitching process includes performing feature matching on two adjacent ultrasound images to determine matching points, determining an affine transformation matrix based on the matching points, using the affine transformation matrix to project the ultrasound images on the target scanning path onto the same coordinate system, and fusing them through a linear fusion method.

7. The method according to claim 1, characterized in that, The method also includes optimizing the ultrasonic wide-view image using a bundle adjustment method.

8. The method according to claim 1, characterized in that, Before acquiring the probe's attitude information, the process also includes: Infrared and depth images of the scanning scene are acquired, and based on the infrared and depth images, a three-dimensional image of the scanning scene and the pose information of the probe are obtained.

9. The method according to claim 8, characterized in that, After receiving the splicing instruction, the process also includes: The target scanning path line corresponding to the target scanning path and the current scanning path line corresponding to the current scanning path are displayed in different forms on the three-dimensional image, wherein the current scanning path is determined by the current attitude information of the probe.

10. The method according to claim 8, characterized in that, The probe is marked, and the acquisition of infrared and depth images of the scanning scene, and the determination of the probe's attitude information based on the infrared and depth images, includes: Infrared and depth images of the scanning scene are acquired using an active infrared structured light depth camera; the pose information of the marker is obtained based on the infrared and depth images; and the pose information of the probe is determined based on the pose information of the marker.

11. An ultrasonic wide-view imaging device, characterized in that, The device includes: The first acquisition unit is used to acquire the probe's attitude information; The first determining unit determines the scanning surface and the emitting surface of the probe based on the attitude information, and determines the scanning path line and the scanning normal vector based on the scanning surface and the emitting surface; and determines the target scanning path based on the scanning path line and the scanning normal vector; wherein the scanning normal vector is a unit vector perpendicular to the emitting surface. The second acquisition unit is used to acquire ultrasound images on the target scanning path based on the received splicing instructions; An image generation unit is used to stitch together the ultrasound images to generate a wide-view ultrasound image; The first determining unit is further configured to: The line where the scanning surface intersects the emitting surface is determined as the scanning path line; The scanning path line and the scanning normal vector determined by the scanning surface and the emitting surface when the splicing command is received are used as the target scanning path.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images