Ultrasonic wide-scene imaging method and device and storage medium

Abstract

The invention relates to an ultrasonic wide-scene imaging method and device and a storage medium. The method comprises the following steps: acquiring attitude information of a probe; determining a scanning surface and an emitting surface of the probe based on the attitude information, and determining a scanning path line and a scanning normal vector based on the scanning surface and the emitting surface; receiving a splicing instruction, determining a target scanning path based on the scanning path line and the scanning normal vector, obtaining multiple frames of ultrasonic images on the target scanning path, and splicing two adjacent frames of ultrasonic images to generate an ultrasonic wide-scene image. According to the method, only the ultrasonic images on the target scanning path are collected and spliced, the collected ultrasonic images on the non-planned path are filtered, the quality of the input image is ensured, and the quality of the ultrasonic wide-scene image is improved.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to an ultrasonic wide-view imaging method, device, and storage medium. Background Art

[0002] Ultrasound wide-view imaging, a technology that stitches multiple ultrasound images together to create a wide-field image, can provide more comprehensive lesion information and is widely used in medical diagnosis. However, the quality of ultrasound wide-view imaging is limited by the operator's difficulty maintaining the correct probe posture and position during scanning. This results in a lack of effective overlap between adjacent images, which in turn leads to poor image stitching and ultimately reduces the clinical value of wide-view imaging.

[0003] Currently, various solutions have been proposed in the prior art to improve the quality of wide-view ultrasound imaging, including integrating a scanning target positioning module, a coordinate conversion module, a posture positioning module, a posture correction module, a force feedback correction module, a robotic arm, and a matching fixture to achieve precise positioning and posture control of the ultrasound probe. Although this solution can improve the accuracy of scanning, it requires multiple sensor modules and a complex hardware system, resulting in high costs and complex operations, making it difficult to popularize in ordinary medical institutions. There is also an ultrasound wide-view imaging method based on magnetic navigation in the prior art, that is, binding the ultrasound probe with a magnetic sensor to monitor the spatial position and posture of the probe in real time. Although this method can improve the accuracy of image stitching to a certain extent, it requires additional magnetic navigation equipment, which increases the complexity and cost of the system, and may be affected by magnetic field interference, resulting in measurement errors.

[0004] Therefore, there is an urgent need for an efficient, low-cost and accurate wide-field ultrasound imaging method that can simplify the operating process in actual clinical applications and meet the needs of medical diagnosis. Summary of the Invention

[0005] Based on this, it is necessary to provide an ultrasonic wide-view imaging method, device and storage medium to address the above technical problems.

[0006] An ultrasonic wide-view imaging method, the method comprising:

[0007] Get the probe's posture information;

[0008] Determine a scanning surface and an emitting surface of the probe based on the posture information, and determine a scanning path line and a scanning normal vector based on the scanning surface and the emitting surface;

[0009] The system receives a stitching instruction, determines a target scanning path based on the scanning path line and the scanning normal vector, obtains multiple frames of ultrasound images on the target scanning path, and stitches two adjacent frames of the ultrasound images to generate an ultrasound wide-view image.

[0010] An ultrasonic wide-view imaging device, comprising:

[0011] A first acquiring unit, configured to acquire the posture information of the probe;

[0012] a first determining unit, configured to determine a scanning surface and an emitting surface of the probe based on the posture information, determine a scanning path line and a scanning normal vector based on the scanning surface and the emitting surface; and determine a target scanning path based on the scanning path line and the scanning normal vector;

[0013] a second acquiring unit, configured to acquire an ultrasonic image on the target scanning path based on the received stitching instruction;

[0014] The image generating unit is configured to perform splicing processing on the ultrasound images to generate a wide-view ultrasound image.

[0015] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned ultrasonic wide-view imaging method when executed by a processor.

[0016] The present invention obtains the probe's posture information, determines the target scanning path based on the probe's posture information, and only collects ultrasound images on the target scanning path for stitching, filtering out ultrasound images collected on unplanned paths, thereby ensuring the input image quality and improving the quality of ultrasound wide-view images. At the same time, the target scanning path line provides users with clear scanning guidance, making the scanning process more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 1 is a flow chart of an ultrasonic wide-view imaging method according to an embodiment;

[0018] Figure 2 Schematic diagram of the process of step S106 in one embodiment;

[0019] Figure 3 This is another schematic flow chart of step S106 in one embodiment;

[0020] Figure 4 is another schematic flow chart of an ultrasonic wide-view imaging method according to an embodiment;

[0021] Figure 5 is a schematic diagram of a three-dimensional image of a scanned scene in one embodiment;

[0022] Figure 6 1 is a schematic structural diagram of an ultrasonic wide-view imaging device in one embodiment;

[0023] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0025] Example 1

[0026] This embodiment provides an ultrasonic wide-view imaging method, such as Figure 1 As shown, the method includes the following steps:

[0027] S102 obtains the posture information of the probe.

[0028] In this embodiment, the posture information of the probe includes the position and orientation information of the probe in three-dimensional space. Specifically, the posture information of the probe can be obtained by devices such as depth cameras, sensors, gyroscopes, and accelerometers, which can monitor the position coordinates and angle information of the probe in real time. The posture information of the probe can be represented by a six-dimensional vector, including three position coordinates (x, y, z) and three rotation angles (α, β, γ), which respectively represent the position of the probe in three-dimensional space and the rotation angles around the three coordinate axes.

[0029] S104 determines the scanning surface and the emission surface of the probe based on the posture information, and determines the scanning path line and the scanning normal vector based on the scanning surface and the emission surface.

[0030] In this embodiment, according to the scanning direction of the probe (usually determined by the direction of the probe axis), the scanning surface is defined as the plane pointed to by the probe axis direction, which can be calculated by the position coordinates and rotation angles in the probe posture information. Specifically, the scanning surface can be expressed as a plane equation: ax+by+cz+d=0, where (x, y, z) represents the coordinates of any point on the scanning surface in three-dimensional space, a, b, c are the normal vectors of the scanning surface, and d is the constant term of the plane equation. The rotation angle (α0, β0, γ0) is used to calculate the probe axis direction vector n. Specifically, the rotation angle (α0, β0, γ0) is converted into a direction vector n=(a, b, c) through a rotation matrix or Euler angle formula.

[0031] In this embodiment, the transmitting surface of the probe is the plane on the probe that contacts the area to be scanned and is used to transmit ultrasonic waves (usually perpendicular to the probe axis). The direction of the probe axis is vector n, and the transmitting surface can be expressed as a plane passing through a point P on the transmitting surface of the probe and with a normal vector m, where m is perpendicular to n; the transmitting surface can be expressed as another plane equation: ex+fy+gz+h=0, where (x, y, z) represents the coordinates of any point on the transmitting surface in three-dimensional space, e, f, g are the normal vectors of the transmitting surface, and h is the constant term of the plane equation.

[0032] In this embodiment, the specific method for determining the scanning path line based on the scanning surface and the transmitting surface includes: determining the line where the scanning surface intersects the transmitting surface as the scanning path line. The scanning surface and the transmitting surface are both planes in three-dimensional space, and their intersection can be a straight line, which is the scanning path line. The straight line where the scanning surface intersects the transmitting surface is determined as the scanning path line. The straight line can be expressed as a set of parametric equations: x = x0 + at, y = y0 + bt, z = z0 + ct, where (x0, y0, z0) is the intersection point of the scanning surface and the transmitting surface, that is, the initial position; (a, b, c) is the cross product of the scanning surface normal vector n and the transmitting surface normal vector m, that is, n × m, and t is a parameter.

[0033] The scan normal vector is a unit vector perpendicular to the emission surface and can be calculated using the plane equation of the emission surface. Specifically, if the plane equation of the emission surface is ex+fy+gz+h=0, then the scan normal vector is

[0034] The scanning path can be represented as a series of posture information {(x0,y0,z0,a,b,c,α i ,β i ,γ i )}, where (x0, y0, z0 a, b, c) are the parameters of the straight line equation corresponding to the scanning path line, (α i ,β i ,γ i ) is the scanning normal vector, that is, the scanning path includes the scanning path line and the scanning normal vector.

[0035] S106 receives the stitching instruction, determines the target scanning path based on the scanning path line and the scanning normal vector, obtains multiple frames of ultrasound images on the target scanning path, stitches two adjacent frames of ultrasound images, and generates an ultrasound wide-view image.

[0036] In this embodiment, the target scanning path refers to the path that the probe should follow during the scanning process. It includes not only the position trajectory of the probe (i.e., the scanning path line), but also the direction of the probe at each position (i.e., the scanning normal vector). In actual applications, before receiving the splicing instruction, the posture information of the probe will be acquired in real time to generate the corresponding scanning path; at the same time, the probe will emit and receive ultrasonic waves to obtain the current scanning image. In order to obtain a high-quality scanning image, the user needs to continuously move and rotate the probe. During this process, the posture information of the probe will also change, and the corresponding scanning path will also change continuously. When the user determines that the ultrasonic image obtained at a certain time meets the requirements, the user will issue a splicing instruction, and the scanning path at this time is the target scanning path. Among them, the scanning path line at this time is the target scanning path line, and the scanning normal vector at this time is the target normal vector. That is, the scanning path line and scanning normal vector determined by the scanning surface and the transmitting surface when the splicing instruction is received are used as the target scanning path.

[0037] In this embodiment, after receiving a stitching instruction from the user, the ultrasound imaging device begins acquiring ultrasound images along the target scanning path. Specifically, the probe moves along the target scanning path while continuously transmitting and receiving ultrasound waves, thereby generating multiple frames of ultrasound images. These ultrasound images are then stitched together to generate a complete wide-view ultrasound image. The specific stitching method will be described in detail below.

[0038] In this embodiment, if Figure 2 As shown, step S106 may include:

[0039] After receiving the stitching instruction, S11 obtains the scanning path corresponding to each frame of the ultrasound image.

[0040] S12: If the scanning path corresponding to the acquired ultrasound image matches the target scanning path, the ultrasound image corresponding to the frame is the ultrasound image on the target scanning path.

[0041] S13 stitches two adjacent frames of ultrasound images on the target scanning path to generate an ultrasound wide-view image.

[0042] Specifically, the ultrasound imaging device checks whether each ultrasound image frame was acquired along the target scanning path. Only those ultrasound images acquired along the target scanning path are used for stitching. This ensures the quality and accuracy of the generated wide-view ultrasound image. The stitching command can be issued by the user.

[0043] During the actual scanning process, the current scanned image may not meet the requirements. In this case, the user may return and rescan. This method can handle the situation where the probe partially returns and rescans during the scanning process. When the probe moves along the target scanning path line in the direction that has been scanned, multiple frames of ultrasound images may be obtained at the same position. In order to avoid repeated stitching, the images obtained by rescanning are used for stitching, and the previous repeated images need to be deleted.

[0044] In order to solve the above problem, in this embodiment, Figure 3 As shown, step S106 also includes:

[0045] S21 acquires ultrasound images on the target scanning path in real time, stitches two adjacent frames of ultrasound images, and generates a pre-stitched image.

[0046] Specifically, ultrasound images along the target scanning path are collected in real time, and feature points (such as edges and key points) are extracted from two adjacent frames of images for matching. Through feature point matching, the transformation matrix (such as translation, rotation, and scaling) between the two frames of images is calculated. The two adjacent frames of images are aligned according to the transformation matrix, fused, and spliced in real time to generate a pre-stitched image.

[0047] S22 obtains the probe's posture information in real time and compares the intersection distance between the current frame and the previous frame.

[0048] Specifically, while stitching, the probe's posture information, including position (x, y, z) and direction (rotation angle or direction vector), is acquired in real time. Based on the posture information, the spatial equation of the probe centerline (i.e., the probe's central axis) is calculated. Based on the probe centerline corresponding to the target scan path line, the intersection of the target scan path line and the probe centerline is determined and used as the reference point, i.e., the initial intersection point. Based on the probe's current posture, the intersection of the probe centerline corresponding to the current frame and the target scan path line is calculated.

[0049] Calculate the distance between the current frame intersection and the initial intersection, recorded as d N ; Calculate the distance between the previous frame intersection and the initial intersection, recorded as d N-1 Compare d N and d N-1 :If d N >d N-1 , indicating that the probe is moving away from the initial point; if d N <d N-1 , indicating that the probe is moving towards the initial point.

[0050] If the intersection distance of the current frame is smaller than the intersection distance of the previous frame, S23 triggers the comparison of the intersection distance of the current frame with the intersection distance of the previous frames in sequence until the intersection distance of the current frame is larger than the intersection distance of a certain frame.

[0051] When the ultrasonic imaging device compares the intersection distance of the current frame with the intersection distance of the previous frame, if it detects that the intersection distance d of the current frame is N Greater than the intersection distance d of the previous frame N-1 , indicating that the probe is moving away from the initial point; if the intersection distance d of the current frame is detected N Less than the intersection distance d of the previous frame N-1 , indicating that the probe is moving towards the initial point and may need to be re-scanned. The user may be dissatisfied with the current scanned image and re-scan it. At this time, the intersection distances of the previous frames (such as d N-2 d N-3 d N-1 …d1), and the intersection distance with the current frame is d N Compare. The intersection distance d of the current frame N Greater than the intersection distance d of the previous frame N-M , then stop the comparison and record the position of the frame. If the intersection distance of the current frame is d N Less than the intersection distance d of the previous frame N-M , then continue to compare the intersection distance d of the next frame N-M-1 , until a frame that meets the conditions is found or all previous frames are traversed, where N and M are both integers and N>M. Through this process, the position where the probe rescans on the target scanning path can be determined, thus avoiding repeated stitching of multiple frames of images at the same location.

[0052] S24 deletes the ultrasound images corresponding to the frames whose intersection distances on the pre-stitched image are greater than the intersection distance of the current frame.

[0053] By comparing the intersection distance of the current frame with the intersection distances of each frame in the pre-stitched image, frames with larger intersection distances than the current frame are identified and their corresponding ultrasound images are deleted from the pre-stitched image to avoid duplicate stitching or unsatisfactory images in the pre-stitched image. The stitching process of adjacent ultrasound frames continues to update the pre-stitched image.

[0054] S25 obtains an ultrasonic wide-view image based on the pre-stitched images.

[0055] Continue the above process and update the pre-stitched images until the stitching is complete to obtain the ultrasound wide-view image.

[0056] In this embodiment, the stitching process in step S106 includes performing feature matching on two adjacent frames of ultrasound images to determine matching points, determining an affine transformation matrix based on the matching points, using the affine transformation matrix to project the ultrasound images on the target scanning path into the same coordinate system, and fusing them through linear fusion.

[0057] Among them, feature matching refers to finding corresponding feature points in two adjacent frames of ultrasound images. These feature points are usually significant features in the image, such as edges, corners, etc. Specifically, you can use algorithms such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) to extract image feature points, and then use the RANSAC (Random Sample Consensus) algorithm for feature matching to find matching points in the two frames of images. Determining the affine transformation matrix based on matching points means calculating a transformation matrix based on the matching points in the two frames of images. This matrix can transform one frame of image into the coordinate system of another frame of image. The affine transformation matrix can be a 3×3 matrix, which can represent transformations such as translation, rotation, and scaling. Specifically, assuming that n pairs of matching points {(x i ,y i ),(x i ',y i ')}, i = 1, 2, ..., n, n is an integer, then the affine transformation matrix A can be obtained by solving the following system of equations:

[0058] [x i 'y i '1] T =A*[x i y i 1] T , i=1,2,...,n

[0059] The ultrasound images along the target scanning path are projected into the same coordinate system using the affine transformation matrix. This means that each ultrasound image frame is transformed into the same coordinate system (i.e., the wide-view image coordinate system) using the calculated affine transformation matrix. This allows all ultrasound images to be stitched together in the same coordinate system.

[0060] The fusion is performed by linear fusion. Specifically, in the stitching area, the pixel values of two adjacent frames are weighted averaged according to a certain weight to achieve a smooth transition. Specifically, the pixel value of the stitching area can be calculated using the following formula:

[0061] I(x,y)=w1*I1(x,y)+w2*I2(x,y)

[0062] Where I(x,y) is the pixel value at point (x,y) in the stitched image, I1(x,y) and I2(x,y) are the pixel values at point (x,y) in the two adjacent original frames, respectively. w1 and w2 are weights, satisfying w1 + w2 = 1. The weights are determined based on the position of the point (x,y) in the stitched area, so that at the edge of the stitched area, the weight of one image frame is 1 and the weight of the other image frame is 0; in the center of the stitched area, the weights of both images are 0.5.

[0063] After stitching to generate the wide-view ultrasound image, Figure 4 As shown, this embodiment further includes step S108, optimizing the ultrasonic wide-view image using the bundle adjustment method.

[0064] Bundle adjustment is a method used to optimize three-dimensional reconstruction results. It can optimize the position and orientation of each frame of ultrasound image, making the stitched wide-view image more accurate.

[0065] Specifically, the objective function of the bundle adjustment method is to minimize the sum of squares of the reprojection errors:

[0066] minΣ||X ij -P(X i ,C j )||2

[0067] Among them, X ij is the observation position of the i-th feature point in the j-th frame image, X i is the three-dimensional coordinate of the i-th feature point in the wide-view image, C j is the probe posture information (including position and direction) of the j-th frame image, and P is the projection function, which projects the three-dimensional point onto the image plane.

[0068] After receiving the splicing instruction, such as Figure 4 As shown, this embodiment also includes the steps of:

[0069] S107 compares the current scanning path of the probe with the target scanning path and generates a probe adjustment prompt; wherein the current scanning path of the probe is determined by the current posture information of the probe.

[0070] In this embodiment, the probe's current scanning path is determined by the probe's current posture information. Specifically, the current scanning path is calculated based on the probe's current posture information acquired in real time and compared with the previously determined target scanning path. If there is a discrepancy between the two, the system generates a probe adjustment prompt, guiding the user to adjust the probe's position and orientation to align with the target scanning path.

[0071] Step S107 specifically includes the following contents:

[0072] If the current scanning path matches the target scanning path, no probe adjustment prompt is performed or the current scanning path line is displayed in the first preset color.

[0073] If the current scan path does not match the target scan path, the current scan path line is displayed in a second preset color.

[0074] Specifically, when determining the current scanning path line and the current scanning normal vector of the probe based on the current posture information of the probe, the determination method is the same as the method for determining the scanning path line and the scanning normal vector, and will not be repeated here. If the current scanning path line coincides with the target scanning path line, and the current scanning normal vector is parallel to or coincides with the target scanning normal vector, it indicates that the current scanning path matches the target scanning path, indicating that the scanning direction and posture are correct at this time, and the probe adjustment prompt may not be performed or the current scanning path line may be displayed in the first preset color; for example, in this embodiment, the first preset color is green to indicate that the current scanning path matches the target scanning path, indicating that the current scan is correct. If the current scanning path line does not overlap with the target scanning path line, or the current scanning normal vector is not parallel to the target scanning normal vector, it means that the current scanning path does not match the target scanning path, indicating that the scanning direction or posture needs to be adjusted at this time, and the current scanning path line is displayed in a second preset color; wherein the first preset color and the second preset color are different colors; for example, in this embodiment, the second preset color is red to indicate that the current scanning path does not match the target scanning path, to prompt that the current scan needs to be adjusted.

[0075] Furthermore, the target scanning path line and the current scanning path line are displayed on the interface. Specifically, before accepting the splicing instruction, the target scanning path line is displayed in the first preset form, such as: displayed as a white dotted line; after accepting the splicing instruction, the target scanning path line is displayed in the second preset form, such as: displayed as a blue solid line, and the current scanning path line is displayed in the third preset form or the fourth preset form. The third preset form can be a dotted line of the first preset color, and the fourth preset form can be a dotted line of the second preset color; it is convenient for users to distinguish between the target scanning path line and the current scanning path line, and quickly understand whether the current scanning status is correct.

[0076] In this embodiment, if Figure 4 As shown, before obtaining the probe's posture information, the following steps are also included:

[0077] Step S101 acquires an infrared image and a depth image of the scanned scene, and determines a three-dimensional image of the scanned scene and posture information of the probe based on the infrared image and the depth image.

[0078] Specifically, using an infrared camera to capture an infrared image of the scanned scene can clearly show the outline of the object under low light conditions, at least including the outline of the probe and the area to be inspected; at the same time, using an infrared camera to obtain a depth image of the scanned scene, recording the distance from each point in the scene to the camera, can provide three-dimensional information of the scene. Based on the infrared image and depth image, a three-dimensional image of the scanned scene can be obtained through three-dimensional reconstruction technology, such as Figure 5 As shown, the three-dimensional image can display the posture information of the probe and the area to be scanned. Through image processing and pattern recognition technology, the probe is located in the three-dimensional image and its position and orientation are calculated to obtain the probe posture information.

[0079] Preferably, a marker is provided on the probe, and an active infrared structured light depth camera is used to obtain infrared images and depth images of the scanned scene. Based on the infrared and depth images, the marker's posture information is obtained; and the probe's posture information is determined based on the marker's posture information. The active infrared structured light depth camera includes an infrared floodlight, an infrared speckle projector, and an infrared camera for receiving light information. The infrared floodlight and infrared camera can obtain an infrared image of the scanned scene, and the infrared speckle projector and infrared camera can obtain a depth image of the scanned scene. The marker can be a special pattern affixed to the probe or a specially shaped object attached to the probe, which is easily identified and tracked in the infrared and depth images. By identifying and tracking the marker, the marker's three-dimensional coordinates can be determined based on its position in the infrared and depth images, combined with the depth camera's internal and external parameters. Then, based on the known relationship between the marker and the probe, the probe's posture information can be calculated.

[0080] In practical applications, a marker can be a planar pattern containing multiple feature points, such as a black square, black dot, QR code, ArUco marker, or AprilTag marker. These markers can be accurately identified and located in infrared and depth images. By analyzing the marker's position and shape in the image, the marker's 3D position and orientation can be calculated. The PnP (Perspective-n-Point) algorithm is then used to determine the probe's posture information (including position and rotation angle).

[0081] For example, suppose the marker is a black square. Based on its position in the infrared and depth images, combined with the intrinsic and extrinsic parameters of the depth camera, the marker's three-dimensional coordinates are calculated. The coordinates of its four corner points in three-dimensional space are {(X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), (X4, Y4, Z4)}, relative to the marker's own coordinate system. In the infrared and depth images, the positions of these four corner points {(u1, v1), (u2, v2), (u3, v3), (u4, v4)} and their depth values {d1, d2, d3, d4} can be found. By solving the PnP (Perspective-n-Point) problem, the position and orientation of the marker relative to the camera can be calculated. Then, based on the known relationship between the marker and each point on the probe, the position and orientation of the probe relative to the camera, i.e., the probe's pose information, can be calculated. Assume that there are three black circles marked on the probe handle and three black circles marked on the probe shell. The position and outline of the black circles can be detected in the infrared image through circle detection, template matching, or feature detection algorithm (such as SIFT or ORB). Then, based on the position of the black circles in the infrared image and depth image, combined with the intrinsic and extrinsic parameters of the depth camera, the three-dimensional coordinates of the black circles can be calculated.

[0082] In this embodiment, after receiving the splicing instruction, the following steps are also included:

[0083] Step S109 displays the target scanning path line corresponding to the target scanning path and the current scanning path line corresponding to the current scanning path in different forms on the three-dimensional image, wherein the current scanning path is determined by the current posture information of the probe.

[0084] Specifically, if Figure 5 As shown, the 3D image can intuitively display the three-dimensional structure of the scanning scene, including the scanning surface A and the emission surface B. The target scanning path line L1 and the current scanning path line L2 are also displayed in different forms on the 3D image, helping users better understand and adjust the position and direction of the probe. For example, after receiving the stitching command, the target scanning path line L1 on the 3D image can change from the original white dashed line to a blue solid line, making it easier for users to identify the target scanning path line L1. The position of this line does not change. It can be understood that when the stitching command is received, it indicates that the scanning path line at this time is the target scanning path line L1 that the user wants, which will be used as a reference for subsequent probe scanning and adjustment. At the same time, the current scanning path line L2, which is acquired in real time, is displayed as a black dashed line to distinguish it from the target scanning path line L1. When the current scanning path matches the target scanning path, the current scanning path line L2 changes from black to green. The specific color and line style can be selected as needed.

[0085] The high efficiency, accuracy and ease of operation of this embodiment in practical applications provide a low-cost, high-quality solution for wide-view ultrasound imaging.

[0086] It should be understood that although Figure 1-4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1-4 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0087] Example 2

[0088] This embodiment provides an ultrasonic wide-view imaging device, such as Figure 6 As shown, it includes: a first acquisition unit, a first determination unit, a second acquisition unit, and an image generation unit, wherein:

[0089] The first acquiring unit is used to acquire the posture information of the probe.

[0090] The first determination unit is used to determine the scanning surface and the emission surface of the probe based on the posture information of the probe, determine the scanning path line and the scanning normal vector based on the scanning surface and the emission surface; and determine the target scanning path based on the scanning path line and the normal vector.

[0091] The second acquisition unit is used to acquire an ultrasonic image on the target scanning path based on the stitching instruction.

[0092] The image generation unit is used to stitch the ultrasound images on the target scanning path to generate a wide-view ultrasound image.

[0093] In this embodiment, the first determination unit is configured to determine a target scanning path based on the scanning path line and the normal vector. Specifically, the first determination unit may use the scanning path line and the scanning normal vector of the probe when receiving the stitching instruction as the target scanning path. Furthermore, the scanning path line may be determined as the line where the scanning plane intersects the transmitting plane.

[0094] In this embodiment, the first acquiring unit is further configured to acquire current posture information of the probe, and the first determining unit is further configured to determine a current scanning path based on the current posture information.

[0095] In this embodiment, the ultrasonic wide-view imaging device further includes a second determination unit configured to compare the probe's current scanning path with the target scanning path and generate a probe adjustment prompt. Specifically, if the current scanning path matches the target scanning path, no probe adjustment prompt is provided or the current scanning path line is displayed in a first preset color. If the current scanning path does not match the target scanning path, the current scanning path line is displayed in a second preset color. The first preset color and the second preset color are different colors. The criteria for determining whether a scan is matched or not have been described in the previous embodiment and will not be repeated here.

[0096] In this embodiment, the second acquisition unit acquires the ultrasound image on the target scanning path, specifically including: acquiring the scanning path corresponding to each frame of the ultrasound image; if the scanning path matches the target scanning path, the ultrasound image corresponding to the frame is the ultrasound image on the target scanning path.

[0097] In this embodiment, the image generation unit stitches the ultrasound images along the target scanning path to generate a wide-view ultrasound image. This may include stitching two adjacent frames of ultrasound images along the target scanning path to generate the wide-view ultrasound image. The process may also include acquiring ultrasound images along the target scanning path in real time, stitching the two adjacent frames to generate a pre-stitched image. The stitching process may refer to the previous embodiment. Simultaneously, probe posture information is acquired in real time, and the intersection distance between the current frame and the previous frame is compared. If the intersection distance of the current frame is less than the intersection distance of the previous frame, the intersection distance of the current frame is sequentially compared with the intersection distance of the previous frames, and the comparison is stopped until the intersection distance of the current frame is greater than the intersection distance of a certain frame. The ultrasound images corresponding to the frames in the pre-stitched image whose intersection distance is greater than the intersection distance of the current frame are deleted. The pre-stitched image is updated to obtain the wide-view ultrasound image.

[0098] In this embodiment, the image generation unit may further utilize the bundle adjustment method to optimize the ultrasonic wide-view image.

[0099] The specific limitations of the ultrasonic wide-view imaging device can be found in the limitations of the ultrasonic wide-view imaging method described above and will not be further elaborated here. Each module in the aforementioned ultrasonic wide-view imaging device may be implemented in whole or in part through software, hardware, or a combination thereof. Each of the aforementioned modules may be embedded in or independent of a processor in a computer device in hardware form, or may be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each of the aforementioned modules.

[0100] Example 3

[0101] A computer device is provided, which may be a terminal, and its internal structure diagram may be as follows Figure 7As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an ultrasonic wide-view imaging method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0102] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0103] The computer device of this embodiment includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above-mentioned ultrasonic wide-view imaging method embodiment are implemented.

[0104] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned ultrasound wide-view imaging method embodiment are implemented.

[0105] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0106] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. An ultrasonic wide-view imaging method, characterized in that: The method comprises: Get the probe's posture information; Determine a scanning surface and an emitting surface of the probe based on the posture information, and determine a scanning path line and a scanning normal vector based on the scanning surface and the emitting surface; The system receives a stitching instruction, determines a target scanning path based on the scanning path line and the scanning normal vector, obtains multiple frames of ultrasound images on the target scanning path, and stitches two adjacent frames of the ultrasound images to generate an ultrasound wide-view image.

2. The method according to claim 1, characterized in that The determining of a scanning path line based on the scanning surface and the transmitting surface comprises: The line where the scanning surface intersects the emitting surface is determined as the scanning path line.

3. The method according to claim 1, characterized in that The receiving stitching instruction and determining the target scanning path based on the scanning path and the scanning normal vector include: using the scanning path line and the scanning normal vector determined by the scanning surface and the transmitting surface when the stitching instruction is received as the target scanning path.

4. The method according to claim 1, wherein After receiving the splicing instruction, the method further includes: Compare the current scanning path of the probe with the target scanning path to generate a probe adjustment prompt; wherein the current scanning path is determined by the current posture information of the probe.

5. The method according to claim 4, characterized in that The comparing the current scanning path of the probe with the target scanning path and generating a probe adjustment prompt includes: If the current scanning path matches the target scanning path, no probe adjustment prompt is performed or the current scanning path line is displayed in a first preset color; If the current scanning path does not match the target scanning path, the current scanning path line is displayed in a second preset color.

6. The method according to claim 1, characterized in that The step of acquiring multiple frames of ultrasound images on the target scanning path and splicing two adjacent frames of the ultrasound images to generate a wide-view ultrasound image includes: A scanning path corresponding to each frame of the ultrasound image is obtained. If the scanning path matches the target scanning path, the ultrasound image corresponding to the frame is the ultrasound image on the target scanning path. The ultrasound images on the target scanning path of two adjacent frames are spliced to generate the ultrasound wide-view image.

7. The method according to claim 1, characterized in that The step of acquiring multiple frames of ultrasound images on the target scanning path and splicing two adjacent frames of the ultrasound images to generate a wide-view ultrasound image further includes: Acquire an ultrasonic image on the target scanning path in real time, and perform stitching processing on two adjacent frames of the ultrasonic image to generate a pre-stitched image; Acquire the posture information of the probe in real time and compare the intersection distance between the current frame and the previous frame; If the intersection distance of the current frame is less than the intersection distance of the previous frame, the intersection distance of the current frame is triggered to be compared with the intersection distances of the previous frames in sequence, and the comparison is stopped when the intersection distance of the current frame is greater than the intersection distance of a certain frame; Deleting the ultrasound image corresponding to the frame whose intersection distance on the pre-stitched image is greater than the intersection distance of the current frame; The ultrasonic wide-view image is obtained based on the pre-stitched image.

8. The method according to any one of claims 1 to 7, characterized in that The stitching process includes performing feature matching on two adjacent frames of ultrasound images to determine matching points, determining an affine transformation matrix based on the matching points, using the affine transformation matrix to project the ultrasound images on the target scanning path into the same coordinate system, and fusing them through linear fusion.

9. The method according to claim 1, characterized in that The method further includes optimizing the ultrasonic wide-view image using a bundle adjustment method.

10. The method according to claim 1, characterized in that Before acquiring the posture information of the probe, the method further includes: An infrared image and a depth image of the scanned scene are acquired, and based on the infrared image and the depth image, a three-dimensional image of the scanned scene and posture information of the probe are obtained.

11. The method according to claim 10, characterized in that After receiving the splicing instruction, the method further includes: The target scanning path line corresponding to the target scanning path and the current scanning path line corresponding to the current scanning path are displayed in different forms on the three-dimensional image, wherein the current scanning path is determined by the current posture information of the probe.

12. The method according to claim 10, characterized in that The probe is provided with a mark, and the infrared image and the depth image of the scanning scene are obtained, and the posture information of the probe is determined based on the infrared image and the depth image, including: An infrared image and a depth image of the scanned scene are acquired based on an active infrared structured light depth camera; posture information of the marker is acquired based on the infrared image and the depth image, and posture information of the probe is determined based on the posture information of the marker.

13. An ultrasonic wide-view imaging device, characterized in that: The device comprises: A first acquiring unit, configured to acquire the posture information of the probe; a first determining unit, configured to determine a scanning surface and an emitting surface of the probe based on the posture information, determine a scanning path line and a scanning normal vector based on the scanning surface and the emitting surface; and determine a target scanning path based on the scanning path line and the scanning normal vector; a second acquiring unit, configured to acquire an ultrasonic image on the target scanning path based on the received stitching instruction; The image generating unit is configured to perform splicing processing on the ultrasound images to generate a wide-view ultrasound image.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images