Ultrasonic probe positioning method and device, electronic equipment and computer readable medium

Abstract

The embodiment of the invention discloses an ultrasonic probe positioning method and device, electronic equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: acquiring a current electromagnetic signal, a skin marker image set and an axial displacement value; determining initial position information; performing region-of-interest image extraction on the skin marker image set to obtain a skin marker region-of-interest image set, and performing skin marker recognition processing to obtain a skin marker feature point set; determining a feature point three-dimensional coordinate set; determining a coordinate system transformation matrix; determining expected three-dimensional position information; probe position adjustment is conducted on the expected three-dimensional position information of the ultrasonic probe, and adjusted position information is obtained; and controlling the ultrasonic probe to generate a skin ultrasonic image. According to the embodiment, through the visual ultrasonic probe based on the skin marker image and the mechanical ultrasonic probe positioning of the axial displacement value, the ultrasonic probe positioning accuracy can be improved, and then the quality of the generated ultrasonic probe is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to an ultrasound probe positioning method, device, electronic device, and computer-readable medium. Background Art

[0002] At present, with the rapid development of science and technology, medical equipment for detecting the user's body is becoming more and more common and mature. How to improve the accuracy of medical equipment for detecting the user's body has become an issue that people are paying more and more attention to. For the positioning of the ultrasound probe, the method usually adopted is: extracting feature points from the ultrasound image collected by the ultrasound probe to obtain an ultrasound image feature point set. Then, calibration and positioning are performed using the ultrasound image feature point set and the three-dimensional space where the ultrasound probe is located.

[0003] However, it is found in practice that when the above method is used to position the ultrasonic probe, the following technical problems often occur: since the ultrasonic probe is calibrated and positioned only through the collected ultrasonic images, the positioning factors considered are too single, and there are a large number of coordinate system conversion operations in the probe positioning through ultrasonic images. The positioning process is relatively cumbersome, resulting in low positioning accuracy of the ultrasonic probe, which in turn reduces the quality of the ultrasonic images collected by the ultrasonic probe, and there is a large amount of useless information in the ultrasonic images.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure provide an ultrasound probe positioning method, an apparatus, an electronic device, and a computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide an ultrasound probe positioning method, including: acquiring a current electromagnetic signal of a sensor excitation coil in a medical imaging robotic arm, a skin marker image set, and an axial displacement value of an ultrasound probe; determining initial position information of the ultrasound probe in the medical imaging robotic arm based on the current electromagnetic signal; performing region of interest image extraction on the skin marker image set to obtain a skin marker region of interest image set; performing skin marker recognition processing on the skin marker region of interest image set to obtain a skin marker feature point set; determining a feature point in a probe coordinate system where the ultrasound probe is located that corresponds to each skin marker feature point in the skin marker feature point set three-dimensional coordinates, and obtain a three-dimensional coordinate set of feature points; determine the coordinate system conversion matrix of the probe coordinate system and the image coordinate system corresponding to the skin marker image according to the two-dimensional image pixel coordinate set corresponding to the above skin marker feature point set and the above three-dimensional coordinate set of feature points; determine the expected three-dimensional position information of the ultrasound probe with respect to the above initial position information according to the above skin marker feature point set and the above coordinate system conversion matrix; adjust the probe position of the expected three-dimensional position information of the ultrasound probe according to the above axial displacement value and the preset probe pressure information to obtain the adjusted position information; and control the ultrasound probe to generate a skin ultrasound image at the above adjusted position information according to the above adjusted position information.

[0008] In a second aspect, some embodiments of the present disclosure provide an ultrasound probe positioning device, including: an acquisition unit, configured to acquire a current electromagnetic signal of a sensor excitation coil in a medical imaging robot arm, a skin marker image set, and an axial displacement value of the ultrasound probe; a first determination unit, configured to determine the initial position information of the ultrasound probe in the medical imaging robot arm according to the current electromagnetic signal; a region of interest image extraction unit, configured to perform region of interest image extraction on the skin marker image set to obtain a skin marker region of interest image set; a skin marker recognition unit, configured to perform skin marker recognition processing on the skin marker region of interest image set to obtain a skin marker feature point set; and a second determination unit, configured to determine the location of each skin marker feature point in the skin marker feature point set corresponding to the location of the ultrasound probe. The three-dimensional coordinates of the feature points in the probe coordinate system are obtained to obtain the three-dimensional coordinate set of the feature points; the third determining unit is configured to determine the coordinate system conversion matrix of the probe coordinate system and the image coordinate system corresponding to the skin marker image according to the two-dimensional image pixel coordinate set corresponding to the skin marker feature point set and the three-dimensional coordinate set of the feature points; the fourth determining unit is configured to determine the expected three-dimensional position information of the ultrasound probe for the initial position information according to the skin marker feature point set and the coordinate system conversion matrix; the probe position adjustment unit is configured to adjust the probe position of the expected three-dimensional position information of the ultrasound probe according to the axial displacement value and the preset probe pressure information to obtain the adjusted position information; the control unit is configured to control the ultrasound probe to generate the skin ultrasound image at the adjusted position information according to the adjusted position information.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0011] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the ultrasonic probe positioning method of some embodiments of the present disclosure can improve the accuracy of ultrasonic probe positioning through the mechanical ultrasonic probe positioning based on the visual ultrasonic probe of the skin marker image and the axial displacement value, thereby improving the quality of the generated ultrasonic probe. Specifically, the reason why the positioning accuracy of the relevant ultrasonic probe is low, thereby reducing the quality of the ultrasonic image collected by the ultrasonic probe, and there is a lot of useless information in the ultrasonic image is that: since the calibration and positioning of the ultrasonic probe is performed only through the collected ultrasonic image, the positioning factors considered are too single, and there are a large number of coordinate system conversion operations in the probe positioning through the ultrasonic image, the positioning process is relatively cumbersome, resulting in a low positioning accuracy of the ultrasonic probe, thereby reducing the quality of the ultrasonic image collected by the ultrasonic probe, and there is a lot of useless information in the ultrasonic image. Based on this, the ultrasonic probe positioning method of some embodiments of the present disclosure can first obtain the current electromagnetic signal of the sensor excitation coil in the medical imaging manipulator, the skin marker image set and the axial displacement value of the ultrasonic probe. Here, the current electromagnetic signal facilitates the subsequent determination of the initial position information of the ultrasonic probe, and the skin marker image set and the axial displacement value facilitate the subsequent positioning of the ultrasonic probe based on vision and mechanics. Secondly, according to the current electromagnetic signal, the initial position information of the ultrasonic probe in the medical imaging manipulator is determined. Here, the initial position information of the ultrasonic probe is determined, which can shorten the positioning range of the ultrasonic probe. Thirdly, the image of the region of interest is extracted from the skin marker image set to obtain the skin marker region of interest image set. Here, the recognition range of the skin marker can be further narrowed, which is convenient for improving the recognition efficiency in the future. Next, the skin marker region of interest image set is processed for skin marker recognition to obtain a skin marker feature point set. Here, the recognition efficiency and accuracy of the skin marker can be improved. Subsequently, the three-dimensional coordinates of the feature points corresponding to each skin marker feature point in the skin marker feature point set in the probe coordinate system where the ultrasonic probe is located are determined to obtain a three-dimensional coordinate set of feature points. Here, the three-dimensional coordinate set of feature points facilitates the subsequent determination of the conversion matrix of the coordinate system of the ultrasonic image and the skin marker image. Afterwards, according to the two-dimensional image pixel coordinate set corresponding to the skin marker feature point set and the three-dimensional coordinate set of feature points, the coordinate system conversion matrix of the probe coordinate system and the image coordinate system corresponding to the skin marker image is determined. Here, the obtained coordinate system conversion matrix facilitates the subsequent conversion of the image two-dimensional coordinates into three-dimensional coordinates, thereby improving the accuracy of the image-based visual ultrasound positioning. Then, based on the skin marker feature point set and the coordinate system conversion matrix, the expected three-dimensional position information of the ultrasound probe for the initial position information is determined. Here, the accuracy of the expected three-dimensional position information can be improved.Then, according to the above-mentioned axial displacement value and the preset probe pressure information, the probe position of the expected three-dimensional position information of the above-mentioned ultrasonic probe is adjusted to obtain the adjusted position information. Here, by adjusting the probe positioning of the ultrasonic probe based on the image-based visual ultrasonic probe positioning through the mechanical angle, the accuracy of the ultrasonic probe positioning can be further improved, and the distance to the skin corresponding to the adjusted position information can be reduced, thereby improving the signal loss of the ultrasonic probe received by the ultrasonic probe. Finally, according to the above-mentioned adjusted position information, the above-mentioned ultrasonic probe is controlled to generate a skin ultrasonic image at the above-mentioned adjusted position information. Here, the quality of the skin ultrasonic image generated by the ultrasonic probe can be improved. It can be obtained that the ultrasonic probe positioning can improve the accuracy of the ultrasonic probe positioning through the mechanical ultrasonic probe positioning based on the visual ultrasonic probe of the skin marker image and the axial displacement value, thereby improving the quality of the generated ultrasonic probe. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flow chart of some embodiments of the ultrasound probe positioning method according to the present disclosure;

[0014] Figure 2 is a schematic structural diagram of some embodiments of the ultrasound probe positioning device according to the present disclosure;

[0015] Figure 3 is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure;

[0016] Figure 4 is a product diagram of a medical imaging robot arm in some embodiments of the ultrasound probe positioning method disclosed herein;

[0017] Figure 5 is a product diagram of a sensor excitation coil in some embodiments of the ultrasound probe positioning method according to the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1 The process 100 of some embodiments of the ultrasound probe positioning method according to the present disclosure is shown. The ultrasound probe positioning method comprises the following steps:

[0025] Step 101, obtaining the current electromagnetic signal of the sensor excitation coil in the medical imaging robot arm, the skin marker image set and the axial displacement value of the ultrasound probe.

[0026] In some embodiments, the execution subject (e.g., electronic device) of the above-mentioned ultrasound probe positioning method can obtain the current electromagnetic signal of the sensor excitation coil, the skin marker image set, and the axial displacement value of the ultrasound probe in the medical imaging robot arm through a wired connection or a wireless connection. The above-mentioned medical imaging robot arm can be a device for medical imaging by placing an ultrasound probe and / or other medical probes. The above-mentioned medical imaging robot arm is as follows: Figure 4 As shown. The sensor exciting coil may be a shear wave transducer that emits electromagnetic signals. Figure 5As shown. The above-mentioned current electromagnetic signal may be a signal currently emitted by the above-mentioned sensor excitation coil and received by the sensor in the above-mentioned ultrasound probe. The skin marker image in the above-mentioned skin marker image set may be a skin image with a marker collected by a camera placed on the above-mentioned medical imaging robotic arm and adjacent to the ultrasound probe. The above-mentioned marker may be an elliptical marker that is pre-marked in the area to be detected to facilitate the ultrasound probe to perform ultrasound imaging of the skin area corresponding to the above-mentioned marker. The above-mentioned axial displacement value may be the moving distance of the above-mentioned ultrasound probe on the vertical axis on the above-mentioned medical imaging robotic arm. The above-mentioned axial displacement value may characterize the pressure information of the above-mentioned ultrasound probe and the skin of the area to be detected.

[0027] Step 102: determining the initial position information of the ultrasound probe in the medical imaging robot arm according to the current electromagnetic signal.

[0028] In some embodiments, the execution subject may determine the initial position information of the ultrasound probe in the medical imaging robot arm according to the current electromagnetic signal, wherein the initial position information may be the initial position information of the ultrasound probe in the area to be detected.

[0029] In some optional implementations of some embodiments, the step of determining the initial position information of the ultrasound probe in the imaging mechanical arm according to the current electromagnetic signal may include the following steps:

[0030] The first step is to obtain the historical electromagnetic signal set of the imaging robot arm and the historical position information set of the ultrasound probe corresponding to the historical electromagnetic signal set. The historical electromagnetic signal in the historical electromagnetic signal set may be the electromagnetic signal emitted by the sensor excitation coil before the current time. The historical position information in the historical position information set may be the position information of the ultrasound probe on the skin of the area to be detected before the current time.

[0031] The second step is to preprocess the historical electromagnetic signal set and the historical position information set to obtain a preprocessed historical electromagnetic signal set and a preprocessed historical position information set.

[0032] The third step is to perform two-dimensional curve fitting on the pre-processed historical electromagnetic signal set and the pre-processed historical position information set to obtain an electromagnetic position fitting curve function. The electromagnetic position fitting curve function can characterize the correlation between the pre-processed historical electromagnetic signal set and the pre-processed historical position information set. The two-dimensional curve fitting can be a curve fitting performed by a regression analysis algorithm.

[0033] The fourth step is to input the current electromagnetic signal into the electromagnetic position fitting curve function to obtain the initial position information of the ultrasonic probe.

[0034] Step 103: extract region of interest images from the skin marker image set to obtain a skin marker region of interest image set.

[0035] In some embodiments, the execution subject may extract the region of interest image from the skin marker image set to obtain the skin marker region of interest image set. The skin marker region of interest image in the skin marker region of interest image set may be a cropped image including the skin marker obtained by window cropping the skin marker image. In practice, the execution subject may input the skin marker image set into a twin fully convolutional neural network to obtain a skin marker region of interest image set.

[0036] Step 104: Perform skin marker recognition processing on the skin marker region of interest image set to obtain a skin marker feature point set.

[0037] In some embodiments, the execution subject may perform skin marker recognition processing on the skin marker region of interest image set to obtain a skin marker feature point set, wherein the skin marker feature points in the skin marker feature point set may be identified feature points that can characterize feature information of the skin marker region of interest.

[0038] In some optional implementations of some embodiments, performing skin marker recognition processing on the skin marker region of interest image set to obtain a skin marker feature point set may include the following steps:

[0039] In the first step, for each skin marker region of interest image in the skin marker region of interest image set, the following ellipse contour fitting step is performed:

[0040] Sub-step 1: grayscale the skin marker region of interest image to obtain a grayscale skin region of interest image.

[0041] Sub-step 2: Perform Gaussian filtering on the grayscale skin region of interest image to obtain a filtered skin region of interest image.

[0042] Sub-step 3, performing morphological opening operation on the above-mentioned filtered skin region of interest image to obtain a target skin region of interest image. The above-mentioned target skin region of interest image may be an image after removing small edge objects and bright spot noise from the above-mentioned filtered skin region of interest image. The above-mentioned morphological opening operation may be to first perform an erosion operation on the above-mentioned filtered skin region of interest image to eliminate the small edges or connecting parts of objects in the above-mentioned filtered skin region of interest image so as to make it easier to separate the objects; and then perform an expansion operation on the image after the erosion operation to remove small holes, small objects and bright spot noise in the image to maintain the completeness of the image. The above-mentioned morphological opening operation is performed twice.

[0043] Sub-step 4, performing Canny edge detection on the target skin region of interest image to obtain a skin region of interest contour image. The skin region of interest contour image may be an image from which the contour of the object in the target skin region of interest image is extracted. The Canny edge detection may be an edge detection performed using a Canny edge detection algorithm.

[0044] Sub-step 5, performing a morphological closing operation on the above-mentioned skin region of interest contour image to obtain a target skin region of interest contour image. The above-mentioned target skin region of interest contour image may be obtained by first dilating the above-mentioned skin region of interest contour image to fill the internal holes and broken parts in the above-mentioned skin region of interest contour image to make the processed image more complete; and then performing an erosion operation on the dilated image. The above-mentioned morphological closing operation may be performed twice.

[0045] Sub-step 6, performing ellipse contour recognition processing on the above target skin contour image of interest to obtain an ellipse contour feature point group, wherein the ellipse contour feature point in the above ellipse contour feature point group is the central feature point of the ellipse contour. The ellipse contour feature point in the above ellipse contour feature point group can be the center point of the extracted ellipse skin marker.

[0046] In the second step, each obtained ellipse contour feature point group is determined as a skin marker feature point set.

[0047] In some optional implementations of some embodiments, performing ellipse contour recognition processing on the target skin contour image of interest to obtain an ellipse contour feature point group may include the following steps:

[0048] In the first step, in response to determining that the target skin contour image of interest is not the target skin contour image of interest at the initial displacement position, the target skin contour image of interest is subjected to elliptical contour recognition processing to obtain an initial elliptical contour group, wherein the initial elliptical contour in the initial elliptical contour group can be an elliptical contour obtained by elliptical contour recognition of the target skin contour image of interest.

[0049] The second step is to determine the ellipse distance value of any two adjacent initial ellipse contours in the initial ellipse contour group to obtain an ellipse distance value group, wherein the ellipse distance value may be a pixel distance value between the center points of any two adjacent initial ellipse contours.

[0050] In the third step, in response to determining that there is at least one ellipse distance value greater than or equal to the preset ellipse distance threshold in the above-mentioned ellipse distance value group, the ellipse contour distance value of each previous ellipse contour in the previous ellipse contour group and each initial ellipse contour in the initial ellipse contour group in the previous target skin contour image of interest is determined as the target ellipse contour distance value, and the target ellipse contour distance value group is obtained. Among them, the above-mentioned preset ellipse distance threshold may be the maximum value of the preset ellipse distance value. The above-mentioned preset ellipse distance threshold may be the ellipse contours that are repeatedly detected. The above-mentioned previous target skin contour image of interest may be an image located in the previous frame of the above-mentioned target skin contour image of interest. The previous ellipse contour in the above-mentioned previous ellipse contour group may be the ellipse contour in the above-mentioned target skin contour image of interest.

[0051] In step 4, in response to determining that the target ellipse contour distance value group is not within the preset ellipse contour distance threshold range, the previous ellipse contour group is determined as the target ellipse contour group of the target skin contour image of interest. The preset ellipse contour distance threshold range may be the maximum and minimum values ​​of the preset ellipse contour distance values.

[0052] The fifth step is to determine the central feature point group of the target ellipse contour group as the ellipse contour feature point group. The fourth and fifth steps can be steps for generating the ellipse contour feature point group under the condition that the target ellipse contour distance value group is not within the preset ellipse contour distance threshold range.

[0053] In step 6, in response to determining that there is at least one target ellipse contour distance value within the preset ellipse distance threshold in the target ellipse contour distance value group, the at least one target ellipse contour distance value is grouped to obtain an ellipse contour distance group value set. The ellipse contour distance group value set in the ellipse contour distance group value set may be an ellipse contour distance group value set having the same ellipse contour as the previous ellipse contour corresponding to the target ellipse contour distance value.

[0054] Step 7: For each ellipse contour distance grouping value group in the ellipse contour distance grouping value group set, perform the following determination steps:

[0055] Sub-step 1, determining the ellipse area difference between the ellipse area set of the target ellipse contour set corresponding to the above-mentioned ellipse contour distance grouping value group and the ellipse area of ​​the previous target ellipse contour, and obtaining the ellipse area difference set, wherein the above-mentioned previous target ellipse contour is the same previous ellipse contour corresponding to the above-mentioned ellipse contour distance grouping value group.

[0056] Sub-step 2: Select the ellipse area difference value with the smallest absolute value of the corresponding ellipse area difference from the above ellipse area difference value set as the target ellipse area difference value.

[0057] Sub-step 3, determining the target grouping ellipse contour by the target ellipse contour corresponding to the target ellipse area difference.

[0058] Step 8: The central feature point group of each target group ellipse contour is determined as the ellipse contour feature point group. The sixth to eighth steps can be steps of generating the ellipse contour feature point group under the condition that the target ellipse contour distance value group is within the preset ellipse contour distance threshold range.

[0059] In some optional implementations of some embodiments, in response to determining that there is at least one ellipse distance value greater than or equal to a preset ellipse distance threshold in the ellipse distance value group, determining the ellipse contour distance value of each previous ellipse contour in the previous ellipse contour group in the previous target skin contour image of interest and each initial ellipse contour in the initial ellipse contour group as the target ellipse contour distance value, after obtaining the target ellipse contour distance value group, the method may further include the following steps:

[0060] In the first step, in response to determining that there is at least one ellipse distance value in the above-mentioned ellipse distance value group that is less than a preset ellipse distance threshold, the ellipse distance mean of the two target ellipse contours corresponding to each ellipse distance value in the above-mentioned at least one ellipse distance value is determined to obtain an ellipse distance mean set.

[0061] In the second step, the ellipse contours corresponding to the above ellipse distance mean value set are determined as the target ellipse contour set corresponding to the above at least one ellipse distance value.

[0062] In some optional implementations of some embodiments, the above-mentioned ellipse contour recognition processing is performed on the above-mentioned target skin contour image of interest to obtain an initial ellipse contour group, which may include the following steps:

[0063] The first step is to perform smoothing filtering on the above-mentioned contour image of interest of the target skin to obtain a filtered contour image of interest of the target skin. The above-mentioned smoothing filtering can be filtering processing performed on the gray value mean of the above-mentioned contour image of interest of the target skin.

[0064] The second step is to generate a contour gradient image of the above-mentioned filtered target skin contour image of interest. The above-mentioned contour gradient image may be an image that characterizes the grayscale change rate and change direction of the above-mentioned filtered target skin contour image of interest.

[0065] The third step is to determine the local maximum pixel set in the above contour gradient image. The local maximum pixel in the above local maximum pixel set can be the pixel with the largest maximum value corresponding to the pixel value in the above contour gradient image. In practice, the above execution subject can use the Laplace operator to determine the local maximum pixel set in the above contour gradient image.

[0066] The fourth step is to sample candidate points of the above local maximum pixel point set to obtain a sampling candidate point set. The above candidate point sampling can be random sampling. The sampling candidate points in the above sampling candidate point set can be sampling points obtained by random sampling from the above local maximum pixel point.

[0067] The fifth step is to perform interpolation processing on the above-mentioned sampling candidate point set to obtain a pixel threshold surface. The above-mentioned pixel threshold surface can be a surface formed by a continuous pixel threshold set. The above-mentioned interpolation processing can be an interpolation processing performed using a bilinear interpolation algorithm.

[0068] Step 6: According to the pixel threshold surface, the target skin contour image of interest is segmented to obtain a skin contour image segmentation region set. The skin contour image of interest segmentation region set may be a region set obtained by performing pixel gray value segmentation on the target skin contour image of interest using a pixel threshold set corresponding to the pixel threshold surface.

[0069] As an example, the execution subject may use a watershed segmentation algorithm to perform image segmentation processing on the target skin contour image of interest according to the pixel threshold surface to obtain a skin contour image of interest segmentation region set.

[0070] In the seventh step, the above-mentioned skin contour image segmentation region set is subjected to regional fusion processing to obtain a contour region fusion region set. Among them, the contour region fusion region in the above-mentioned contour region fusion region set can be a region obtained by fusing the above-mentioned skin contour image segmentation region of interest. In practice, the above-mentioned execution subject uses the RAG (Region Adjacency Graph) algorithm to perform regional fusion processing on the above-mentioned skin contour image segmentation region set of interest to obtain a contour region fusion region set.

[0071] In the eighth step, edge contour extraction is performed on each contour region fusion region in the contour region fusion region set to generate an elliptical region edge contour and obtain an elliptical region edge contour set. The elliptical region edge contour in the elliptical region edge contour set may be a contour obtained by extracting the elliptical contour in the contour region fusion region.

[0072] In the ninth step, the ellipse contour set corresponding to the ellipse region edge contour set is determined as the initial ellipse contour group.

[0073] Step 105 , determining the three-dimensional coordinates of the feature point corresponding to each skin marker feature point in the skin marker feature point set in the probe coordinate system where the ultrasound probe is located, to obtain a feature point three-dimensional coordinate set.

[0074] In some embodiments, the execution subject may determine the three-dimensional coordinates of the feature points corresponding to each skin marker feature point in the skin marker feature point set in the probe coordinate system where the ultrasound probe is located, and obtain the feature point three-dimensional coordinate set. The three-dimensional coordinates of the feature points in the feature point three-dimensional coordinate set may be the coordinates of the skin marker feature point in the three-dimensional coordinate system where the ultrasound probe is located.

[0075] Step 106 , determining a coordinate system conversion matrix between the probe coordinate system and the image coordinate system corresponding to the skin marker image according to the two-dimensional image pixel coordinate set corresponding to the skin marker feature point set and the three-dimensional coordinate set of the feature points.

[0076] In some embodiments, the execution subject may determine a coordinate system conversion matrix between the probe coordinate system and the image coordinate system corresponding to the skin marker image based on the two-dimensional image pixel coordinate set corresponding to the skin marker feature point set and the three-dimensional coordinate set of the feature point. The coordinate system conversion matrix may characterize the conversion relationship between the two-dimensional image coordinates corresponding to the skin marker feature point and the three-dimensional coordinates where the ultrasound probe is located.

[0077] As an example, the execution entity may use a singular value decomposition method to determine a coordinate system transformation matrix between the probe coordinate system and the image coordinate system corresponding to the skin marker image according to the two-dimensional image pixel coordinate set corresponding to the skin marker feature point set and the three-dimensional coordinate set of the feature points.

[0078] Step 107 : determining the expected three-dimensional position information of the ultrasound probe with respect to the initial position information according to the skin marker feature point set and the coordinate system conversion matrix.

[0079] In some embodiments, the execution subject may determine the expected three-dimensional position information of the ultrasound probe for the initial position information based on the skin marker feature point set and the coordinate system conversion matrix. The expected three-dimensional position information may be the position information obtained by adjusting the initial position information of the ultrasound probe determined by the skin marker feature points.

[0080] As an example, the execution subject may utilize position visual servoing and posture transformation algorithms to determine the desired three-dimensional position information of the ultrasound probe with respect to the initial position information according to the skin marker feature point set and the coordinate system transformation matrix.

[0081] In the process of adopting technical solutions to solve the above-mentioned technical problem one, the following technical problem two is often accompanied: Since the ultrasound probe and the skin marker region of interest image are two relatively independent coordinate systems, to locate the ultrasound probe, it is necessary to convert the skin marker region of interest image to the three-dimensional coordinate system where the ultrasound probe is located. There are inaccurate coordinate system conversion parameters, which leads to low positioning accuracy of the ultrasound probe and low quality of the generated skin image. For the above-mentioned technical problem two, the conventional solution is generally: using the Eagle optimizer algorithm to optimize the conversion parameter set corresponding to the coordinate system conversion matrix to obtain the optimized conversion parameter set. However, the above-mentioned conventional solution still has the following technical problems: due to the insufficient local search capability of the Eagle optimizer algorithm, it is easy to fall into the local optimal solution, resulting in low algorithm search efficiency, long optimization time and low optimization accuracy of the conversion parameter set corresponding to the coordinate system conversion matrix, resulting in low accuracy of ultrasound probe positioning. Combined with the advantages / technical status of ultrasound probe positioning, it can be decided to adopt the following solution:

[0082] In some optional implementations of some embodiments, determining the expected three-dimensional position information of the ultrasound probe for the initial position information according to the skin marker feature point set and the coordinate system conversion matrix may include the following steps:

[0083] The first step is to construct a feature point reprojection target fitness function. The feature point reprojection target fitness function can be to reproject each feature point in the feature point set and average the reprojection error between the reprojected feature point coordinates and the actual feature point coordinates. The feature point reprojection target fitness function can be expressed as:

[0084]

[0085] Where E represents the feature point reprojection target fitness function. S represents the number of skin marker region of interest images corresponding to the skin marker feature point set. h represents the number of feature points included in the feature point set. p ij represents the real pixel coordinates of the i-th feature point on the j-th skin marker region of interest image. ij Indicates p ij The corresponding reprojected pixel coordinates. f1, f2, u0, v0 represent the transformation parameter set corresponding to the coordinate system transformation matrix.

[0086] The second step is to generate an initial population according to the conversion parameter set corresponding to the coordinate system conversion matrix, wherein each initial individual in the initial population can be an individual formed by combining a set of conversion parameter sets.

[0087] As an example, the execution subject may first determine the parameter optimization interval of the conversion parameter set corresponding to the coordinate system conversion matrix. The parameter optimization interval may be a numerical interval of adding or subtracting [0, 45] on the basis of the conversion parameter set. Then, the conversion parameter set is added with the internal optimization interval to obtain a conversion parameter set group, and is determined as the initial population.

[0088] The third step is to perform the following population update steps based on the initial population:

[0089] Sub-step 1, inputting the initial individuals included in the initial population into the above-mentioned feature point reprojection target fitness function to obtain an initial reprojection fitness value set. The initial reprojection fitness value in the above-mentioned initial reprojection fitness value set can characterize the quality of the skin ultrasound image taken by the ultrasound image of the initial individual.

[0090] Sub-step 2, screening the initial reprojection fitness value set to obtain the screened initial reprojection fitness value. The screened initial reprojection fitness value may be the initial reprojection fitness value with the largest value.

[0091] Sub-step 3, determining the average value of the initial reprojection fitness value set to obtain the initial reprojection fitness mean.

[0092] Sub-step 4, for each initial individual included in the initial population, generate an adaptive update allocation parameter according to the initial reprojection fitness value corresponding to the initial individual, the initial reprojection fitness value after screening, the initial reprojection fitness mean and the preset tuning parameter set. The above-mentioned adaptive update allocation parameter can represent the probability value of the initial individual selecting different update strategies in the subsequent update process. The above-mentioned preset tuning parameter can be a pre-set parameter that represents the optimization of the population. The above-mentioned adaptive update allocation parameter can be expressed as:

[0093]

[0094] Wherein, M(t) represents the adaptive update allocation parameter. represents the local optimization adjustment parameter included in the preset tuning parameter set, and its value range is (0, 0.5). φ represents the global optimization adjustment parameter included in the preset tuning parameter set, and its value range is (0.25, 0.75). F(t) represents the initial reprojection fitness value corresponding to the initial individual. F B (t) represents the initial reprojection fitness value after screening. F M (t) represents the initial reprojection fitness mean. t represents the number of executions.

[0095] Sub-step 5, determining the number of times the above population update step has been executed.

[0096] Sub-step 6, for each adaptive update allocation parameter in the adaptive update allocation parameter set, perform the following update steps:

[0097] The first sub-step is to determine the position dynamic update parameter in response to determining that the number of executions is less than or equal to the product of the preset execution threshold and the first preset threshold, and the preset random threshold is less than or equal to the adaptive update allocation parameter. The preset execution threshold may be a preset maximum number of executions. For example, the preset execution threshold may be 100. The first preset threshold may be a preset value. For example, the first preset threshold may be two-thirds. The preset random threshold may be a random number generated by a random function. The position dynamic update parameter may be characterized as assigning a weight value to the position of the initial individual to avoid falling into a local extreme value. The position dynamic update parameter may be expressed as:

[0098]

[0099] Where C(t) represents the dynamic update parameter of the position. N represents the number of initial individuals included in the initial population. i (t) represents the i-th initial individual in the initial population. i (t) represents X i (t) The corresponding initial reprojection fitness value.

[0100] The second sub-step is to perform a first position update process on the initial individual corresponding to the adaptive update allocation parameter according to the position dynamic update parameter and the adaptive update allocation parameter, and obtain a first updated individual as the updated individual. The updated individual can be expressed as:

[0101]

[0102] Among them, X new (t+1) represents the updated individual. best (t) represents the individual with the largest initial reprojection fitness value obtained in the tth cycle. T0 represents the preset execution threshold. M (t) represents the mean of the initial reprojected fitness value set corresponding to the population obtained in the tth cycle. r represents any random number in the range of [0, 1].

[0103] The third sub-step is to determine the spiral tidal fluctuation parameter in response to determining that the preset random threshold is greater than the adaptive update allocation parameter. The spiral tidal fluctuation parameter can represent the adjustment of the search step size of the initial individual. The spiral tidal fluctuation parameter can be expressed as:

[0104]

[0105] Where W(t) represents the spiral tidal fluctuation parameter. ε represents the wavelength constant, which represents the amplitude of tidal fluctuation and has a value of 1.5.

[0106] The fourth sub-step is to perform a second position update process on the initial individual corresponding to the adaptive update allocation parameter according to the spiral tidal fluctuation parameter to obtain a second updated individual as the updated individual. The updated individual can be expressed as:

[0107] X new (t+1)=X best (t)×G levy +W(t)×r+X R (t).

[0108] Among them, G levy represents the Levy branch distribution function. X R (t) represents the random individual obtained in the tth cycle.

[0109] In a fifth sub-step, in response to determining that the number of executions is greater than the product of the preset execution threshold and the first preset threshold, and the preset random threshold is less than or equal to the adaptive update allocation parameter, a third position update process is performed on the initial individual corresponding to the adaptive update allocation parameter according to the preset search area range to obtain a third updated individual as the updated individual. The preset search area range may be a preset optimization range of the conversion parameter set. The updated individual may be represented as:

[0110] X new (t+1)=[X best (t)-X M (t)]×ζ-r+[(UB-LB)×r+LB]×η.

[0111] Wherein, ζ represents the first development adjustment parameter, and its value is 0.1. UB represents the upper limit of the preset search area range. LB represents the lower limit of the preset search area range. η represents the second development adjustment parameter, and its value is 0.1.

[0112] The sixth sub-step is, in response to determining that the preset random threshold is greater than the adaptive update allocation parameter, performing a fourth position update process on the initial individual corresponding to the adaptive update allocation parameter as an updated individual. The updated individual can be represented as:

[0113]

[0114] Among them, Q F represents the equilibrium function used for equilibrium search. G1 represents the flight search movement of the individual. G2 represents the flight slope value of the individual in linear decreasing, and the value range is [0, 2]. X(t) represents the individual obtained in the tth cycle.

[0115] Sub-step 7, inputting the obtained updated population into the above-mentioned feature point reprojection target fitness function to obtain an updated reprojection fitness value set.

[0116] Sub-step 8, screening the updated reprojection fitness value set to obtain a target updated reprojection fitness value, wherein the target updated reprojection fitness value may be the updated reprojection fitness value with the largest value.

[0117] Sub-step 9, in response to determining that the target updated reprojection fitness value is less than the screened initial reprojection fitness value, determining the updated population as the initial population, and performing the above population updating step again.

[0118] Sub-step 10, in response to determining that the number of executions is greater than or equal to the above-mentioned preset execution threshold, determining the conversion parameter set included in the updated individual corresponding to the target updated reprojection fitness value as the target conversion parameter set.

[0119] In step 4, in response to determining that the number of executions is less than or equal to the preset execution threshold, the updated population is determined as the initial population, and the sum of the number of executions and the second preset threshold is determined as the number of executions, so as to perform the population update step again. The second preset threshold may be a preset value. For example, the second preset threshold may be 1.

[0120] Step 5: Determine the expected three-dimensional position information corresponding to the feature point set of the skin marker region of interest image according to the target conversion parameter set. The expected three-dimensional position information can be the coordinate information of the feature point in the world coordinate system obtained by substituting the pixel coordinates of the feature point into the coordinate system conversion formula determined by the calibrated target conversion parameter set.

[0121] As an example, the execution subject may input the skin marker feature point set into a target coordinate system conversion formula to obtain desired three-dimensional position information.

[0122] The above-mentioned first to fifth steps and their related contents, combined with step "step 109", as an inventive point of an embodiment of the present disclosure, solve the technical problem 2 mentioned in the background technology "Due to the insufficient local search ability of the Eagle optimizer algorithm, it is easy to fall into the local optimal solution, resulting in low algorithm search efficiency, long conversion parameter set optimization time and low optimization accuracy, which leads to low accuracy of ultrasound probe positioning and low quality of skin ultrasound images generated by the ultrasound probe. The factors are often as follows: Due to the insufficient local search ability of the Eagle optimizer algorithm, it is easy to fall into the local optimal solution, resulting in low algorithm search efficiency, long conversion parameter set optimization time and low optimization accuracy, which leads to low accuracy of ultrasound probe positioning and low quality of skin ultrasound images generated by the ultrasound probe. In order to achieve this effect, the present disclosure first constructs a feature point reprojection target fitness function to determine the optimization effect and improve the ultrasound probe positioning accuracy. Secondly, through the initial reprojection fitness value corresponding to the initial individual, the initial reprojection fitness value after screening, the initial reprojection fitness mean and the preset adjustment The optimal parameter set is used to generate adaptive update allocation parameters. The adaptive update allocation parameters can flexibly select individual position update strategies, enhance global exploration capabilities, improve search capabilities and search efficiency, and shorten optimization time. Again, position dynamic update parameters and spiral tidal fluctuation parameters are introduced into the position update strategy. The position dynamic update parameters can avoid falling into local extreme values ​​and improve the accuracy of parameter optimization; the spiral tidal fluctuation parameters give individuals a larger random search step in the early stage of the loop iteration, which can increase the global search ability of the population; in the middle and late stages of the loop iteration, the individual search step slowly weakens with the tidal fluctuation, which can improve the local search ability. At the same time, the tidal fluctuation has a nonlinear effect, which can avoid falling into the local optimal value to a certain extent. Then, the third position update processing and the fourth position update processing in the position update can further narrow the population search range and improve the parameter optimization efficiency. Finally, through the above-mentioned conversion parameter set, the expected three-dimensional position information corresponding to the feature point set of the image of the area of ​​interest of the skin marker is determined, which can improve the accuracy of camera calibration.

[0123] Step 108 , adjusting the probe position of the desired three-dimensional position information of the ultrasonic probe according to the axial displacement value and the preset probe pressure information, to obtain adjusted position information.

[0124] In some embodiments, the execution subject may adjust the probe position of the desired three-dimensional position information of the ultrasound probe according to the axial displacement value and the preset probe pressure information to obtain adjusted position information. The preset probe pressure information may be a preset probe pressure value of the skin corresponding to the probe and the desired three-dimensional position information. The adjusted position information may be information of the position obtained by adjusting the axial displacement value of the ultrasound probe to conform to the preset probe pressure information.

[0125] As an example, the execution subject may firstly perform regression prediction on the collected historical ultrasonic probe pressure information and historical axial displacement values ​​to obtain a pressure-axial displacement correlation function of the probe pressure information and the axial displacement value. Then, the preset probe pressure information is input into the pressure-axial displacement correlation function to obtain a target axial displacement value. Finally, the ultrasonic probe is shifted toward the target axial displacement value based on the axial displacement value to obtain adjusted position information.

[0126] Step 109: Based on the adjusted position information, control the ultrasound probe to generate a skin ultrasound image at the adjusted position information.

[0127] In some embodiments, the execution subject may control the ultrasound probe to generate a skin ultrasound image at the adjusted position information according to the adjusted position information. The skin ultrasound image may be an image obtained by controlling the ultrasound probe to perform ultrasound imaging at the skin area to be detected corresponding to the adjusted position information.

[0128] As an example, the execution subject may control the ultrasound probe to move to the adjusted position information to generate a skin ultrasound image.

[0129] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the ultrasonic probe positioning method of some embodiments of the present disclosure can improve the accuracy of ultrasonic probe positioning through the mechanical ultrasonic probe positioning based on the visual ultrasonic probe of the skin marker image and the axial displacement value, thereby improving the quality of the generated ultrasonic probe. Specifically, the reason why the positioning accuracy of the relevant ultrasonic probe is low, thereby reducing the quality of the ultrasonic image collected by the ultrasonic probe, and there is a lot of useless information in the ultrasonic image is that: since the calibration and positioning of the ultrasonic probe is performed only through the collected ultrasonic image, the positioning factors considered are too single, and there are a large number of coordinate system conversion operations in the probe positioning through the ultrasonic image, the positioning process is relatively cumbersome, resulting in a low positioning accuracy of the ultrasonic probe, thereby reducing the quality of the ultrasonic image collected by the ultrasonic probe, and there is a lot of useless information in the ultrasonic image. Based on this, the ultrasonic probe positioning method of some embodiments of the present disclosure can first obtain the current electromagnetic signal of the sensor excitation coil in the medical imaging manipulator, the skin marker image set and the axial displacement value of the ultrasonic probe. Here, the current electromagnetic signal facilitates the subsequent determination of the initial position information of the ultrasonic probe, and the skin marker image set and the axial displacement value facilitate the subsequent positioning of the ultrasonic probe based on vision and mechanics. Secondly, according to the current electromagnetic signal, the initial position information of the ultrasonic probe in the medical imaging manipulator is determined. Here, the initial position information of the ultrasonic probe is determined, which can shorten the positioning range of the ultrasonic probe. Thirdly, the image of the region of interest is extracted from the skin marker image set to obtain the skin marker region of interest image set. Here, the recognition range of the skin marker can be further narrowed, which is convenient for improving the recognition efficiency in the future. Next, the skin marker region of interest image set is processed for skin marker recognition to obtain a skin marker feature point set. Here, the recognition efficiency and accuracy of the skin marker can be improved. Subsequently, the three-dimensional coordinates of the feature points corresponding to each skin marker feature point in the skin marker feature point set in the probe coordinate system where the ultrasonic probe is located are determined to obtain a three-dimensional coordinate set of feature points. Here, the three-dimensional coordinate set of feature points facilitates the subsequent determination of the conversion matrix of the coordinate system of the ultrasonic image and the skin marker image. Afterwards, according to the two-dimensional image pixel coordinate set corresponding to the skin marker feature point set and the three-dimensional coordinate set of feature points, the coordinate system conversion matrix of the probe coordinate system and the image coordinate system corresponding to the skin marker image is determined. Here, the obtained coordinate system conversion matrix facilitates the subsequent conversion of the image two-dimensional coordinates into three-dimensional coordinates, thereby improving the accuracy of the image-based visual ultrasound positioning. Then, based on the skin marker feature point set and the coordinate system conversion matrix, the expected three-dimensional position information of the ultrasound probe for the initial position information is determined. Here, the accuracy of the expected three-dimensional position information can be improved.Then, according to the above-mentioned axial displacement value and the preset probe pressure information, the probe position of the expected three-dimensional position information of the above-mentioned ultrasonic probe is adjusted to obtain the adjusted position information. Here, by adjusting the probe positioning of the ultrasonic probe based on the image-based visual ultrasonic probe positioning through the mechanical angle, the accuracy of the ultrasonic probe positioning can be further improved, and the distance to the skin corresponding to the adjusted position information can be reduced, thereby improving the signal loss of the ultrasonic probe received by the ultrasonic probe. Finally, according to the above-mentioned adjusted position information, the above-mentioned ultrasonic probe is controlled to generate a skin ultrasonic image at the above-mentioned adjusted position information. Here, the quality of the skin ultrasonic image generated by the ultrasonic probe can be improved. It can be obtained that the ultrasonic probe positioning can improve the accuracy of the ultrasonic probe positioning through the mechanical ultrasonic probe positioning based on the visual ultrasonic probe of the skin marker image and the axial displacement value, thereby improving the quality of the generated ultrasonic probe.

[0130] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an ultrasound probe positioning device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the ultrasound probe positioning device can be specifically applied to various electronic devices.

[0131] like Figure 2As shown, an ultrasonic probe positioning device 200 includes: an acquisition unit 201, a first determination unit 202, an area of ​​interest image extraction unit 203, a skin marker recognition unit 204, a second determination unit 205, a third determination unit 206, a fourth determination unit 207, a probe position adjustment unit 208 and a control unit 209. The acquisition unit 201 is configured to: acquire the current electromagnetic signal of the sensor excitation coil in the medical imaging manipulator, the skin marker image set and the axial displacement value of the ultrasonic probe. The first determination unit 202 is configured to: determine the initial position information of the ultrasonic probe in the medical imaging manipulator according to the current electromagnetic signal. The area of ​​interest image extraction unit 203 is configured to: perform area of ​​interest image extraction on the skin marker image set to obtain a skin marker area of ​​interest image set. The skin marker recognition unit 204 is configured to: perform skin marker recognition processing on the skin marker area of ​​interest image set to obtain a skin marker feature point set. The second determination unit 205 is configured to: determine the three-dimensional coordinates of the feature points corresponding to each skin marker feature point in the skin marker feature point set in the probe coordinate system where the ultrasound probe is located, and obtain the feature point three-dimensional coordinate set. The third determination unit 206 is configured to: determine the coordinate system conversion matrix of the probe coordinate system and the image coordinate system corresponding to the skin marker image according to the image two-dimensional pixel coordinate set corresponding to the skin marker feature point set and the feature point three-dimensional coordinate set. The fourth determination unit 207 is configured to: determine the expected three-dimensional position information of the ultrasound probe for the initial position information according to the skin marker feature point set and the coordinate system conversion matrix. The probe position adjustment unit 208 is configured to: perform probe position adjustment on the expected three-dimensional position information of the ultrasound probe according to the axial displacement value and the preset probe pressure information, and obtain the adjusted position information. The control unit 209 is configured to: control the ultrasound probe to generate the skin ultrasound image at the adjusted position information according to the adjusted position information.

[0132] It can be understood that the units described in the ultrasonic probe positioning device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the ultrasound probe positioning device 200 and the units included therein, and will not be described in detail here.

[0133] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0134] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0135] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0136] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0137] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0138] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0139] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the current electromagnetic signal of the sensor excitation coil in the medical imaging robot arm, the skin marker image set and the axial displacement value of the ultrasonic probe; determines the initial position information of the ultrasonic probe in the above-mentioned medical imaging robot arm according to the above-mentioned current electromagnetic signal; performs region of interest image extraction on the above-mentioned skin marker image set to obtain a skin marker region of interest image set; performs skin marker recognition processing on the above-mentioned skin marker region of interest image set to obtain a skin marker feature point set; determines the location of each skin marker feature point in the above-mentioned skin marker feature point set corresponding to the location of the above-mentioned ultrasonic probe. The three-dimensional coordinates of the feature points in the probe coordinate system are used to obtain a three-dimensional coordinate set of the feature points; the coordinate system conversion matrix of the probe coordinate system and the image coordinate system corresponding to the skin marker image is determined based on the two-dimensional image pixel coordinate set corresponding to the skin marker feature point set and the three-dimensional coordinate set of the feature points; the expected three-dimensional position information of the ultrasound probe relative to the initial position information is determined based on the skin marker feature point set and the coordinate system conversion matrix; the probe position is adjusted for the expected three-dimensional position information of the ultrasound probe based on the axial displacement value and the preset probe pressure information to obtain the adjusted position information; based on the adjusted position information, the ultrasound probe is controlled to generate a skin ultrasound image at the adjusted position information.

[0140] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be set in a processor, for example, may be described as: a processor including an acquisition unit, a first determination unit, an area of ​​interest image extraction unit, a skin marker recognition unit, a second determination unit, a third determination unit, a fourth determination unit, a probe position adjustment unit, and a control unit. The names of these units do not, in some cases, constitute limitations on the units themselves. For example, the acquisition unit may also be described as "a unit for acquiring the current electromagnetic signal of the sensor excitation coil in the medical imaging robot arm, the skin marker image set, and the axial displacement value of the ultrasound probe."

[0143] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0144] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for positioning an ultrasonic probe, comprising: Obtaining current electromagnetic signals of a sensor excitation coil in a medical imaging robot arm, a skin marker image set, and an axial displacement value of an ultrasound probe; Determining initial position information of the ultrasound probe in the medical imaging robotic arm according to the current electromagnetic signal; Extracting region-of-interest images from the skin marker image set to obtain a skin marker region-of-interest image set; Performing skin marker recognition processing on the skin marker region of interest image set to obtain a skin marker feature point set; Determine the three-dimensional coordinates of the feature point corresponding to each skin marker feature point in the skin marker feature point set in the probe coordinate system where the ultrasound probe is located, to obtain a three-dimensional coordinate set of the feature points; Determine a coordinate system conversion matrix between the probe coordinate system and the image coordinate system corresponding to the skin marker image according to the two-dimensional image pixel coordinate set corresponding to the skin marker feature point set and the three-dimensional coordinate set of the feature points; Determining expected three-dimensional position information of the ultrasound probe with respect to the initial position information according to the skin marker feature point set and the coordinate system transformation matrix; According to the axial displacement value and the preset probe pressure information, the probe position is adjusted for the expected three-dimensional position information of the ultrasonic probe to obtain adjusted position information; According to the adjusted position information, the ultrasound probe is controlled to generate a skin ultrasound image at the adjusted position information.

2. The method according to claim 1, wherein: Determining the initial position information of the ultrasound probe in the medical imaging mechanical arm according to the current electromagnetic signal includes: Acquire a historical electromagnetic signal set of the medical imaging robot arm and a historical position information set of the ultrasound probe corresponding to the historical electromagnetic signal set; Preprocessing the historical electromagnetic signal set and the historical position information set to obtain a preprocessed historical electromagnetic signal set and a preprocessed historical position information set; Performing two-dimensional curve fitting on the preprocessed historical electromagnetic signal set and the preprocessed historical position information set to obtain an electromagnetic position fitting curve function; The current electromagnetic signal is input into the electromagnetic position fitting curve function to obtain the initial position information of the ultrasonic probe.

3. The method according to claim 1, wherein: The step of performing skin marker recognition processing on the skin marker region of interest image set to obtain a skin marker feature point set includes: For each skin marker region of interest image in the skin marker region of interest image set, the following ellipse contour fitting step is performed: gray-scale the skin marker region of interest image to obtain a gray-scale skin region of interest image; Performing Gaussian filtering on the grayscale skin region of interest image to obtain a filtered skin region of interest image; Performing morphological opening processing on the filtered skin region of interest image to obtain a target skin region of interest image; Performing Canny edge detection on the target skin region of interest image to obtain a contour image of the skin region of interest; Performing a morphological closing operation on the contour image of the skin region of interest to obtain a target skin contour image of interest; Performing ellipse contour recognition processing on the target skin contour image of interest to obtain an ellipse contour feature point group, wherein the ellipse contour feature point in the ellipse contour feature point group is a central feature point of the ellipse contour; The obtained ellipse contour feature point groups are determined as skin marker feature point sets.

4. The method according to claim 3, wherein: The step of performing ellipse contour recognition processing on the target skin contour image of interest to obtain an ellipse contour feature point group includes: In response to determining that the target skin contour image of interest is not the target skin contour image of interest shifted to the initial position, performing ellipse contour recognition processing on the target skin contour image of interest to obtain an initial ellipse contour group; Determine the ellipse distance values ​​of any two adjacent initial ellipse contours in the initial ellipse contour group to obtain an ellipse distance value group; In response to determining that there is at least one ellipse distance value greater than or equal to a preset ellipse distance threshold in the ellipse distance value group, determining an ellipse contour distance value of each previous ellipse contour in the previous ellipse contour group and each initial ellipse contour in the initial ellipse contour group in the previous target skin contour image of interest as a target ellipse contour distance value, and obtaining a target ellipse contour distance value group; In response to determining that the target ellipse contour distance value group is not within a preset ellipse contour distance threshold range, determining the previous ellipse contour group as the target ellipse contour group of the target skin contour image of interest; Determine the central feature point group of the target ellipse contour group as the ellipse contour feature point group; In response to determining that there is at least one target ellipse contour distance value located at the preset ellipse distance threshold in the target ellipse contour distance value group, grouping the at least one target ellipse contour distance value to obtain an ellipse contour distance group value set; For each ellipse contour distance grouping value group in the ellipse contour distance grouping value group set, the following determination steps are performed: Determine the ellipse area difference between the ellipse area set of the target ellipse contour set corresponding to the ellipse contour distance grouping value group and the ellipse area of ​​the previous target ellipse contour, and obtain an ellipse area difference value set, wherein the previous target ellipse contour is the same previous ellipse contour corresponding to the ellipse contour distance grouping value group; Selecting an ellipse area difference value with the smallest absolute value of the corresponding ellipse area difference from the ellipse area difference value set as the target ellipse area difference value; Determine the target grouping ellipse contour by using the target ellipse contour corresponding to the target ellipse area difference; The central feature point group of the ellipse contour of each target group obtained is determined as the ellipse contour feature point group.

5. The method according to claim 4, wherein: After, in response to determining that there is at least one ellipse distance value greater than or equal to a preset ellipse distance threshold in the ellipse distance value group, determining an ellipse contour distance value of each previous ellipse contour in the previous ellipse contour group in the previous target skin contour image of interest and each initial ellipse contour in the initial ellipse contour group as a target ellipse contour distance value, and obtaining the target ellipse contour distance value group, the method further includes: In response to determining that there is at least one ellipse distance value less than a preset ellipse distance threshold in the ellipse distance value group, determining an ellipse distance mean of two target ellipse contours corresponding to each ellipse distance value in the at least one ellipse distance value, to obtain an ellipse distance mean value set; The ellipse contours corresponding to the ellipse distance mean value set are determined as the target ellipse contour set corresponding to the at least one ellipse distance value.

6. The method according to claim 4, wherein: The step of performing ellipse contour recognition processing on the target skin contour image of interest to obtain an initial ellipse contour group includes: Performing smoothing filtering on the target skin contour image of interest to obtain a filtered target skin contour image of interest; Generating a contour gradient image of the filtered target skin contour image of interest; Determine a local maximum pixel point set in the contour gradient image; Performing candidate point sampling on the local maximum pixel point set to obtain a sampled candidate point set; Performing interpolation processing on the sampling candidate point set to obtain a pixel threshold surface; According to the pixel threshold surface, performing image segmentation processing on the target skin contour image of interest to obtain a skin contour image of interest segmentation region set; Performing region fusion processing on the skin interesting contour image segmentation region set to obtain a contour region fusion region set; Extracting edge contours of each contour region fusion region in the contour region fusion region set to generate elliptical region edge contours, thereby obtaining an elliptical region edge contour set; The ellipse contour set corresponding to the ellipse region edge contour set is determined as an initial ellipse contour group.

7. An ultrasonic probe positioning device, comprising: An acquisition unit configured to acquire a current electromagnetic signal of an exciting coil of a sensor in a medical imaging robot arm, a skin marker image set, and an axial displacement value of an ultrasound probe; A first determining unit is configured to determine initial position information of the ultrasound probe in the medical imaging robot arm according to the current electromagnetic signal; an area of ​​interest image extraction unit, configured to perform area of ​​interest image extraction on the skin marker image set to obtain a skin marker area of ​​interest image set; a skin marker recognition unit configured to perform skin marker recognition processing on the skin marker region of interest image set to obtain a skin marker feature point set; A second determining unit is configured to determine the three-dimensional coordinates of a feature point corresponding to each skin marker feature point in the skin marker feature point set in the probe coordinate system where the ultrasound probe is located, to obtain a three-dimensional coordinate set of the feature points; A third determining unit is configured to determine a coordinate system conversion matrix between the probe coordinate system and the image coordinate system corresponding to the skin marker image according to the two-dimensional image pixel coordinate set corresponding to the skin marker feature point set and the three-dimensional coordinate set of the feature point; a fourth determining unit, configured to determine expected three-dimensional position information of the ultrasound probe with respect to the initial position information according to the skin marker feature point set and the coordinate system conversion matrix; A probe position adjustment unit is configured to adjust the probe position of the desired three-dimensional position information of the ultrasound probe according to the axial displacement value and the preset probe pressure information to obtain adjusted position information; The control unit is configured to control the ultrasound probe to generate a skin ultrasound image at the adjusted position information according to the adjusted position information.

8. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images