A control method and device of an ultrasonic tracer, a robot, and a storage medium

Abstract

The application is suitable for the technical field of device control, and provides a control method and device of an ultrasonic tracer, electronic equipment and a storage medium, including: obtaining a bone structure model of a target object and first constraint information of the ultrasonic tracer determined based on the bone structure model; for any candidate configuration parameter of the ultrasonic tracer, determining a plurality of candidate poses of an ultrasonic probe of the ultrasonic tracer under the candidate configuration parameter according to the bone structure model and the first constraint information; calculating shooting coverage information of the target region by the ultrasonic tracer under the candidate configuration parameter according to the observation range of the target region by each candidate pose; and determining a control strategy corresponding to the ultrasonic tracer from all candidate configuration parameters and all candidate poses according to the shooting coverage information corresponding to each candidate configuration parameter. The above method can reduce the configuration difficulty of the ultrasonic tracer and improve the imaging accuracy when the ultrasonic tracer is actually controlled to obtain an ultrasonic image.

Description

Technical Field

[0001] This application belongs to the field of equipment control technology, and in particular relates to a control method, device, robot, and storage medium for an ultrasonic tracer. Background Technology

[0002] Tracers, as important instruments in the biomedical field, are frequently used in biological surgery. By combining preoperative images such as computed tomography (CT) or magnetic resonance imaging (MRI), the tracer's navigation path within the organism is determined. Subsequently, the tracer can be controlled to follow this path to acquire ultrasound images of the organism's interior, enabling observation and understanding of the target area and its internal conditions. Therefore, accurate control of the tracer directly impacts the accuracy of internal biological observations.

[0003] With the increasing demand for acquiring information about soft tissues within organisms, ultrasound imaging, as a supplementary imaging technique, has been introduced into the aforementioned navigation process. However, in practical applications, the installation location of the ultrasound tracer and the orientation of the ultrasound probe are usually determined manually, which can easily lead to situations where the tracer's field of view is limited within the organism, making it impossible to acquire ultrasound images of the target area. Therefore, existing tracer control technologies suffer from problems such as high configuration difficulty and low imaging accuracy. Summary of the Invention

[0004] This application provides a control method, device, robot, and storage medium for an ultrasonic tracer, which can solve the problems of high configuration difficulty and low imaging accuracy in existing tracer control technologies.

[0005] In a first aspect, embodiments of this application provide a control method for an ultrasonic tracer, the ultrasonic tracer including an ultrasonic probe, the control method comprising: Obtain the bone structure model of the target object and the first constraint information of the ultrasound tracer determined based on the bone structure model; For any candidate configuration parameters of the ultrasound tracer, based on the bone structure model and the first constraint information, determine multiple candidate poses of the ultrasound probe under the candidate configuration parameters for the ultrasound tracer. Based on the observation range of the target region by each of the candidate poses, the imaging coverage information of the target region by the ultrasound tracer under the candidate configuration parameters is calculated; the target region is the spatial region of the target to be observed in the bone structure model. Based on the imaging coverage information corresponding to each of the candidate configuration parameters, a control strategy corresponding to the ultrasonic tracer is determined from all the candidate configuration parameters and all the candidate poses; the control strategy includes target configuration parameters and at least one target pose.

[0006] In one possible implementation of the first aspect, after obtaining the bone structure model of the target object and the first constraint information of the ultrasound tracer determined based on the bone structure model, the method further includes: The positional information of each bone screw in the bone structure model is obtained; the bone screws are used to construct the movement path of the ultrasound tracer. Based on the position information of all the bone screws, the adjustable range of the movement path in the movement direction is determined; the adjustable range includes a height adjustment range and an orientation adjustment range. Based on the adjustable range, at least one candidate configuration parameter of the ultrasonic tracer is determined; the candidate configuration parameter includes the installation height and spatial orientation of the ultrasonic tracer.

[0007] In one possible implementation of the first aspect, the first constraint information includes at least one of the following conditions: The ultrasonic tracer under the aforementioned candidate configuration parameters is within the tracking range of the optical tracking system; The ultrasonic probe in the candidate pose is within the tracking range of the optical tracking system; The angle between the normal of the ultrasonic tracer and the observation direction of the ultrasonic probe is less than a preset angle threshold.

[0008] In one possible implementation of the first aspect, determining, for any candidate configuration parameters of the ultrasound tracer, multiple candidate poses of the ultrasound probe under the candidate configuration parameters based on the bone structure model and the first constraint information includes: Based on the tomographic scan image of the target object, determine the skin region of the target object and the skin contact region between the target and the skin; The accessible range of the ultrasound probe is determined based on the skin area and the skin contact area. The pose set of the ultrasonic probe is determined based on the second constraint information of the ultrasonic probe and the accessible range; Multiple probe poses that satisfy the first constraint information in the pose set are determined as candidate poses.

[0009] In one possible implementation of the first aspect, the second constraint information includes at least one of the following: The normal of the ultrasonic probe is within a preset first angle range; The attitude angle of the ultrasonic probe is within a preset second angle range; The range of motion of the ultrasonic probe is within a preset mechanical motion range.

[0010] In one possible implementation of the first aspect, calculating the imaging coverage information of the target area by the ultrasonic tracer under the candidate configuration parameters based on the observation range of the target area of ​​each of the candidate poses includes: For any of the candidate poses, obtain the observation range of the target area corresponding to the ultrasonic probe in the candidate pose; The shooting coverage information corresponding to the candidate configuration parameters is determined based on the union of the observation ranges of each candidate pose.

[0011] In one possible implementation of the first aspect, determining the shooting coverage information corresponding to the candidate configuration parameters based on the union of the observation ranges of each of the candidate poses includes: Construct the adjustment loss function corresponding to the ultrasonic probe's pose adjustment; Based on the adjusted loss function and the union, the shooting coverage information of the candidate configuration parameters is determined; the shooting coverage information is represented as:

[0012] Where m is the shooting coverage information; C(m) is the union; R(m) is the adjustment loss function; and λ is the preset weight coefficient.

[0013] In one possible implementation of the first aspect, after determining the control strategy corresponding to the ultrasonic tracer from all the candidate configuration parameters and all the candidate poses based on the imaging coverage information corresponding to each of the candidate configuration parameters, the method further includes: The ultrasonic tracer is configured according to the target configuration parameters in the control strategy; For any target pose in the control strategy, the ultrasonic probe is controlled to acquire the observation data corresponding to the target under the target pose; Based on the observation data acquired under all target poses, the observation results of the target to be observed are generated.

[0014] Secondly, embodiments of this application provide a control device for an ultrasonic tracer, the ultrasonic tracer including an ultrasonic probe, the device comprising: The first constraint information acquisition unit is used to acquire the bone structure model of the target object and the first constraint information of the ultrasound tracer determined based on the bone structure model. The candidate pose determination unit is used to determine, for any candidate configuration parameters of the ultrasound tracer, multiple candidate poses of the ultrasound probe under the candidate configuration parameters, based on the bone structure model and the first constraint information; The coverage information acquisition unit is used to calculate the imaging coverage information of the target area by the ultrasound tracer under the candidate configuration parameters based on the observation range of the target area of ​​each candidate pose; the target area is the spatial region of the target to be observed in the bone structure model; A control strategy determination unit is used to determine the control strategy corresponding to the ultrasonic tracer from all the candidate configuration parameters and all the candidate poses based on the imaging coverage information corresponding to each of the candidate configuration parameters; the control strategy includes target configuration parameters and at least one target pose.

[0015] Thirdly, embodiments of this application provide an electronic device, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any of the first aspects above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the first aspects above.

[0017] Fifthly, embodiments of this application provide a computer program product that, when run on a robotic arm, causes a drone to perform the method described in any one of the first aspects above.

[0018] The beneficial effects of this application embodiment compared with the prior art are as follows: The ultrasound tracer can be preset with multiple different initial configuration states, and different initial configuration states can correspond to different candidate configuration parameters. The electronic device can acquire the bone structure model of the target object and the first constraint information determined based on the bone structure model. For one initial configuration state, based on the bone structure model and the first constraint information, multiple candidate poses of the ultrasound probe in the target object body under the configuration parameters can be determined, and then the observation range corresponding to each candidate pose can be acquired. The imaging coverage information of the target area of ​​the target user by the ultrasound tracer in the above-mentioned initial configuration state can be obtained. Thus, based on the imaging coverage information corresponding to all candidate configuration parameters, the target configuration parameters with the best imaging effect and at least one target pose can be determined, that is, the control strategy of the ultrasound tracer in the navigation process in the target object body can be determined. Compared with the control technology of existing tracers, the initial configuration of the ultrasound tracer and the pose used when imaging the target area in this application do not need to be configured based on human experience. Instead, the optimal combination of imaging effect on the target area can be determined by simulating multiple combinations of different candidate configuration parameters and candidate poses, that is, the above-mentioned control strategy can be obtained. This reduces the configuration difficulty and can also improve the imaging accuracy when the ultrasound tracer is actually controlled to acquire ultrasound images. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the various stages included in the control method of an ultrasonic tracer provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the implementation of a control method for an ultrasonic tracer provided in an embodiment of this application; Figure 3 This is a schematic diagram of the adjustment of an ultrasonic tracer provided in one embodiment of this application; Figure 4 This is a schematic diagram of the spatial relationship between an optical tracking system and an ultrasonic tracer provided in an embodiment of this application; Figure 5 This is a flowchart illustrating the specific implementation of the control method S202 for an ultrasonic tracer provided in the second embodiment of this application. Figure 6 This is a schematic diagram of the motion of an ultrasonic probe provided in one embodiment of this application; Figure 7This is a flowchart illustrating the specific implementation of the control method S203 for an ultrasonic tracer provided in the third embodiment of this application; Figure 8 This is a schematic diagram of the observation range provided in one embodiment of this application; Figure 9 This is a schematic diagram of the shooting coverage area provided in one embodiment of this application; Figure 10 This is a schematic diagram of the structure of a control device for an ultrasonic tracer provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0023] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] The control method for an ultrasound tracer provided in this application can be applied to scenarios involving the control of an ultrasound tracer. This control method can be applied to electronic devices, such as computers, laptops, and servers. The type of device used can be determined based on the actual application scenario and is not limited here. Specifically, the electronic device can be equipped with an application program for building three-dimensional structures. Within this application, biological models of the target object and corresponding medical device models can be constructed, thereby simulating the usage scenarios of various medical devices in actual applications. This provides a guiding control strategy for practical use scenarios, such as the control strategy for configuring an ultrasound tracer provided in this application embodiment.

[0025] In this embodiment, the control method of the ultrasound tracer described above can be applied to the observation of any object with a bone structure. That is, the target object can be an animal, such as a cat, dog, or tiger, or a human. The specific type of target object can be determined according to the actual observation needs and is not limited here.

[0026] For example, Figure 1 A flowchart illustrating the various stages of a control method for an ultrasonic tracer according to an embodiment of this application is shown. See also... Figure 1 As shown, the control of the aforementioned ultrasonic tracer can be divided into three stages: Stage 1, for acquiring basic data; Stage 2, for determining the control strategy; and Stage 3, for configuring the device based on the control strategy. The specific implementation logic of these three stages can be described as follows: Phase 1: The phase of collecting basic data In this embodiment, when determining the control strategy, it is necessary to use the bone structure information of the target object and construct the corresponding instrument model of the ultrasound tracer in the application of the three-dimensional model. In stage 1, the above two types of data can be obtained to provide a data basis for the subsequent determination of the control strategy.

[0027] The aforementioned bone structure information can be obtained from CT or MRI scans of the target object, or through other scanning methods. This bone structure information can be used to determine the distribution, length, and relative positional relationships of bones within the target object, enabling the construction of a biological structural model, such as a bone structure model, of the target object using the aforementioned application. Optionally, the biological structural model may also include a skin model, internal organ model, and blood model of the target object. Combining these multiple models yields a biological structural model of the target object.

[0028] In this embodiment, the electronic device can acquire the device information of the ultrasonic tracer, and then create a corresponding instrument model in the application for building a three-dimensional model based on the device information. This allows the device model of the ultrasonic tracer in the application to simulate the navigation and imaging process of the ultrasonic tracer within the target object.

[0029] In some possible implementations, the electronic device can store standard instrument models of different tracer models. The electronic device can determine the corresponding instrument model based on the model of the ultrasonic tracer used in this observation. Subsequently, when determining the control strategy, it can simulate the navigation process within the organism based on the instrument model corresponding to that model.

[0030] In some implementations, the electronic device can also download the instrument model of the ultrasound tracer from a cloud server, or obtain the instrument model through 3D scanning. The specific method of obtaining the instrument model of the ultrasound tracer can be selected according to the actual situation and is not limited here.

[0031] Phase 2: Determining the Control Strategy Please see Figure 2 , Figure 2 This illustration shows a schematic diagram of an implementation of a control method for an ultrasonic tracer according to an embodiment of this application, specifically describing how to determine the control strategy for controlling the ultrasonic tracer. Specifically, the method includes the following steps: In S201, a bone structure model of the target object and first constraint information of the ultrasound tracer determined based on the bone structure model are obtained.

[0032] In this embodiment, stage 1 can acquire the basic data required for determining the control strategy. This basic data may include the bone structure information used to construct the bone structure model and the device information corresponding to the ultrasound tracer. Specifically, the electronic device can create a 3D simulation scene for determining the control strategy in a 3D model application based on the bone structure information, and construct the bone structure model of the target object in the 3D simulation scene based on the bone structure information. Optionally, the basic data may also include other biological data of the target object, such as skin data, internal organ data, and blood data, etc., and subsequently, skin models, internal organ models, and blood models of the target object can be constructed in the 3D simulation scene.

[0033] In this embodiment, the electronic device can identify the target to be observed in the target object, such as the heart, liver, or diseased cell area of ​​an organism. Based on the location of the target to be observed in the target object, the area corresponding to the target to be observed is marked in the bone structure model and used as the target area. This target area is the area to be mainly photographed in this observation so as to obtain the ultrasound image corresponding to the target to be observed.

[0034] In this embodiment, after determining the target region of the target object within the bone structure model of the target object, first constraint information for observing the target region can be determined based on the location and size of the target region. Specifically, if the initial placement position of the ultrasound tracer and the pose of the ultrasound probe satisfy the first constraint information, an ultrasound image of the target region can be captured; conversely, if the initial placement position of the ultrasound tracer or the pose of the ultrasound probe does not satisfy the first constraint information, an ultrasound image of the target region may not be captured. Therefore, by constructing the first constraint information, the electronic device can improve the success rate of subsequent ultrasound image acquisition and the imaging effect of the ultrasound image on the target region.

[0035] For example, the first constraint information mentioned above may be: the distance between the target area and the target area is less than a preset distance threshold; the angle between the shooting direction of the ultrasonic probe and the target area is less than a preset angle threshold, etc.

[0036] In this embodiment, the first constraint information may include one or more constraint conditions. When the first constraint information includes two or more constraint conditions, the relationship between different constraint conditions can be set according to the actual situation. For example, the pose of the ultrasonic probe may require multiple constraint conditions to be satisfied simultaneously, or only one constraint condition needs to be satisfied among multiple constraint conditions, or only some of the multiple constraint conditions need to be satisfied. The specific relationship between each constraint condition can be set according to the actual situation.

[0037] For example, the first constraint information includes one or more constraints for the initial configuration of the ultrasonic tracer, and also includes one or more constraints for the ultrasonic probe. In this case, the electronic device can determine whether the ultrasonic tracer meets its corresponding constraint, and whether the ultrasonic probe meets its corresponding constraint, respectively.

[0038] In S202, for any candidate configuration parameters of the ultrasound tracer, based on the bone structure model and the first constraint information, multiple candidate poses of the ultrasound probe under the candidate configuration parameters are determined for the ultrasound tracer.

[0039] In this embodiment, the electronic device can create a device model of the ultrasound tracer in a three-dimensional simulation scene. Based on the positions and relative positions of the bone screws in the target object's bone structure model, the configuration space of the ultrasound tracer can be determined, which can be represented as M. This configuration space has a certain installation height and spatial orientation. Based on different combinations of installation height and spatial orientation, multiple different candidate configuration parameters can be obtained. For example, a candidate configuration parameter can be represented as m, meaning that any candidate configuration parameter belongs to the aforementioned configuration space. .

[0040] For example, Figure 3 A schematic diagram of the adjustment of an ultrasonic tracer according to an embodiment of this application is shown. See also Figure 3 As shown, the ultrasonic tracer includes two adjustment methods: one is adjustment in the installation height dimension, and the other is adjustment based on spatial orientation. For example, the ultrasonic tracer can slide on the track 31 to adjust its installation height; the ultrasonic tracer can also adjust its orientation in two rotational directions, namely direction a32 and direction b33. That is, different combinations of the above two adjustment results can correspond to different candidate configuration parameters of the ultrasonic tracer. The moving distance of the track 31 and the range of the two rotation angles are determined according to the size of the configuration space corresponding to the ultrasonic tracer.

[0041] In some possible implementations, the determination of the candidate configuration parameters of the ultrasound tracer can be achieved by: obtaining the position information of each bone screw in the bone structure model; the bone screws being used to construct the movement path of the ultrasound tracer; determining the adjustable range of the movement path in the movement direction based on the position information of all the bone screws; the adjustable range including a height adjustment range and an orientation adjustment range; and determining at least one candidate configuration parameter of the ultrasound tracer based on the adjustable range; the candidate configuration parameter including the installation height and spatial orientation of the ultrasound tracer.

[0042] In this embodiment, to enable the ultrasound tracer to move inside the target object, bone screws can be pre-installed in the bone structure of the target object. These bone screws specifically provide a movable path for the ultrasound tracer. The electronic device can mark various pre-set fixed positions in the bone structure model to obtain corresponding position information and determine the corresponding movement path of the ultrasound tracer based on the position information of each bone screw. After determining the movement path of the ultrasound tracer, the electronic device can determine the movement direction corresponding to that path. Then, based on the movement direction and information such as the size of the bone screws, it can determine the adjustable height and adjustable orientation of the ultrasound tracer. Based on all adjustable heights, the height adjustment range of the ultrasound tracer is obtained, and based on the adjustable orientation, the orientation adjustment range of the ultrasound tracer is determined. Based on these two parameters, the adjustable range of the ultrasound tracer can be determined, for example, denoted as M. Within the above adjustable range, any combination of height and orientation can be determined, resulting in a candidate configuration parameter, which can be denoted as m. .

[0043] In this embodiment, the electronic device can select any candidate configuration parameter from multiple candidate configuration parameters corresponding to the configuration space. Based on the candidate configuration parameter, it sets the instrument model corresponding to the ultrasound tracer in a three-dimensional simulation scene. Then, based on the installation position and spatial orientation of the ultrasound tracer, it can determine multiple selectable poses where each ultrasound probe on the ultrasound tracer can be placed. The electronic device can determine whether each selectable pose satisfies the first constraint information based on the bone structure model, and identify the selectable pose that satisfies the first constraint information as the candidate pose of the ultrasound tracer under that candidate configuration parameter.

[0044] In some possible implementations, the aforementioned first constraint information may include at least one of the following conditions: Condition 1: The ultrasonic tracer under the aforementioned candidate configuration parameters is within the tracking range of the optical tracking system; Condition 2: The ultrasonic probe in the candidate pose is within the tracking range of the optical tracking system; Condition 3: The angle between the normal of the ultrasonic tracer and the observation direction of the ultrasonic probe is less than a preset angle threshold.

[0045] In this embodiment, to ensure effective control of the ultrasound tracer in capturing ultrasound images, both the ultrasound tracer and the ultrasound probe must be within the field of view of the optical tracking system. The electronic device can determine the tracking range of the optical tracking system within the target object based on the visible range of the optical tracking system and the bone structure model; this tracking range is the field of view of the optical tracking system.

[0046] In this embodiment, the electronic device can determine the position of the ultrasound tracer in the bone structure model based on the candidate configuration parameters, and determine whether the position is within the tracking range of the optical tracking system. If the position of the device model corresponding to the ultrasound tracer is within the tracking range of the optical tracking system, the candidate configuration parameter is identified as valid, i.e., condition 1 is satisfied. Otherwise, if the installation position of the ultrasound tracer under the candidate configuration parameters is not within the tracking range of the optical tracking system, the candidate configuration parameter is identified as invalid, and the subsequent operations of determining the candidate pose and determining the imaging coverage information can be skipped.

[0047] In this embodiment, the electronic device can also determine the various selectable poses of the ultrasonic probe in one of the candidate configuration parameters when the ultrasonic tracer is in the installation position, and determine whether the ultrasonic probe is within the tracking range of the optical tracking system in any selectable pose. If the ultrasonic probe is within the tracking range in the selectable pose, the selectable pose is identified as satisfying condition 2; if the ultrasonic probe is outside the tracking range in the selectable pose, the selectable pose is identified as not satisfying condition 2 and can be identified as an invalid pose.

[0048] For example, Figure 4 A schematic diagram illustrating the spatial relationship between an optical tracking system and an ultrasonic tracer according to an embodiment of this application is shown. See also Figure 4 As shown, the optical tracking system can perform optical tracking when the ultrasonic tracer is inside the target object. The corresponding tracking range can be represented as region a41. The ultrasonic tracer can observe the target area corresponding to the object to be observed in the target object through the ultrasonic probe. The range corresponding to the target area can be represented as region b42, and the ultrasonic scanning range corresponding to the ultrasonic probe can be represented as region c43. The electronic device can determine whether the ultrasonic tracer is within the tracking range of the optical tracking system based on the installation position corresponding to a certain candidate configuration parameter and the posture of the ultrasonic probe, thereby determining whether it meets conditions 1 and 2 in the first constraint information, thus improving the tracking accuracy of the ultrasonic tracer.

[0049] In this embodiment, condition 3 is used to determine whether there is any obstruction to the observation path between the ultrasonic tracer and the ultrasonic probe, that is, to determine the orientation feasibility of the ultrasonic probe in the aforementioned selectable pose. Specifically, the electronic device can determine the angle between the normal corresponding to the tracer plane of the ultrasonic tracer and the observation direction (or observation line of sight) of the ultrasonic probe. For example, if the direction of the ultrasonic tracer is n and the observation direction of the ultrasonic probe is u, then condition 3 can be expressed as follows: The above θ is the angle threshold in condition 3.

[0050] In this embodiment, the electronic device can determine multiple optional poses corresponding to the ultrasonic probe, and determine whether each optional pose satisfies the first constraint information. The optional poses that satisfy the first constraint information are used as candidate poses of the ultrasonic tracer under the aforementioned candidate configuration parameters. For example, when the ultrasonic tracer is under the candidate configuration parameters, all its corresponding optional poses can be represented as T. Correspondingly, the set of each optional pose that satisfies the first constraint information is the set of candidate poses.

[0051] For a given ultrasonic tracer configuration m, define a set of candidate poses of the corresponding ultrasonic probe that satisfy the first constraint information. It can be represented as .

[0052] In this embodiment, based on the bone structure model, the installation position of the ultrasound tracer, and the tracking range of the optical tracking system, first constraint information for defining the ultrasound probe can be determined, thereby improving the accuracy of the candidate pose acquisition target area.

[0053] In S203, based on the observation range of the target area by each of the candidate poses, the imaging coverage information of the target area by the ultrasound tracer under the candidate configuration parameters is calculated; the target area is the spatial region of the target to be observed in the bone structure model.

[0054] In this embodiment, after determining multiple candidate poses of the ultrasonic probe, the electronic device can determine the observation range of the target area for each candidate pose. For example, if the ultrasonic probe has six candidate poses, the electronic device can determine the observation range corresponding to the ultrasonic probe in each of the six candidate poses through a three-dimensional simulation scene. The observation ranges corresponding to different candidate poses may or may not overlap, depending on the actual situation.

[0055] Optionally, the above method for determining the observation range corresponding to each candidate pose can be as follows: Based on the instrument model of the ultrasound probe in the three-dimensional simulation scene and the calibration relationship between the ultrasound probe and the image coordinate system, the plane or imaging volume of the ultrasound image under the candidate pose is mapped to the object coordinate system corresponding to the target object, thereby determining the spatial observation area corresponding to the pose. Furthermore, occlusion detection is performed using a bone structure model to obtain the actually observable portion of the target area under the candidate pose, i.e., the aforementioned observation range, which can be expressed as V(T). Wherein, V(T) can satisfy the following condition:

[0056] in, The target area is as described above; the unobstructed condition can be determined by ray detection, geometric intersection detection, or voxel occupancy query.

[0057] In some possible implementations, the calibration relationship between the ultrasound probe and the image coordinates can be determined as follows: using the position of the three-dimensional point p=(x, y, z) corresponding to the known coordinates of the ultrasound probe in the three-dimensional simulation scene, and the image pixel point I=(u, v) corresponding to the known coordinates in the ultrasound image, a corresponding transformation matrix C can be constructed. That is, the above-mentioned transformed coordinates C satisfies: (x, y, z, 1)=C(u, v, 0, 1). Thus, C=(sx, sy, 0, 1) *(rx, ry, 0, T) can be solved.

[0058] Where sx and sy represent scaling in the x and y directions (i.e., the actual size of each pixel). rx, ry, and T represent the three-dimensional rotation and translation relationship from the image matrix to the probe matrix.

[0059] In this embodiment, the electronic device can obtain the corresponding imaging coverage information when the ultrasonic tracer is installed at the position corresponding to the above-mentioned candidate configuration parameters, based on the set of observation ranges corresponding to all candidate poses. This imaging coverage information can be the size of the set of observation ranges or the coverage percentage of the target area, which can be determined according to the actual situation.

[0060] In S204, based on the imaging coverage information corresponding to each of the candidate configuration parameters, a control strategy corresponding to the ultrasonic tracer is determined from all the candidate configuration parameters and all the candidate poses; the control strategy includes target configuration parameters and at least one target pose.

[0061] In this embodiment, the ultrasonic tracer can correspond to multiple different installation positions, i.e., different candidate configuration parameters. For each candidate configuration parameter, the imaging coverage information corresponding to each candidate configuration parameter can be determined through the operations in S202 and S203 described above. Based on the imaging coverage information corresponding to each candidate configuration parameter, the candidate configuration parameter with the best imaging effect can be determined as the target configuration parameter, and the candidate pose corresponding to the target configuration parameter can be used as the target pose. Therefore, based on the target configuration parameter and at least one target pose, the control strategy corresponding to the ultrasonic tracer can be obtained. Subsequently, when observing the target, the navigation and imaging processes of the ultrasonic tracer can be controlled according to the control strategy.

[0062] As can be seen from the above, the control method of the ultrasound tracer provided in this application embodiment allows the ultrasound tracer to have multiple different initial configuration states preset. Different initial configuration states can correspond to different candidate configuration parameters. The electronic device can obtain the bone structure model of the target object and the first constraint information determined based on the bone structure model. For one of the initial configuration states, it can determine multiple candidate poses of the ultrasound probe in the target object body based on the configuration parameters according to the bone structure model and the first constraint information. Then, it can obtain the observation range corresponding to each candidate pose and obtain the imaging coverage information of the target area of ​​the target user by the ultrasound tracer in the above initial configuration state. Thus, based on the imaging coverage information corresponding to all candidate configuration parameters, it can determine the target configuration parameters with the best imaging effect and at least one target pose, that is, determine the control strategy of the ultrasound tracer in the navigation process in the target object body. Compared with existing tracer control technologies, the initial configuration of the ultrasonic tracer in this application and the pose used when imaging the target area do not need to be configured based on human experience. Instead, the optimal combination for imaging the target area can be determined by simulating various combinations of different candidate configuration parameters and candidate poses, thus obtaining the above-mentioned control strategy. This reduces the configuration difficulty and also improves the imaging accuracy when actually controlling the ultrasonic tracer to acquire ultrasonic images.

[0063] Figure 5 A flowchart illustrating the specific implementation of a control method S202 for an ultrasonic tracer according to a second embodiment of this application is shown. See also... Figure 5 As shown, relative to Figure 2 In the embodiment provided in this application, the control method for an ultrasonic tracer includes steps S2021 to S2024, which are described in detail below: Specifically, before determining the first deviation data corresponding to the base based on the first posture data and the coordinate transformation matrix between the robotic arm and the base on which the robotic arm is located, the method further includes: In S2021, based on the tomographic scan image of the target object, the skin region of the target object and the skin contact region between the target object and the skin are determined.

[0064] In this embodiment, before determining the control strategy of the ultrasound tracer, a tomographic image of the target object, i.e., a CT image, can be acquired. Based on the CT image, the skin model of the target object can be determined in a three-dimensional simulation scene. Based on the skin model and the bone structure model, the skin area of ​​the target object and the skin contact area between the target model and the skin model corresponding to the target to be observed can be determined.

[0065] In S2022, the accessible range of the ultrasound probe is determined based on the skin area and the skin contact area.

[0066] In this embodiment, the electronic device can determine the accessible range of the ultrasound probe as it moves around the target, based on the skin region and the skin contact area. Since the installation position of the ultrasound tracer corresponding to the ultrasound probe is determined according to candidate configuration parameters, the relative positional relationship between the ultrasound probe and the target is relatively fixed, and there is a certain range of motion. Furthermore, this range of motion is limited by the skin region. Therefore, by determining the skin region and the skin contact area corresponding to the target, the range of motion of the ultrasound probe can be further defined to obtain the accessible range corresponding to the ultrasound probe. The accessible orientation can be represented using point clouds, triangular networks, or parametric surfaces.

[0067] In S2023, the pose set of the ultrasonic probe is determined based on the second constraint information of the ultrasonic probe and the accessible range.

[0068] In this embodiment, the electronic device can determine the second constraint information of the ultrasonic probe. This second constraint information can be determined based on the mechanical structure of the ultrasonic probe, such as the range of motion corresponding to the mechanical structure connecting the ultrasonic probe and the ultrasonic oscilloscope. Additionally, to ensure that the ultrasonic probe can capture an ultrasonic image of the target, the probe's posture is also subject to certain limitations. Based on one or more of these factors, the second constraint information corresponding to the ultrasonic probe can be obtained, i.e., the subsequently determined pose set must satisfy the aforementioned second constraint information.

[0069] In some possible implementations, the second constraint information mentioned above includes at least one of the following conditions: Condition 1: The normal of the ultrasonic probe is within the preset first angle range.

[0070] Condition 2: The attitude angle of the ultrasonic probe is within the preset second angle range.

[0071] Condition 3: The range of motion of the ultrasonic probe is within the preset mechanical motion range.

[0072] In this embodiment, the electronic device can sample the contact point of the ultrasonic probe on the determined feasible contact area, and combine it with the second constraint information corresponding to the ultrasonic probe, such as the normal constraint of the ultrasonic probe (which needs to be within the first angle range), the attitude angle constraint (which needs to be within the second angle range), and the mechanical motion range constraint (that is, the motion range of the ultrasonic probe needs to be within the preset mechanical motion range), to obtain a set of all possible poses corresponding to the ultrasonic probe, namely the pose set mentioned above.

[0073] In S2024, multiple probe poses that satisfy the first constraint information in the pose set are determined as candidate poses.

[0074] In this embodiment, after determining the pose set corresponding to the ultrasonic tracer being in a certain installation position, the electronic device can determine whether each selectable pose in the pose set satisfies the first constraint information determined based on the bone structure model, thereby enabling it to determine each candidate pose in the pose set that satisfies both the second and first constraint information.

[0075] For example, Figure 6 A schematic diagram of the motion of an ultrasonic probe according to an embodiment of this application is shown. See also... Figure 6 As shown, an ultrasonic tracer can be used to acquire ultrasonic images of an object to be observed. The ultrasonic probe on the ultrasonic tracer can be... Figure 6 Component 61 in the ultrasound probe can be located in regions A62, B63, C64, and D65, where bone tissue can be located in region E66. The ultrasound probe can correspond to multiple different candidate poses. For example, in pose a67, the acquired ultrasound image covers region F68; in pose b69, the acquired ultrasound image covers region G610, and so on. Depending on the pose, ultrasound images of different regions of the object can be acquired, enabling multi-directional imaging of the object.

[0076] In this embodiment of the application, by obtaining the skin contact area of ​​the target object, multiple candidate postures that can be effectively captured by the ultrasound probe can be determined based on the skin contact area and the second constraint information corresponding to the ultrasound probe. This can improve the accuracy of the candidate posture determination and thus improve the imaging effect of the target in the ultrasound image.

[0077] Figure 7 A flowchart illustrating the specific implementation of a control method S203 for an ultrasonic tracer according to a third embodiment of this application is shown. See also... Figure 7 As shown, relative to Figure 2 or Figure 5 In the embodiment provided in this application, the control method for an ultrasonic tracer includes steps S2031 to S2032, which are described in detail below: In S2031, for any of the candidate poses, the observation range of the target area corresponding to the ultrasonic probe in the candidate pose is obtained.

[0078] In this embodiment, the electronic device can set the model pose of the simulated probe corresponding to the ultrasonic probe in the three-dimensional simulation scene according to the candidate pose, and obtain the observation range corresponding to the target area where the target to be observed is located based on the three-dimensional simulation scene according to the shooting range corresponding to the ultrasonic probe.

[0079] For example, Figure 8 A schematic diagram of the observation range provided in one embodiment of this application is shown. See also Figure 8 As shown, region aa81 represents muscle tissue in the target object, region bb82 represents bone tissue in the target object, and region cc83 represents the target area of ​​the object to be observed. When the ultrasound probe is in pose A84, its corresponding imaging range is region dd85. However, since the area where the target object is located is obscured by bone tissue, the obscured area is region ee86. Therefore, the ultrasound probe cannot effectively capture ultrasound images of the object under this pose. The ultrasound probe's imaging range can be adjusted to overlap with the target area.

[0080] In S2032, the shooting coverage information corresponding to the candidate configuration parameters is determined based on the union of the observation ranges of each candidate pose.

[0081] In this embodiment, since the ultrasonic probe on the ultrasonic tracer can capture ultrasonic images of the target under multiple poses, that is, the total observation range of the target is determined based on the ultrasonic images captured under all candidate poses, after the electronic device determines the observation range corresponding to each candidate pose, it can determine the shooting coverage information of the ultrasonic tracer under a certain candidate configuration parameter based on the union of all observation ranges.

[0082] For example, when the candidate configuration parameter of the ultrasound tracer is defined as m, its corresponding imaging coverage information can be expressed as: .

[0083] in, The target region can be the total volume of the object to be observed, or the total number of primes or discrete units occupied by the region in the three-dimensional simulation scene. This indicates the observation range observed by the ultrasonic probe in any candidate pose under the current ultrasonic tracer configuration. This observation range can be the range volume, or the number of voxels or discrete units occupied by the range in the three-dimensional simulation scene.

[0084] Optionally, the electronic device can identify the candidate pose as invalid based on the degree of overlap between the observation range and the target area. If the degree of overlap between the observation range and the target area is less than a preset range threshold, the subsequent generated control strategy can filter out all invalid poses.

[0085] For example, Figure 9 A schematic diagram illustrating the shooting coverage area provided in one embodiment of this application is shown. See also... Figure 9 As shown, region aA91 is the region corresponding to the observation range of the ultrasound probe scan, region bB92 is other regions not within the observation range, region 93 is the effective region occupied by the target to be observed in region cC91 corresponding to the observation range, that is, the overlapping region between the observation range and the target region, and region dD94 is the obscured region.

[0086] Furthermore, as another embodiment of this application, the above-described S2032 may specifically include the following two steps: In S2032.1, the adjustment loss function corresponding to the ultrasonic probe's pose adjustment is constructed.

[0087] In S2032.2, the shooting coverage information of the candidate configuration parameters is determined based on the adjusted loss function and the union; the shooting coverage information is represented as:

[0088] Where m is the shooting coverage information; C(m) is the union; R(m) is the adjustment loss function; and λ is the preset weight coefficient.

[0089] In this embodiment, since there may be some image loss when adjusting the ultrasonic probe, during simulated imaging, the constraint or penalty terms of the ultrasonic probe during pose adjustment can be used to construct the loss of imaging range due to pose adjustment, i.e., the aforementioned adjustment loss function. The electronic device can determine the imaging coverage information corresponding to a certain candidate configuration parameter based on the aforementioned adjustment loss function and the observation range corresponding to each candidate pose, thereby improving the accuracy of the imaging coverage information.

[0090] In this embodiment of the application, the electronic device can determine the observation range corresponding to each candidate pose and determine the union of each observation range, thereby determining the shooting coverage information corresponding to the candidate configuration parameters.

[0091] Phase 3: Device Configuration Based on Control Strategies In this embodiment, after the electronic device determines the control strategy, it can subsequently control the ultrasonic tracer to operate according to the control strategy. For example, the ultrasonic tracer can be configured according to the target configuration parameters in the control strategy, that is, its actual installation height and orientation during data acquisition can be determined. Then, according to the target pose in the control strategy, the ultrasonic probe in the ultrasonic tracer can be controlled to move to the corresponding target pose and acquire the observation data corresponding to the target to be observed, such as capturing the corresponding ultrasonic image. Then, based on the observation data under all target poses, the observation results of the target to be observed can be obtained.

[0092] In this embodiment, Figure 10 This diagram illustrates a structural block diagram of a control device for an ultrasonic tracer according to an embodiment of this application. The control device includes units for performing various operations. Figure 2 The corresponding embodiment describes the steps implemented by the control device of the ultrasonic tracer. Please refer to [link / reference] for details. Figure 2 and Figure 2 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown.

[0093] See Figure 10 A control device for an ultrasonic tracer, the ultrasonic tracer including an ultrasonic probe, the device comprising: The first constraint information acquisition unit 101 is used to acquire the bone structure model of the target object and the first constraint information of the ultrasound tracer determined based on the bone structure model. The candidate pose determination unit 102 is used to determine, for any candidate configuration parameters of the ultrasound tracer, multiple candidate poses of the ultrasound probe under the candidate configuration parameters, based on the bone structure model and the first constraint information; The coverage information acquisition unit 103 is used to calculate the imaging coverage information of the target area by the ultrasound tracer under the candidate configuration parameters based on the observation range of the target area of ​​each candidate pose; the target area is the spatial area of ​​the target to be observed in the bone structure model; The control strategy determination unit 104 is used to determine the control strategy corresponding to the ultrasonic tracer from all the candidate configuration parameters and all the candidate poses based on the imaging coverage information corresponding to each of the candidate configuration parameters; the control strategy includes target configuration parameters and at least one target pose.

[0094] It should be understood that, Figure 10 In the structural block diagram of the device shown, each module is used to perform... Figure 2 , Figure 5 as well as Figure 7 Each step in any corresponding embodiment, and for Figure 2, Figure 5 as well as Figure 7 The steps in the corresponding embodiments have been explained in detail in the above embodiments. Please refer to them for details. Figure 2 , Figure 5 , Figure 7 The relevant descriptions in the embodiments corresponding to the above figures will not be repeated here.

[0095] Figure 11 This is a structural block diagram of an electronic device provided in another embodiment of this application. For example... Figure 11 The electronic device 1100 of this embodiment includes a processor 1110, a memory 1120, and a computer program 1130 stored in the memory 1120 and executable on the processor 1110, such as a program for a control method of an ultrasonic tracer. When the processor 1110 executes the computer program 1130, it implements the steps of each embodiment of the control method for the ultrasonic tracer described above, for example... Figure 2 S201 to S204 are described above. Alternatively, the processor 1110 may implement the above when executing the computer program 1130. Figure 10 The functions of each module in the corresponding embodiments, for example, Figure 10 For details regarding the functions of units 101 to 104, please refer to [link / reference needed]. Figure 10 The relevant descriptions in the corresponding embodiments.

[0096] For example, computer program 1130 may be divided into one or more modules, one or more of which are stored in memory 1120 and executed by processor 1110 to complete this application. One or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 1130 in electronic device 1100. For example, computer program 1130 may be divided into various unit modules, each with the specific functions described above.

[0097] Electronic device 1100 may include, but is not limited to, processor 1110 and memory 1120. Those skilled in the art will understand that... Figure 11 This is merely an example of electronic device 1100 and does not constitute a limitation on electronic device 1100. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0098] The processor 1110 may be a central processing unit, or it may be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0099] The memory 1120 can be an internal storage unit of the electronic device 1100, such as a hard disk or memory of the electronic device 1100. The memory 1120 can also be an external storage device of the electronic device 1100, such as a plug-in hard disk, smart memory card, flash memory card, etc. equipped on the electronic device 1100. Furthermore, the memory 1120 can include both internal storage units and external storage devices of the electronic device 1100.

[0100] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A control method for an ultrasonic tracer, characterized in that, The ultrasonic tracer includes an ultrasonic probe, and the control method includes: A bone structure model of the target object and first constraint information of the ultrasound tracer determined based on the bone structure model are obtained; the first constraint information is used to capture an ultrasound image of the target area when the initial placement position of the ultrasound tracer and the pose of the ultrasound probe satisfy the first constraint information. The positional information of each bone screw in the bone structure model is obtained; the bone screws are used to construct the movement path of the ultrasound tracer. Based on the position information of all the bone screws, the adjustable range of the movement path in the movement direction is determined; the adjustable range includes a height adjustment range and an orientation adjustment range. Based on the adjustable range, at least one candidate configuration parameter of the ultrasonic tracer is determined; the candidate configuration parameter includes the installation height and spatial orientation of the ultrasonic tracer. For any candidate configuration parameters of the ultrasound tracer, based on the bone structure model and the first constraint information, determine multiple candidate poses of the ultrasound probe under the candidate configuration parameters for the ultrasound tracer. Based on the observation range of the target region by each of the candidate poses, the imaging coverage information of the target region by the ultrasound tracer under the candidate configuration parameters is calculated; the target region is the spatial region of the target to be observed in the bone structure model. Based on the imaging coverage information corresponding to each of the candidate configuration parameters, a control strategy corresponding to the ultrasonic tracer is determined from all the candidate configuration parameters and all the candidate poses; the control strategy includes target configuration parameters and at least one target pose.

2. The control method according to claim 1, characterized in that, The first constraint information includes at least one of the following conditions: The ultrasonic tracer under the aforementioned candidate configuration parameters is within the tracking range of the optical tracking system; The ultrasonic probe in the candidate pose is within the tracking range of the optical tracking system; The angle between the normal of the ultrasonic tracer and the observation direction of the ultrasonic probe is less than a preset angle threshold.

3. The control method according to any one of claims 1-2, characterized in that, For any candidate configuration parameters of the ultrasound tracer, based on the bone structure model and the first constraint information, the determination of multiple candidate poses of the ultrasound probe under the candidate configuration parameters includes: Based on the tomographic scan image of the target object, determine the skin region of the target object and the skin contact region between the target and the skin; The accessible range of the ultrasound probe is determined based on the skin area and the skin contact area. The pose set of the ultrasonic probe is determined based on the second constraint information of the ultrasonic probe and the accessible range; Multiple probe poses that satisfy the first constraint information in the pose set are determined as candidate poses.

4. The control method according to claim 3, characterized in that, The second constraint information includes at least one of the following: The normal of the ultrasonic probe is within a preset first angle range; The attitude angle of the ultrasonic probe is within a preset second angle range; The range of motion of the ultrasonic probe is within a preset mechanical motion range.

5. The control method according to claim 1, 2, or 4, characterized in that, The step of calculating the imaging coverage information of the target area by the ultrasonic tracer under the candidate configuration parameters based on the observation range of the target area of ​​each candidate pose includes: For any of the candidate poses, obtain the observation range of the target area corresponding to the ultrasonic probe in the candidate pose; The shooting coverage information corresponding to the candidate configuration parameters is determined based on the union of the observation ranges of each candidate pose.

6. The control method according to claim 2, characterized in that, The step of determining the shooting coverage information corresponding to the candidate configuration parameters based on the union of the observation ranges of each candidate pose includes: Construct the adjustment loss function corresponding to the ultrasonic probe's pose adjustment; Based on the adjusted loss function and the union, the shooting coverage information of the candidate configuration parameters is determined; the shooting coverage information is represented as: Where M is the shooting coverage information; C(m) is the union; R(m) is the adjustment loss function; λ is the preset weight coefficient; and m is the candidate configuration parameter.

7. The control method according to claim 1, 2, 4 or 6, characterized in that, After determining the control strategy corresponding to the ultrasonic tracer from all the candidate configuration parameters and all the candidate poses based on the imaging coverage information corresponding to each of the candidate configuration parameters, the method further includes: The ultrasonic tracer is configured according to the target configuration parameters in the control strategy; For any target pose in the control strategy, the ultrasonic probe is controlled to acquire the observation data corresponding to the target under the target pose; Based on the observation data acquired under all target poses, the observation results of the target to be observed are generated.

8. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images