Detection method and device, equipment and medium for the arm posture of operating an ultrasonic probe

Abstract

The present disclosure relates to a method and apparatus, device, and medium for detecting the arm posture of operating an ultrasonic probe. It relates to the field of ultrasonic technology. The method for detecting the arm posture of operating an ultrasonic probe includes: obtaining multiple arm posture images of operating the probe at the current moment and multiple moments before the current moment; extracting the sequence of arm joint point information and the sequence of motion speed information of the arm joint points from the multiple arm posture images; performing arm posture recognition based on the sequence of arm joint point information and the sequence of motion speed information of the arm joint points to determine the arm posture of operating the probe. The embodiments of the present disclosure can achieve the detection of the arm posture of operating an ultrasonic probe.

Description

Technical Field

[0001] The present disclosure relates to the field of ultrasonic technology, and in particular, to a method and device for detecting the arm posture of operating an ultrasonic probe, an electronic device, and a storage medium. Background Art

[0002] An ultrasonic system radiates ultrasonic signals generated by a transducer of a probe onto an object and receives information on echo signals reflected from the object, thereby obtaining an image of an internal part of the object. In particular, the ultrasonic system is used for the medical purpose of observing the inside of the human body and then judging injuries. However, currently, due to the limited level of the operator of the ultrasonic device, satisfactory ultrasonic images cannot be obtained. Therefore, it is necessary to detect the arm posture of operating the ultrasonic probe, and then standardize the operation of the operator of the ultrasonic probe. Summary of the Invention

[0003] The present disclosure provides a technical solution for a method and device, equipment, and medium for detecting the arm posture of operating an ultrasonic probe.

[0004] According to an aspect of the present disclosure, there is provided a method for detecting the arm posture of operating an ultrasonic probe, including:

[0005] Obtaining multiple arm posture images of operating the probe at the current moment and multiple moments before the current moment;

[0006] Extracting an arm joint point information sequence and an arm joint point movement speed information sequence of the multiple arm posture images;

[0007] Based on the arm joint point information sequence and the arm joint point movement speed information sequence, performing arm posture recognition to determine the arm posture of operating the probe.

[0008] Preferably, the method for extracting the arm joint point information sequence and the arm joint point movement speed information sequence of the multiple arm posture images includes:

[0009] Respectively determining a first arm joint point corresponding to the wrist and a second arm joint point where the lower arm and the upper arm of the arm are connected in the multiple arm posture images;

[0010] Respectively calculating multiple relative displacements of the first arm joint point and the second arm joint point at multiple moments before the current moment to obtain an arm joint point movement speed information sequence.

[0011] Preferably, the method for respectively determining the first arm joint point corresponding to the wrist and the second arm joint point where the lower arm and the upper arm of the arm are connected in the multiple arm posture images includes:

[0012] Perform arm segmentation on the multiple arm pose images to obtain corresponding multiple arm images;

[0013] Perform edge detection on the multiple arm images to obtain corresponding multiple arm edge images;

[0014] Based on the arm geometry, use the multiple arm edge images to determine the first arm joint point corresponding to the wrist in the multiple arm pose images and the second arm joint point where the lower arm and the upper arm of the arm are connected;

[0015] And / or,

[0016] The method of respectively calculating the multiple relative displacements of the first arm joint point and the second arm joint point at multiple moments before the current moment to obtain the motion speed information sequence of the arm joint points includes:

[0017] Respectively determine the multiple first coordinates and multiple second coordinates corresponding to the first arm joint point and the second arm joint point at the current moment and multiple moments before the current moment;

[0018] Respectively calculate the distances between the first coordinates corresponding to the current moment and the first coordinates corresponding to multiple moments before the current moment to obtain the motion speed information sequence of the first arm joint point;

[0019] Respectively calculate the distances between the second coordinates corresponding to the current moment and the second coordinates corresponding to multiple moments before the current moment to obtain the motion speed information sequence of the second arm joint point.

[0020] Preferably, the method of performing arm pose recognition based on the arm joint point information sequence and the motion speed information sequence of the arm joint points to determine the arm pose of the operating probe includes:

[0021] Respectively perform spatial feature extraction on the arm joint point information sequence and the motion speed information sequence of the arm joint points to obtain corresponding first spatial features and second spatial features;

[0022] And perform feature fusion on the first spatial feature and the second spatial feature to obtain an initial fusion feature;

[0023] Complete the arm pose detection of the operating probe based on the initial fusion feature.

[0024] According to one aspect of the present disclosure, there is provided a control method for operating an ultrasonic probe, including: the detection method as described above.

[0025] According to one aspect of the present disclosure, there is provided a detection device for the arm pose of an operating ultrasonic probe, including:

[0026] An acquisition unit for acquiring a plurality of arm posture images of an operating probe at the current moment and multiple moments before the current moment;

[0027] Extract the sequence of arm joint point information and the sequence of motion speed information of the arm joint points from the plurality of arm posture images;

[0028] Perform arm posture recognition based on the sequence of arm joint point information and the sequence of motion speed information of the arm joint points to determine the arm posture of the operating probe.

[0029] According to one aspect of the present disclosure, there is provided a control device for operating an ultrasonic probe, including: the detection device as described above.

[0030] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0031] A processor;

[0032] A memory for storing instructions executable by the processor;

[0033] Wherein, the processor is configured to: execute the method for detecting the arm posture of the operating ultrasonic probe as described above; and / or, the control method as described above.

[0034] According to one aspect of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method for the arm posture of the operating ultrasonic probe as described above is implemented; and / or, the control method as described above.

[0035] According to one aspect of the present disclosure, there is provided an ultrasonic instrument, including: applying the method for detecting the arm posture of the operating ultrasonic probe as described above, and / or, the control method as described above; and / or, including: the detection device as described above; and / or, the control device as described above; and / or, the electronic device as described above; and / or, the computer-readable storage medium as described above.

[0036] In the embodiments of the present disclosure, the operations of the personnel operating the ultrasonic probe can be standardized, and the problem that satisfactory ultrasonic images cannot be obtained due to the limited level of the personnel operating the ultrasonic can be solved.

[0037] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure.

[0038] According to the following detailed description of the exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear. Description of the Drawings

[0039] The accompanying drawings here are incorporated into the specification and form a part of this specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0040] Figure 1 A flowchart showing the detection of the arm posture for operating an ultrasonic probe according to an embodiment of the present disclosure;

[0041] Figure 2 A flowchart showing the detection of the arm posture for operating a probe according to an embodiment of the present disclosure;

[0042] Figure 3 A schematic diagram showing the network structure of a spatio-temporal adaptive graph convolution module according to an embodiment of the present disclosure;

[0043] Figure 4 A schematic diagram showing the adjacency matrix determined in a spatio-temporal adaptive graph convolution module according to an embodiment of the present disclosure;

[0044] Figure 5 A schematic diagram showing the structure of a control device for operating an ultrasonic probe according to an embodiment of the present disclosure;

[0045] Figure 6 A block diagram of an electronic device shown according to an exemplary embodiment;

[0046] Figure 7 A block diagram of an electronic device shown according to an exemplary embodiment. Detailed Description of the Invention

[0047] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0048] The special term "exemplary" here means "serving as an example, an embodiment, or illustrative". Any embodiment described as "exemplary" here does not have to be construed as superior to or better than other embodiments.

[0049] The term "and / or" in this document merely describes the associated relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" in this document means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set composed of A, B, and C.

[0050] In addition, to better illustrate the present disclosure, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present disclosure can still be implemented without certain specific details. In some instances, methods, means, elements, and circuits well-known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0051] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not elaborate further.

[0052] In addition, the present disclosure also provides a device, an electronic device, a computer-readable storage medium, and a program corresponding to the method for detecting the arm posture of operating an ultrasonic probe or the method for controlling the operation of an ultrasonic probe. The above can all be used to implement any one of the methods for detecting the arm posture of operating an ultrasonic probe or the method for controlling the operation of an ultrasonic probe provided by the present disclosure. For the corresponding technical solutions and descriptions, refer to the corresponding records in the method part and will not be elaborated further.

[0053] Figure 1 The flowchart showing the method for detecting the arm posture of operating an ultrasonic probe according to an embodiment of the present disclosure is shown in FIG. 1. The method for detecting the arm posture of operating an ultrasonic probe includes: Step S101: Obtain multiple arm posture images of operating the probe at the current moment and multiple moments before the current moment; Step S102: Extract the sequence of arm joint point information and the sequence of motion speed information of the arm joint points from the multiple arm posture images; Step S103: Perform arm posture recognition based on the sequence of arm joint point information and the sequence of motion speed information of the arm joint points to determine the arm posture of operating the probe.

[0054] Step S101: Obtain multiple arm posture images of operating the probe at the current moment and multiple moments before the current moment.

[0055] In the embodiments of the present disclosure and other possible embodiments, similarly, a camera or a camera can be used to photograph the patient's body position to obtain multiple arm posture images of operating the probe at the current moment and multiple moments before the current moment.

[0056] Step S102: Extract the sequence of arm joint point information and the sequence of motion speed information of the arm joint points from the multiple arm posture images.

[0057] In the embodiments of the present disclosure and other possible embodiments, the method for extracting the sequence of arm joint point information and the sequence of motion speed information of arm joint points of the multiple arm pose images includes: respectively determining a first arm joint point corresponding to the wrist and a second arm joint point where the lower arm and the upper arm of the arm are connected in the multiple arm pose images; respectively calculating multiple relative displacements of the first arm joint point and the second arm joint point at multiple moments before the current moment to obtain the sequence of motion speed information of the arm joint points.

[0058] In the embodiments of the present disclosure and other possible embodiments, the method for respectively determining a first arm joint point corresponding to the wrist and a second arm joint point where the lower arm and the upper arm of the arm are connected in the multiple arm pose images includes: performing arm segmentation on the multiple arm pose images to obtain corresponding multiple arm images; performing edge detection on the multiple arm images to obtain corresponding multiple arm edge images; based on the arm geometry, using the multiple arm edge images to determine a first arm joint point corresponding to the wrist and a second arm joint point where the lower arm and the upper arm of the arm are connected in the multiple arm pose images.

[0059] In the embodiments of the present disclosure and other possible embodiments, the method for performing arm segmentation on the multiple arm pose images to obtain corresponding multiple arm images includes: obtaining a trained preset arm segmentation model, and using the set arm segmentation model to perform edge detection on the multiple arm images to obtain corresponding multiple arm edge images. Similarly, the trained preset arm segmentation model can be a deep learning model, such as a U-net convolutional neural network model or its improved convolutional neural network model. The training of the arm segmentation model is a commonly used technical means for those skilled in the art and will not be described in detail herein.

[0060] In the embodiments of the present disclosure and other possible embodiments, the method for determining a first arm joint point corresponding to the wrist and a second arm joint point where the lower arm and the upper arm of the arm are connected in the multiple arm pose images based on the arm geometry and using the multiple arm edge images includes: respectively calculating multiple distances between a first arm edge line and a second arm edge line in each arm edge image; determining the 2 edge lines corresponding to the minimum distance among the distances as the first arm joint point; and determining the second arm joint point where the lower arm and the upper arm of the arm are connected according to the first arm joint point and the corresponding arm edge image.

[0061] In the embodiments of the present disclosure and other possible embodiments, the method for determining the second arm joint point where the lower arm and the upper arm of the arm are connected based on the first arm joint point and the corresponding arm edge image includes: determining the direction of the lower arm based on the first arm joint point, and determining the second arm joint point where the lower arm and the upper arm of the arm are connected based on the direction and the corresponding arm edge image. Specifically, the method for determining the direction of the lower arm based on the first arm joint point includes: extracting the arm center line of the arm edge image; respectively calculating a first length and a second length corresponding to the arm center line in two directions of the arm based on the first arm joint point; and determining the maximum length among the first length and the second length as the direction of the lower arm. Specifically, the method for determining the lower arm of the arm based on the direction includes: extracting the arm center line of the arm edge image, and determining whether there is a bend in the arm center line along the direction; if there is a bend, determining the bend as the second arm joint point.

[0062] In the embodiments of the present disclosure and other possible embodiments, the method for determining whether there is a bend in the arm center line along the direction includes: calculating the bend angle of the arm center line along the direction. The method for determining the bend as the second arm joint point if there is a bend includes: obtaining a set bend angle, and if the bend angle of the arm center line along the direction is greater than or equal to the set bend angle, determining the bend as the second arm joint point. Among them, those skilled in the art can configure the set bend angle according to actual needs.

[0063] In the embodiments of the present disclosure and other possible embodiments, the method for respectively calculating multiple relative displacements of the first arm joint point and the second arm joint point at multiple moments before the current moment to obtain a motion speed information sequence of the arm joint points includes: respectively determining multiple first coordinates and multiple second coordinates corresponding to the first arm joint point and the second arm joint point at the current moment and multiple moments before the current moment; respectively calculating the distances between the first coordinates corresponding to the current moment and the first coordinates corresponding to multiple moments before the current moment to obtain a motion speed information sequence of the first arm joint point; and respectively calculating the distances between the second coordinates corresponding to the current moment and the second coordinates corresponding to multiple moments before the current moment to obtain a motion speed information sequence of the second arm joint point. Among them, the distance can adopt the Euclidean distance. At the same time, those skilled in the art can select other distance calculation methods according to needs.

[0064] Step S103: Perform arm posture recognition based on the arm joint point information sequence and the motion speed information sequence of the arm joint points to determine the arm posture of the operation probe.

[0065] In the embodiments of the present disclosure and other possible embodiments, the method for identifying the arm posture of the operation probe based on the arm joint point information sequence and the motion speed information sequence of the arm joint points, and determining the arm posture of the operation probe includes: respectively performing spatial feature extraction on the arm joint point information sequence and the motion speed information sequence of the arm joint points to obtain corresponding first spatial features and second spatial features; and performing feature fusion on the first spatial features and the second spatial features to obtain initial fusion features; and completing the arm posture detection of the operation probe based on the initial fusion features. Wherein, the detected arm posture of the operation probe may be the spatial position where the arm of the operation probe is located.

[0066] In the embodiments of the present disclosure and other possible embodiments, the method for completing the arm posture detection of the operation probe based on the initial fusion features includes: using the obtained set sliding spatio-temporal window to perform a sliding spatial convolution operation on the initial fusion features to obtain a first spatio-temporal graph and a first adjacency matrix; and performing a spatial convolution operation on the initial fusion features based on the first adjacency matrix to obtain a first convolution graph; fusing the first spatio-temporal graph and the first convolution graph to obtain a posture feature; and completing the arm posture detection of the operation probe based on the posture feature.

[0067] In the embodiments of the present disclosure and other possible embodiments, the method for respectively performing spatial feature extraction on the arm joint point information sequence and the motion speed information sequence of the arm joint points to obtain corresponding first spatial features and second spatial features includes: respectively obtaining a first feature extraction model and a second feature extraction model; using the first feature extraction model to perform feature extraction on the arm joint point information sequence to obtain first spatial features; and using the second feature extraction model to perform feature extraction on the motion speed information sequence of the arm joint points to obtain second spatial features. Figure 2 The flowchart showing the arm posture detection of the operation probe according to the embodiments of the present disclosure. Among them, the first feature extraction model and the second feature extraction model may be neural networks based on deep learning. For example, Figure 2 the graph convolutional network in

[0068] For convenience of description, the arm joint point information sequence in the embodiments of the present disclosure is briefly described as the joint point information sequence, and at the same time, the motion speed information sequence of the arm joint points is briefly described as the motion speed information sequence.

[0069] In the disclosed embodiments, the method for respectively performing spatial feature extraction on the joint point information sequence and the motion speed information sequence to obtain corresponding first spatial features and second spatial features includes: performing a sliding spatial convolution operation on the joint point information sequence by using an obtained set sliding spatio-temporal window to obtain a second spatio-temporal graph and a second adjacency matrix; performing a spatial convolution operation on the joint point information sequence based on the second adjacency matrix to obtain a second convolution graph; fusing the second spatio-temporal graph and the second convolution graph to obtain first spatial features; and performing a sliding spatial convolution operation on the motion speed information sequence by using the obtained set sliding spatio-temporal window to obtain a third spatio-temporal graph and a third adjacency matrix; performing a spatial convolution operation on the joint point information sequence based on the third adjacency matrix to obtain a third convolution graph; fusing the third spatio-temporal graph and the third convolution graph to obtain second spatial features.

[0070] Figure 3 FIG. shows a schematic network structure diagram of a cross-temporal adaptive graph convolution module for detecting the arm posture of an operating probe according to an embodiment of the present disclosure. The sliding spatial convolution operation is performed based on the cross-temporal adaptive graph convolution module proposed in the present disclosure. As Figure 2 shown, the number n of times of performing a sliding spatial convolution operation on the joint point information sequence by using the set sliding spatio-temporal window in the cross-temporal adaptive graph convolution module in Figure 3 can be configured to 2 times, and the two cross-temporal adaptive graph convolution modules are in a series relationship.

[0071] In addition, in the embodiments of the present disclosure and other possible embodiments, first, a graph convolution operation is performed on the joint point information sequence by using a graph convolution network to obtain joint point graph convolution features; the joint point graph convolution features are input into a cross-temporal adaptive graph convolution module, and the set sliding spatio-temporal window configured in the cross-temporal adaptive graph convolution module performs a sliding spatial convolution operation on the joint point information sequence to obtain a second spatio-temporal graph and a second adjacency matrix; and a spatial convolution operation is performed on the joint point graph convolution features based on the second adjacency matrix to obtain a second convolution graph; then, the second spatio-temporal graph and the second convolution graph are fused to obtain first spatial features.

[0072] In addition, in the embodiments of the present disclosure and other possible embodiments, before performing a graph convolution operation on the joint point information sequence by using a graph convolution network to obtain joint point graph convolution features, the joint point information sequence is normalized, and a graph convolution operation is performed on the normalized joint point information sequence by using a graph convolution network to obtain joint point graph convolution features.

[0073] As Figure 3As shown, within each spatio-temporal adaptive graph convolution module, there is at least one adaptive graph convolution branch and one spatial graph convolution branch. The adaptive graph convolution branch performs a sliding spatial convolution operation on the joint point graph convolution feature or the joint point information sequence to obtain a second spatio-temporal graph and a second adjacency matrix; based on the spatial graph convolution branch, a spatial convolution operation is performed on the graph convolution feature or the joint point information sequence using the second adjacency matrix to obtain a second convolution graph.

[0074] In the embodiments of the present disclosure and other possible embodiments, it further includes: a channel expansion module or a channel expansion layer. Before performing the sliding spatial convolution operation on the joint point graph convolution feature or the joint point information sequence, the channel expansion module or the channel expansion layer is used to expand the number of channels of the joint point graph convolution feature or the joint point information sequence. Among them, the channel expansion module or the channel expansion layer can be Figure 3 a 1×1 convolution kernel (Conv 1×1) in. At the same time, in the embodiments of the present disclosure and other possible embodiments, the number of the adaptive graph convolution branches can be configured to be 2.

[0075] At the same time, in the embodiments of the present disclosure and other possible embodiments, the spatial graph convolution branch further includes: an adaptive spatial convolution layer and a graph lightweight temporal graph convolution module cascaded therewith. The second adjacency matrix is used to perform a spatial convolution operation on the graph convolution feature or the joint point information sequence successively through the adaptive spatial convolution layer and the graph lightweight temporal graph convolution module to obtain a second convolution graph.

[0076] In the embodiment of the present disclosure, the method of performing a spatial convolution operation on the joint point information sequence based on the second adjacency matrix to obtain a second convolution graph includes: respectively determining a plurality of corresponding first data association graphs based on the joint point information sequence and a plurality of obtained preset embedding functions; respectively fusing the plurality of first data association graphs with the set of corresponding second adjacency matrices to obtain a first association fusion feature; fusing the first association fusion feature with the joint point information sequence to obtain a first association joint point fusion feature; multiplying the first association joint point fusion feature by a first preset weight value to obtain a second convolution graph.

[0077] In the embodiments of the present disclosure and other possible embodiments, the adaptive spatial convolution layer is configured with K partitions, and each partition includes a plurality of preset embedding functions. In each partition, the joint point information sequence or the joint point graph convolution feature is respectively input into the plurality of preset embedding functions to obtain embedding features; the embedding features are input into a regression model to obtain a corresponding plurality of first data association graphs B k;Respectively fuse the multiple first data association graphs with the corresponding set of second adjacency matrices to obtain a first association fusion feature; fuse the first association fusion feature with the joint point information sequence or the joint point graph convolution feature to obtain a first associated joint point fusion feature; multiply the first associated joint point fusion feature by a first set weight value to obtain a second convolution graph for each partition, and use the second convolution graphs of all partitions as the final second convolution graph.

[0078] In the embodiments of the present disclosure and other possible embodiments, as Figure 3 shown, there are 3 partitions configured in the adaptive spatial convolution layer, and each partition includes 2 set embedding functions (the first set embedding function β k and the second set embedding function ψ k ). In each partition, the joint point information sequence or the joint point graph convolution feature is respectively input into the first set embedding function β k and the second set embedding function ψ k to obtain embedding features; the embedding features are input into a regression model to obtain a corresponding multiple first data association graphs B k ; respectively add (fuse) the multiple first data association graphs with the corresponding set of second adjacency matrices A k to obtain a first association fusion feature; multiply (fuse) the first association fusion feature with the joint point information sequence or the joint point graph convolution feature to obtain a first associated joint point fusion feature; multiply the first associated joint point fusion feature by a first set weight value to obtain a second convolution graph for each partition. Among them, the regression model can select a softmax logistic regression model.

[0079] In the embodiments of the present disclosure and other possible embodiments, where A k has the physical meaning of the physical connection structure of the human body, and B k is a data association graph; the data association graph B k is a parameter that can be obtained through sample learning and is used to determine whether there is a connection relationship and the connection strength between two vertices. By calculating the connection relationship between any two nodes (including non-adjacent nodes) in the skeleton graph, then obtaining the long-range dependence between the nodes, and finally calculating the connection relationship between the two nodes according to the normalized embedded Gaussian function, as shown in the formula:

[0080]

[0081] where f′ in is the connectivity between two nodes (vi, vj) of a given input feature sequence, N is the number of joint points, and any two nodes in the skeleton are respectively represented by v i and vj represents is a similarity function used to calculate the information v at the current attention position i and the v that has a potential connection in the global information j to determine whether there is similarity. Then, the values of the matrix are normalized to [0, 1] to serve as the virtual edge between two key points. Thus, the data association graph B k is calculated as shown in the formula:

[0082]

[0083] wherein, and are respectively the parameters of the embedding function β and and are initialized to 0 by default.

[0084] Therefore, the calculation process of the adaptive spatial graph convolution structure diagram can be represented by a formula.

[0085]

[0086] wherein, W k represents the weight parameter of the subset.

[0087] In the embodiments of the present disclosure, the method for performing a sliding spatial convolution operation on the joint point information sequence by using the obtained set sliding spatio-temporal window to obtain a second spatio-temporal graph and a second adjacency matrix includes: obtaining a first sliding window of a set size, controlling the first sliding window to slide on the joint point information sequence according to a set first step length to obtain first sliding window features; performing a spatial convolution operation based on the first sliding window features to obtain a second spatio-temporal graph; and obtaining a second adjacency matrix based on the connection relationship between the joint points calibrated in a certain frame in the first sliding window and the same calibrated joint points in other frames and a set neighborhood.

[0088] In Figure 3 , a schematic diagram of the model structure of the adaptive graph convolution based on a sliding window proposed by the present disclosure is also given in detail. As Figure 3 shown, the set size of the sliding spatio-temporal window is t×d, the set step length of the sliding spatio-temporal window is Stride = 2(d1, d2), inputting the joint point information sequence and the set adjacency matrix A into the model of the adaptive graph convolution, setting the sliding spatio-temporal window to perform a sliding spatial convolution operation on the joint point information sequence to obtain a second spatio-temporal graph and a second adjacency matrix; and performing a spatial convolution operation on the joint point information sequence based on the second adjacency matrix to obtain a second convolution graph; and fusing the second spatio-temporal graph and the second convolution graph to obtain first spatial features.

[0089] In addition, in the embodiments of the present disclosure and other possible embodiments, first, a graph convolution operation is performed on the joint point information sequence by using a graph convolutional network to obtain joint point graph convolution features; the joint point graph convolution features are input into a cross - space - time adaptive graph convolution module, and a set sliding space - time window configured in the cross - space - time adaptive graph convolution module performs a sliding space convolution operation on the joint point information sequence to obtain a second space - time graph and a second adjacency matrix; and a space convolution operation is performed on the joint point information sequence based on the second adjacency matrix to obtain a second convolution graph; the second space - time graph and the second convolution graph are fused to obtain a first space feature.

[0090] In the embodiments of the present disclosure and other possible embodiments, by means of a sliding space - time window, the spatial connection between nodes in the current frame is extended to the time domain, so that there is a connection relationship between the current node and its own node and its first - order adjacent nodes in other frames. Set a sliding space - time window with a size of , that is, each sliding space - time window has frames, then the space - time graph obtained each time the sliding window moves can be expressed as where represents the union of all vertex (calibrated joint points, for example, there are 13 calibrated joint points, and the corresponding labels are 1 - 13 in sequence) sets in the frames in the sliding space - time window, represents the union of all calibrated joint point connection edge sets in the sliding space - time window frames.

[0091] Figure 4 FIG. shows a schematic diagram of an adjacency matrix determined in a cross - space - time adaptive graph convolution module for detecting the arm posture of an operating probe according to an embodiment of the present disclosure. Define A′∈[0,1] n×n to represent the connection relationship of bone - calibrated joint points. When A′ = 1, it means that there is a connection relationship between two key points. Define as the block adjacency matrix tiled by A′ with a size of N×N for each frame in the sliding space - time window, and its size is where represents the connection relationship between the i - th calibrated joint point in a certain frame and the same calibrated joint point in other frames and its 1 - neighborhood (adjacent joint points to the i - th calibrated joint point).

[0092] Among them, a numerical value of 1 means there is a connection relationship, a numerical value of 0 means there is no connection relationship. Thus, it can be obtained that in the space - time graph , In the spatial dimension, for the nodes in the human body skeleton of a frame, except for the node itself, there is a connection relationship with its first-order adjacent nodes in the spatial dimension. Therefore, after passing through this sliding window, the feature vector with the dimension of C×N×T will be changed into a feature vector with the dimension of of the feature vector Among them, is at least configured to be 1.

[0093] In the embodiments of the present disclosure and other possible embodiments, the set of adjacency matrices Among them, is an N×N adjacency matrix Let represent whether the j-th joint v of the human body skeleton j is in the subset of the sampling area (the default sampling distance is 1) of the i-th joint v i to extract the connected vertices (calibrated joint points) in the specific subset from the input f in .

[0094] Among them, ξ ik is a normalized diagonal matrix, and its definition is as the formula: The constant coefficient α is configured to be 0.001 to avoid the situation of being 0.

[0095] In the embodiments of the present disclosure and other possible embodiments, the second adjacency matrix is used to perform spatial convolution operations on the graph convolution features or the joint point information sequence through an adaptive spatial convolution layer and a graph lightweight temporal graph convolution module in sequence to obtain a second convolution graph, which specifically includes: further performing depthwise separable convolution processing on the second convolution graph obtained by the adaptive spatial convolution layer to obtain a final second convolution graph. Among them, the method of further performing depthwise separable convolution processing on the second convolution graph obtained by the adaptive spatial convolution layer to obtain a final second convolution graph includes: performing expansion processing on the second convolution graph obtained by the adaptive spatial convolution layer to map it to high-dimensional space features; performing depthwise convolution on the high-dimensional space features to obtain depthwise convolution features; performing a fusion operation on the depthwise convolution features and the second convolution graph obtained by the adaptive spatial convolution layer to obtain a final second convolution graph. Among them, before performing depthwise convolution on the high-dimensional space features to obtain depthwise convolution features, normalization or regularization processing is performed on the high-dimensional space features. Among them, after performing depthwise convolution on the high-dimensional space features to obtain depthwise convolution features, it further includes: performing normalization or regularization processing on the depthwise convolution features; performing pointwise convolution operations on the depthwise convolution features after normalization or regularization processing to obtain a final depthwise convolution. Among them, before performing a fusion operation on the depthwise convolution features and the second convolution graph obtained by the adaptive spatial convolution layer to obtain a final second convolution graph, normalization or regularization processing is performed on the final depthwise convolution. The method of performing a fusion operation on the depthwise convolution features and the second convolution graph obtained by the adaptive spatial convolution layer to obtain a final second convolution graph includes: performing an addition operation on the depthwise convolution features and the second convolution graph obtained by the adaptive spatial convolution layer to obtain a final second convolution graph.

[0096] Specifically, a 1×1 expansion convolution is used to expand the number of channels of the input features before the data enters the depthwise convolution, mapping the low-dimensional space to the high-dimensional space (high-dimensional space features). Then, a 3×1 depthwise convolution is used to make the convolution kernel process only 1 input feature channel each time, thereby reducing the computational amount. Finally, a 1×1 pointwise convolution is used not only to establish the connection between channels but also to adjust the number of output channels to be the same as the number of input channels, mapping the high-dimensional features to the low-dimensional space by compressing the channels. At the same time, a residual connection is used to combine different convolution layers in parallel to prevent the vanishing gradient caused by the increase in network depth. At the same time, the stride of the first temporal graph convolution module can be configured to 2, aiming to compress the features and reduce the convolution cost.

[0097] In an embodiment of the present disclosure, the method for fusing the first spatial feature and the second spatial feature to obtain a fused feature includes: performing a splicing or addition operation on the first spatial feature and the second spatial feature to obtain a fused feature.

[0098] In an embodiment of the present disclosure, the method for fusing the second spatio-temporal map and the second convolutional map to obtain a first spatial feature includes: fusing the second spatio-temporal map and the second convolutional map to obtain a first joint-point fused feature map; fusing the first joint-point fused feature map with the joint-point information sequence to obtain a second joint-point fused feature map; calculating a first attention weight corresponding to the second joint-point fused feature map, and obtaining a first spatial feature according to the second joint-point fused feature map and its corresponding first attention weight.

[0099] As Figure 3 shown, the method for fusing the second spatio-temporal map and the second convolutional map to obtain a first joint-point fused feature map includes: performing an addition operation on the second spatio-temporal map and the second convolutional map to obtain a first joint-point fused feature map. Before fusing the first joint-point fused feature map with the joint-point information sequence to obtain a second joint-point fused feature map, a lightweight temporal graph convolutional module can be used to perform a convolutional operation first, and then the obtained feature is fused with the joint-point information sequence to obtain a second joint-point fused feature map. An attention network is used to calculate a first attention weight corresponding to the second joint-point fused feature map, and a first spatial feature is obtained according to the second joint-point fused feature map and its corresponding first attention weight. Specifically, the method for obtaining a first spatial feature according to the second joint-point fused feature map and its corresponding first attention weight includes: multiplying the second joint-point fused feature map by its corresponding first attention weight to obtain a first spatial feature.

[0100] In an embodiment of the present disclosure, the method for fusing the third spatio-temporal map and the third convolutional map to obtain a second spatial feature includes: fusing the third spatio-temporal map and the third convolutional map to obtain a first motion speed fused feature map; fusing the first motion speed fused feature map with the motion speed information sequence to obtain a third motion speed fused feature map; calculating a second attention weight corresponding to the second motion speed fused feature map, and obtaining a second spatial feature according to the second motion speed fused feature map and its corresponding second attention weight.

[0101] Similarly, for the embodiment corresponding to the method for fusing the third spatio-temporal map and the third convolutional map to obtain a second spatial feature, reference may be made to the method for fusing the second spatio-temporal map and the second convolutional map to obtain a first spatial feature described above.

[0102] In an embodiment of the present disclosure, the method for performing a sliding spatial convolution operation on the motion speed information sequence by using the obtained set sliding spatio-temporal window to obtain a third spatio-temporal graph and a third adjacency matrix includes: obtaining a second sliding window of a set size, and controlling the second sliding window to slide on the motion speed information sequence according to a set second stride to obtain second sliding window features; performing a spatial convolution operation based on the second sliding window features to obtain a third spatio-temporal graph; and obtaining a third adjacency matrix based on the connection relationship between the joint points calibrated in a certain frame in the second sliding window and the same calibrated joint points in other frames and a set neighborhood.

[0103] In an embodiment of the present disclosure, the method for performing a spatial convolution operation on the joint point information sequence based on the third adjacency matrix to obtain a third convolution graph includes: respectively determining a plurality of corresponding second data association graphs based on the motion speed information and a plurality of obtained set embedding functions; respectively fusing the plurality of second data association graphs with a set of corresponding third adjacency matrices to obtain second association fusion features; fusing the second association fusion features with the motion speed information sequence to obtain second association motion speed fusion features; and multiplying the second association motion speed fusion features by a second set weight value to obtain a third convolution graph.

[0104] Similarly, as Figure 3 shown, the set size of the sliding spatio-temporal window is t×d, the set stride of the sliding spatio-temporal window Stride = 2, inputting the motion speed information and the set adjacency matrix A into the model of the adaptive graph convolution, setting the sliding spatio-temporal window to perform a sliding spatial convolution operation on the motion speed information to obtain a second spatio-temporal graph and a second adjacency matrix; fusing the second spatio-temporal graph and the second convolution graph to obtain a first spatial feature; and performing a spatial convolution operation on the joint point information sequence based on the third adjacency matrix to obtain a third convolution graph; fusing the third spatio-temporal graph and the third convolution graph to obtain a second spatial feature. For details, reference may be made to the specific implementation manner of performing a sliding spatial convolution operation on the joint point information sequence by using the obtained set sliding spatio-temporal window to obtain a second spatio-temporal graph and a second adjacency matrix; and performing a spatial convolution operation on the joint point information sequence based on the second adjacency matrix to obtain a second convolution graph; and fusing the second spatio-temporal graph and the second convolution graph to obtain a first spatial feature.

[0105] In the embodiments of the present disclosure and other possible embodiments, a sliding spatial convolution operation is performed on the initial fusion feature by using the obtained set sliding spatio-temporal window to obtain a first spatio-temporal graph and a first adjacency matrix; and a spatial convolution operation is performed on the initial fusion feature based on the first adjacency matrix to obtain a first convolution graph; the first spatio-temporal graph and the first convolution graph are fused to obtain a pose feature; and an arm pose detection of the operating probe is completed based on the pose feature.

[0106] In the embodiment of the present disclosure, the method of performing a sliding spatial convolution operation on the fusion feature by using the obtained set sliding spatio-temporal window to obtain a first spatio-temporal graph and a first adjacency matrix includes: obtaining a third sliding window of a set size, controlling the third sliding window to slide on the fusion feature according to a set third step length to obtain a third sliding window feature; performing a spatial convolution operation on the third sliding window feature to obtain a first spatio-temporal graph; and obtaining a first adjacency matrix based on the connection relationship between the joint points calibrated in a certain frame in the third sliding window and the same calibrated joint points in other frames and a set neighborhood.

[0107] Similarly, for the embodiment of performing a sliding spatial convolution operation on the fusion feature by using the obtained set sliding spatio-temporal window to obtain a first spatio-temporal graph and a first adjacency matrix, reference may be made to the above method of performing a sliding spatial convolution operation on the joint point information sequence by using the obtained set sliding spatio-temporal window to obtain a second spatio-temporal graph and a second adjacency matrix.

[0108] In the embodiment of the present disclosure, the method of performing a spatial convolution operation on the fusion feature based on the first adjacency matrix to obtain a first convolution graph includes: respectively determining a corresponding plurality of third data association graphs based on the fusion feature and a plurality of obtained set embedding functions; respectively fusing the plurality of third data association graphs with a set of corresponding third adjacency matrices to obtain a second associated fusion feature; fusing the third associated fusion feature with the fusion feature to obtain a third associated motion speed fusion feature; and multiplying the third associated motion speed fusion feature by a third set weight value to obtain a first convolution graph.

[0109] Similarly, for the method of performing a spatial convolution operation on the fusion feature based on the first adjacency matrix to obtain a first convolution graph, reference may be made to the above method of performing a spatial convolution operation on the joint point information sequence based on the second adjacency matrix to obtain a second convolution graph.

[0110] In an embodiment of the present disclosure, the method for fusing the first spatio-temporal map and the first convolutional map to obtain pose features includes: fusing the first spatio-temporal map and the first convolutional map to obtain a first fused feature map; fusing the first fused feature map with the initial fused feature again to obtain a second fused feature map; calculating a first attention weight corresponding to the second fused feature map, and obtaining pose features according to the second fused feature map and its corresponding third attention weight.

[0111] Similarly, for the embodiment corresponding to the method of fusing the first spatio-temporal map and the first convolutional map to obtain pose features, reference may be made to the method of fusing the second spatio-temporal map and the second convolutional map to obtain first spatial features described above.

[0112] In an embodiment of the present disclosure and other possible embodiments, the method for fusing the first spatio-temporal map and the first convolutional map to obtain pose features includes: performing a splicing or addition operation on the first spatio-temporal map and the first convolutional map to obtain pose features.

[0113] As Figure 3 shown, the method for fusing the first spatio-temporal map and the first convolutional map to obtain pose features includes: performing an addition operation on the first spatio-temporal map and the first convolutional map to obtain a fused feature map. For a more specific implementation manner, reference may be made to the embodiment corresponding to the method of fusing the second spatio-temporal map and the second convolutional map to obtain a first joint point fused feature map.

[0114] In an embodiment of the present disclosure and other possible embodiments, as Figure 2 shown, the method for completing behavior detection based on the pose features includes: inputting the pose features into a set fully connected layer to complete behavior detection. Before inputting the pose features into the set fully connected layer, a pooling operation may also be performed on the pose features, and the pose features after the pooling operation are input into the set fully connected layer to complete behavior detection. Among them, the method of the pooling operation may be maximum pooling, average pooling or other existing pooling operations.

[0115] Meanwhile, the present disclosure also proposes a control method for operating an ultrasonic probe, including: the detection method as described above.

[0116] In an embodiment of the present disclosure and other possible embodiments, specifically, the control method for operating the ultrasonic probe includes: obtaining the patient's body position and the arm pose of operating the probe in real time; regulating the movement direction of the arm according to the patient's body position and the arm pose; during the process of regulating the movement direction of the arm, obtaining the contact pressure of the probe in real time, and regulating the distance between the probe and the patient according to the contact pressure and a set pressure.

[0117] Obtain the patient's body position and the arm posture of the operating probe in real time.

[0118] In the embodiment of the present disclosure, before obtaining the patient's body position in real time, the method for determining the method of obtaining the patient's body position includes: obtaining the patient's body position image at the current moment; performing body position segmentation on the patient's body position image to obtain a body position segmentation image; extracting the whole body joint point information of the body position segmentation image; and determining the obtained patient's body position based on the whole body joint point information.

[0119] In the embodiment of the present disclosure and other possible embodiments, a camera or a camera can be used to photograph the patient's body position to obtain the patient's body position image at the current moment.

[0120] In the embodiment of the present disclosure and other possible embodiments, the method for performing body position segmentation on the patient's body position image to obtain a body position segmentation image includes: obtaining a trained preset body position segmentation model, and using the set body position segmentation model to perform body position segmentation on the patient's body position image to obtain a body position segmentation image. The trained preset body position segmentation model can be a deep learning model, such as a U-net convolutional neural network model or an improved convolutional neural network model thereof. The training of the body position segmentation model is a conventional technical means for those skilled in the art, and will not be described in detail herein.

[0121] In the embodiment of the present disclosure and other possible embodiments, the method for extracting the whole body joint point information of the body position segmentation image includes: extracting the human skeleton of the body position segmentation image; and calibrating the joints of the human skeleton to obtain the whole body joint point information.

[0122] In the embodiment of the present disclosure and other possible embodiments, the method for extracting the human skeleton of the body position segmentation image includes: performing edge detection on the body position segmentation image to obtain a body position edge image; and extracting the center line of the body position edge line in the body position edge image to obtain the human skeleton. Among them, the algorithm or operator for edge detection is a conventional technical means in the art and will not be described in detail herein.

[0123] In the embodiment of the present disclosure and other possible embodiments, the method for calibrating the joints of the human skeleton to obtain the whole body joint point information includes: calibrating the end position of the human skeleton as the first whole body joint point; and respectively calibrating the second whole body joint points other than the first whole body joint point based on the first whole body joint point and the human skeleton. Among them, the whole body joint points include the above-mentioned first whole body joint point and the second whole body joint points.

[0124] In the embodiments of the present disclosure and other possible embodiments, the method of respectively calibrating the second whole-body joint points other than the first whole-body joint points based on the first whole-body joint points and the human body skeleton includes: respectively calculating the angular values of adjacent skeletons in the human body skeleton; and calibrating the second whole-body joint points other than the first whole-body joint points based on the angular values and a set angular value. Wherein, the set angular value can be configured as any value between 5° and 20°, or other values, and those skilled in the art can configure the set angular value according to needs.

[0125] In the embodiments of the present disclosure and other possible embodiments, the method of respectively calculating the angular values of adjacent skeletons in the human body skeleton includes: respectively determining a first length and a second length corresponding to the adjacent skeletons; and obtaining the angular value of the adjacent skeletons in the human body skeleton based on the cosine calculation formula by using the first length and the second length.

[0126] In the embodiments of the present disclosure and other possible embodiments, the method of calibrating the second whole-body joint points other than the first whole-body joint points based on the angular values and the set angular value includes: if the angular value is greater than or equal to the set angular value, calibrating the connection point of the adjacent skeleton corresponding to the angular value as the second whole-body joint point; otherwise, no calibration is performed.

[0127] In the embodiments of the present disclosure and other possible embodiments, the method of determining the method of obtaining the patient's body position based on the whole-body joint point information includes: obtaining the preset whole-body joint point information corresponding to the human body skeleton in the preset body position; calculating the similarity between the preset whole-body joint point information and the whole-body joint point information, and determining the method of obtaining the patient's body position based on the similarity and a preset similarity.

[0128] In the embodiments of the present disclosure and other possible embodiments, the method of determining the method of obtaining the patient's body position based on the similarity and the preset similarity includes: if the similarity is greater than or equal to the preset similarity, configuring the method of obtaining the patient's body position as the preset body position. Wherein, those skilled in the art can configure the preset similarity according to needs.

[0129] In the embodiments of the present disclosure, before the arm posture of the operation probe is obtained in real time, the method of determining the arm posture of the operation probe includes: obtaining multiple arm posture images of the operation probe at the current moment and multiple moments before the current moment; extracting the sequence of arm joint point information and the sequence of movement speed information of the arm joints from the multiple arm posture images; and performing arm posture recognition based on the sequence of arm joint point information and the sequence of movement speed information of the arm joints to determine the arm posture of the operation probe.

[0130] Adjust the movement orientation of the arm according to the patient's position and the arm posture.

[0131] In an embodiment of the present disclosure, the method of adjusting the movement orientation of the arm according to the patient's position and the arm posture includes: obtaining in real time a plurality of set position information of the area to be operated in the patient's position, and determining the position information of the probe sound head in the arm posture; determining the movement trajectory of the arm based on the plurality of set position information and the position information of the probe sound head; and adjusting the movement orientation of the arm based on the movement trajectory.

[0132] In an embodiment of the present disclosure, before obtaining in real time a plurality of set position information of the area to be operated in the patient's position, the method of determining the plurality of set position information includes: obtaining a light focusing instruction, irradiating the area to be operated according to the light focusing instruction, and determining a plurality of set position information of the area to be operated in the patient's position.

[0133] In an embodiment of the present disclosure and other possible embodiments, for a plurality of set position information of the area to be operated in the patient's position, by different light focusing instructions, the irradiation position of the area to be operated can be adjusted to obtain corresponding irradiation points, and the plurality of set position information of the area to be operated in the patient's position can be determined by collecting the positions of the irradiation points.

[0134] In an embodiment of the present disclosure, the method of determining the position information of the probe sound head in the arm posture includes: obtaining an arm posture image at the current moment, performing probe segmentation on the arm posture image to obtain a probe segmentation image; and determining the position information of the probe sound head in the arm posture according to the probe segmentation image.

[0135] In an embodiment of the present disclosure and other possible embodiments, the method of performing probe segmentation on the arm posture image to obtain a probe segmentation image includes: obtaining a trained preset probe segmentation model; and performing probe segmentation on the arm posture image based on the preset probe segmentation model to obtain a probe segmentation image.

[0136] In an embodiment of the present disclosure and other possible embodiments, the trained preset probe segmentation model can be a deep learning model, such as a U-net convolutional neural network model or an improved convolutional neural network model thereof. The training of the body position segmentation model is a commonly used technical means for those skilled in the art, and will not be described in detail here.

[0137] In an embodiment of the present disclosure and other possible embodiments, the method of determining the position information of the probe sound head in the arm posture according to the probe segmentation image includes: determining the probe sound head according to the probe segmentation image.

[0138] Determine the geometric center point of the probe head based on the probe head, and configure the position of the geometric center point as the position information of the probe head in the arm posture.

[0139] In the embodiments of the present disclosure and other possible embodiments, the method for determining the probe head according to the probe segmentation image includes: performing edge detection on the probe segmentation image to obtain a probe edge image; determining the shape of the probe edge image, and determining the probe head based on the shape and a preset shape.

[0140] During the process of adjusting the movement orientation of the arm, the contact pressure of the probe is obtained in real time, and the distance between the probe and the patient is adjusted according to the contact pressure and a set pressure.

[0141] In the embodiments of the present disclosure, the method for adjusting the distance between the probe and the patient according to the contact pressure and the set pressure includes: obtaining the set pressure; calculating the difference between the contact pressure and the difference regulated by the set pressure; and adjusting the distance between the probe and the patient based on the difference. Wherein, those skilled in the art can configure the set pressure according to actual needs.

[0142] In the embodiments of the present disclosure and other possible embodiments, the method for adjusting the distance between the probe and the patient based on the difference includes: judging the sign of the difference; if the sign is positive, adjusting the distance between the probe and the patient in a first direction; otherwise, adjusting the distance between the probe and the patient in a second direction opposite to the first direction.

[0143] Furthermore, both the first direction and the second direction are the directions corresponding to the movement orientation of the arm. For example, if the movement orientation of the arm is 45° with respect to the horizontal plane, then the first direction or the second direction is the direction corresponding to the arm at 45° with respect to the horizontal plane.

[0144] In the embodiments of the present disclosure, the method for obtaining the contact pressure of the probe in real time during the process of adjusting the movement orientation of the arm and adjusting the distance between the probe and the patient according to the contact pressure and the set pressure includes: obtaining the contact pressure of the probe in real time during the process of adjusting the movement orientation of the arm, and obtaining multiple set pressures corresponding to multiple set position information of the area to be operated in the patient's body position; respectively calculating multiple differences between the contact pressure and the multiple set pressures; and respectively adjusting the distance between the probe and the patient based on the multiple differences.

[0145] In the embodiments of the present disclosure and other possible embodiments, the method of adjusting the distance between the probe and the patient based on the difference includes: respectively determining the signs of the plurality of differences; if the sign is positive, adjusting the distance between the probe and the patient in a first direction; otherwise, adjusting the distance between the probe and the patient in a second direction opposite to the first direction. For example, both the first direction and the second direction are directions corresponding to the movement orientation of the arm. For example, if the movement orientation of the arm is at 45° to the horizontal plane, then the first direction or the second direction is the direction corresponding to the arm at 45° to the horizontal plane.

[0146] The execution subject of the method for detecting the arm posture of operating an ultrasonic probe or the control method of operating an ultrasonic probe can be a detection device for the arm posture of operating an ultrasonic probe or a control device for operating an ultrasonic probe. For example, the method for detecting the arm posture of operating an ultrasonic probe or the control method of operating an ultrasonic probe can be executed by a terminal device, a server, or other processing devices. Among them, the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the method for detecting the arm posture of operating an ultrasonic probe or the control method of operating an ultrasonic probe can be implemented by a processor invoking computer-readable instructions stored in a memory.

[0147] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic.

[0148] The embodiments of the present disclosure also propose a detection device for the arm posture of operating an ultrasonic probe. The detection device includes: an acquisition unit, configured to acquire multiple arm posture images of operating the probe at the current moment and multiple moments before the current moment; extract an arm joint point information sequence and an arm joint point movement speed information sequence of the multiple arm posture images; and perform arm posture recognition based on the arm joint point information sequence and the arm joint point movement speed information sequence to determine the arm posture of operating the probe.

[0149] In addition, the embodiments of the present disclosure also propose a control device for operating an ultrasonic probe. The control device includes: the above detection device.

[0150] In the embodiments of the present disclosure and other possible embodiments, the control device further includes: an acquisition unit configured to acquire in real time the patient's body position and the arm posture of the operating probe; an orientation adjustment unit configured to adjust the movement orientation of the arm according to the patient's body position and the arm posture; and a distance adjustment unit configured to acquire in real time the contact pressure of the probe during the process of adjusting the movement orientation of the arm, and adjust the distance between the probe and the patient according to the contact pressure and a set pressure.

[0151] Meanwhile, the present disclosure also provides a block diagram of another control device for operating an ultrasonic probe. The control device for operating an ultrasonic probe includes: an orientation adjustment unit, a telescopic unit, a fixing unit, and a control unit. The orientation adjustment unit, the telescopic unit, and the fixing unit are respectively connected to the control unit. The control unit is configured to receive the patient's body position and the arm posture of the operating probe acquired in real time, form a first control instruction according to the patient's body position and the arm posture of the operating probe, and form a second control instruction according to the contact pressure of the probe acquired in real time and the set pressure during the process of adjusting the movement orientation of the arm. The fixing unit and the orientation adjustment unit are respectively configured to fix the arm and adjust the movement orientation of the arm according to the first control instruction. The telescopic unit is configured to adjust the distance between the probe and the patient according to the second control instruction. Among them, the control unit may select a conventional controller, such as a single-chip microcomputer, a programmable logic controller, a computer, or the like.

[0152] In the embodiments of the present disclosure and other possible embodiments, the control unit is configured to receive the patient's body position and the arm posture of the operating probe acquired in real time, and form a first control instruction according to the patient's body position and the arm posture of the operating probe, including: acquiring in real time a plurality of set position information of the area to be operated in the patient's body position, and determining the position information of the probe tip in the arm posture; based on the plurality of set position information and the position information of the probe tip, determining the movement trajectory of the arm (forming a first control instruction); and adjusting the movement orientation of the arm based on the movement trajectory (first control instruction).

[0153] In embodiments of the present disclosure and other possible embodiments, the control unit is configured to receive the real-time patient position and the arm posture for operating the probe, and form a second control instruction according to the contact pressure of the probe and the set pressure obtained in real time during the process of adjusting the movement orientation of the arm, including: obtaining the set pressure; calculating the difference between the contact pressure and the set pressure adjustment (to form the second control instruction); adjusting the distance between the probe and the patient based on the difference (the second control instruction); or, obtaining the contact pressure of the probe in real time during the process of adjusting the movement orientation of the arm, and obtaining a plurality of set pressures corresponding to a plurality of set position information of the area to be operated in the patient position; respectively calculating a plurality of differences between the contact pressure and the plurality of set pressures (to form the second control instruction); respectively adjusting the distance between the probe and the patient based on the plurality of differences (the second control instruction).

[0154] Figure 5 FIG. shows a schematic structural diagram of a control device for operating an ultrasonic probe according to an embodiment of the present disclosure. As Figure 5 shown, the orientation adjustment unit includes: a rotation mechanism 1 and a height adjustment mechanism. One side of the height adjustment mechanism is connected to the rotation mechanism 1, and the other side of the height adjustment mechanism is connected to the outside of the fixing unit 4; the fixing unit 4 includes: an arm accommodating mechanism, and an arm fixing mechanism 5 is provided in the arm accommodating mechanism.

[0155] In embodiments of the present disclosure and other possible embodiments, as Figure 5 shown, one side of the height adjustment mechanism is connected to a first sliding track provided on the rotation mechanism 1. The height adjustment mechanism can slide on the first sliding track, thereby driving the fixing unit 4 to move, so as to adjust the distance between the probe and the patient according to the second control instruction. Wherein, the sliding track can be a multi-directional first sliding track. Furthermore, the arm can be fixed and the movement orientation of the arm can be adjusted based on the first control instruction through the multi-directional first sliding track and the height adjustment mechanism.

[0156] In embodiments of the present disclosure and other possible embodiments, a second sliding track is provided in the rotation mechanism 1, and the height adjustment mechanism can rotate in the height adjustment mechanism. Specifically, the second sliding track includes: a chute, and a ball 1-1 is provided in the chute; the rotation mechanism 1 drives the height adjustment mechanism to rotate by sliding the ball 1-1 in the chute.

[0157] In embodiments of the present disclosure and other possible embodiments, a height limiting mechanism is provided on the height adjusting mechanism. The height limiting mechanism is used to limit the lowest height of the height adjusting mechanism to prevent damage to the arm due to too low a height. Among them, the height limiting mechanism includes: a first main body and a second main body; when the height limiting mechanism descends to the lowest height, the first main body and the second main body come into contact, and the height adjusting mechanism cannot lower the height.

[0158] Furthermore, as Figure 5 shown, the height adjusting mechanism includes: a first height adjusting mechanism 2 and a second height adjusting mechanism 3. One side of the first height adjusting mechanism 2 and the second height adjusting mechanism 3 is slidably connected to one side of the rotating mechanism 1 through a first sliding track; the other sides of the first height adjusting mechanism 2 and the second height adjusting mechanism 3 are respectively connected to the outside of the fixing unit 4 through a first screw and a second screw 2-4. At the same time, both ends of the first height adjusting mechanism 2 and the second height adjusting mechanism 3 respectively have a first main body 2-2 and a second main body 2-3, and the opposite sides of the first main body 2-2 and the second main body 2-3 have inclined surfaces; when the height limiting mechanism descends to the lowest height, the inclined surfaces on the opposite sides of the first main body 2-2 and the second main body 2-3 come into contact, and the height adjusting mechanism cannot lower the height.

[0159] In embodiments of the present disclosure and other possible embodiments, the arm fixing mechanism 5 in the arm accommodating mechanism includes: an expandable or contractible fluid bag; the fixing unit controls the expandable or contractible fluid bag to expand based on the first control instruction, thereby fixing the arm. Among them, the fluid bag can be an air bag or a water bag, etc.

[0160] In some embodiments, the functions or modules included in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be elaborated here.

[0161] Embodiments of the present disclosure also propose a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above methods are implemented. The computer-readable storage medium can be a non-volatile computer-readable storage medium.

[0162] Embodiments of the present disclosure also propose an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to perform the above methods. The electronic device can be provided as a terminal, a server or other forms of devices.

[0163] In addition, embodiments of the present disclosure also propose an ultrasonic instrument that applies the above-described method for detecting the arm posture of operating an ultrasonic probe, and / or the above-described control method; and / or includes: the above-described detection device; and / or the above-described control device; and / or the above-described electronic device; and / or the above-described computer-readable storage medium.

[0164] Figure 6 FIG. 4 is a block diagram of an electronic device 800 shown according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and other terminals.

[0165] Referring to Figure 6 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0166] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0167] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0168] The power supply component 806 provides power to various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0169] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0170] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0171] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a power button, and a lock button.

[0172] The sensor component 814 includes one or more sensors for providing a status assessment of various aspects of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and the keypad of the electronic device 800. The sensor component 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or a CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0173] The communication component 816 is configured to facilitate communication, in a wired or wireless manner, between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0174] In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described methods.

[0175] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as a memory 804 including computer program instructions that may be executed by a processor 820 of the electronic device 800 to complete the above-described methods.

[0176] Figure 7 is a block diagram of an electronic device 1900 shown in accordance with an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Referring to Figure 7 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. Additionally, the processing component 1922 is configured to execute instructions to perform the above-described methods.

[0177] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.

[0178] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as the memory 1932 including computer program instructions, and the computer program instructions can be executed by the processing component 1922 of the electronic device 1900 to complete the above method.

[0179] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0180] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device, such as a punched card or raised structures in grooves storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0181] The computer-readable program instructions described herein can be downloaded to various computing / processing devices from a computer-readable storage medium or downloaded to an external computer or an external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0182] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer-readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present disclosure.

[0183] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0184] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises a manufacture including instructions which implement various aspects of the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0185] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0186] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, and the module, segment of code, or portion of an instruction may include one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the boxes may occur out of the order noted in the figures. For example, two consecutive blocks may, in fact, be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system for performing the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0187] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting the arm posture of operating an ultrasonic probe, characterized in that, Including: Obtaining multiple arm pose images of the operation probe at the current moment and multiple moments before the current moment; Extracting the sequence of arm joint point information and the sequence of motion speed information of the arm joint points of the multiple arm pose images; wherein, the extracting the sequence of arm joint point information of the multiple arm pose images includes: segmenting the multiple arm pose images to obtain corresponding multiple arm images; performing edge detection on the multiple arm images to obtain corresponding multiple arm edge images; based on the arm geometry, using the multiple arm edge images corresponding to the multiple arm pose images, determining the first arm joint point corresponding to the wrist in the multiple arm pose images and the second arm joint point where the lower arm and the upper arm of the arm are connected; wherein, the based on the arm geometry, using the multiple arm edge images corresponding to the multiple arm pose images, determining the first arm joint point corresponding to the wrist in the multiple arm pose images and the second arm joint point where the lower arm and the upper arm of the arm are connected includes: respectively calculating multiple distances between the first arm edge line and the second arm edge line in each arm edge image; determining the joint points corresponding to the two edge lines with the minimum distance among the multiple distances as the first arm joint point; extracting the arm center line of the arm edge image, and determining the bend corresponding to the arm center line along the lower arm direction as the second arm joint point; wherein, determining the direction of the lower arm includes: extracting the arm center line of the arm edge image; respectively calculating the first length and the second length corresponding to the arm center line along two directions of the arm based on the first arm joint point; determining the direction of the lower arm as the direction corresponding to the larger length among the first length and the second length; Performing arm pose recognition based on the sequence of arm joint point information and the sequence of motion speed information of the arm joint points to determine the arm pose of the operation probe; wherein, the performing arm pose recognition based on the sequence of arm joint point information and the sequence of motion speed information of the arm joint points to determine the arm pose of the operation probe includes: respectively performing spatial feature extraction on the sequence of arm joint point information and the sequence of motion speed information of the arm joint points to obtain corresponding first spatial feature and second spatial feature; and performing feature fusion on the first spatial feature and the second spatial feature to obtain an initial fusion feature; using a set sliding spatio-temporal window to perform a sliding spatial convolution operation on the initial fusion feature to obtain a first spatio-temporal graph and a first adjacency matrix; performing a spatial convolution operation on the initial fusion feature based on the first adjacency matrix to obtain a first convolution graph; fusing the first spatio-temporal graph and the first convolution graph to obtain a pose feature, and completing arm pose detection.

2. The detection method according to claim 1, wherein The extracting the sequence of motion speed information of the arm joint points of the multiple arm pose images includes: Respectively calculating multiple relative displacements of the first arm joint point and the second arm joint point at multiple moments before the current moment to obtain the sequence of motion speed information of the arm joint points.

3. The detection method according to claim 2, characterized in that Calculating the relative displacements of the first arm joint point and the second arm joint point at multiple moments before the current moment respectively to obtain a motion speed information sequence of the arm joint points, including: Determining the multiple first coordinates and multiple second coordinates corresponding to the first arm joint point and the second arm joint point at the current moment and multiple moments before the current moment respectively; Calculating the distances between the first coordinates corresponding to the current moment and the first coordinates corresponding to multiple moments before the current moment respectively to obtain a motion speed information sequence of the first arm joint point; Calculating the distances between the second coordinates corresponding to the current moment and the second coordinates corresponding to multiple moments before the current moment respectively to obtain a motion speed information sequence of the second arm joint point.

4. A control method for operating an ultrasonic probe, characterized in that, Including: Using the detection method according to any one of claims 1-3 to obtain the patient's body position and the arm posture of the operating probe in real time; Adjusting the motion orientation of the arm according to the patient's body position and the arm posture; wherein, adjusting the motion orientation of the arm according to the patient's body position and the arm posture includes: obtaining multiple set position information of the area to be operated in the patient's body position in real time, and determining the position information of the probe tip in the arm posture; determining the motion trajectory of the arm based on the multiple set position information and the position information of the probe tip; adjusting the motion orientation of the arm based on the motion trajectory; Wherein, during the process of adjusting the motion orientation of the arm, the contact pressure of the probe is obtained in real time, and the distance between the probe and the patient is adjusted according to the contact pressure and the set pressure.

5. A detection device for the arm posture of operating an ultrasonic probe, characterized in that, Including: An acquisition unit for acquiring arm posture images of the operating probe at the current moment and multiple moments before the current moment; Extract the sequence of arm joint point information and the sequence of motion speed information of the arm joint points of multiple said arm pose images; wherein, the extraction of the sequence of arm joint point information of multiple said arm pose images includes: segmenting the arms of multiple said arm pose images to obtain corresponding multiple arm images; performing edge detection on the multiple arm images to obtain corresponding multiple arm edge images; based on the arm geometry, using the multiple arm edge images corresponding to the multiple arm pose images, determining the first arm joint point corresponding to the wrist in the multiple arm pose images and the second arm joint point where the lower arm and the upper arm of the arm are connected; wherein, the determination of the first arm joint point corresponding to the wrist in the multiple arm pose images and the second arm joint point where the lower arm and the upper arm of the arm are connected based on the arm geometry and using the multiple arm edge images corresponding to the multiple arm pose images includes: respectively calculating multiple distances between the first arm edge line and the second arm edge line in each arm edge image; determining the joint points corresponding to the two edge lines corresponding to the minimum distance among the multiple distances as the first arm joint point; extracting the arm center line of the arm edge image, and determining the bend corresponding to the arm center line along the lower arm direction as the second arm joint point; wherein, determining the direction of the lower arm includes: extracting the arm center line of the arm edge image; respectively calculating the first length and the second length corresponding to the arm center line along two directions of the arm based on the first arm joint point; determining the direction of the lower arm as the direction corresponding to the larger one of the first length and the second length. Based on the sequence of arm joint point information and the sequence of motion speed information of the arm joint points, perform arm pose recognition to determine the arm pose of the operating probe; wherein, the performing of arm pose recognition based on the sequence of arm joint point information and the sequence of motion speed information of the arm joint points to determine the arm pose of the operating probe includes: respectively performing spatial feature extraction on the sequence of arm joint point information and the sequence of motion speed information of the arm joint points to obtain corresponding first spatial features and second spatial features; and performing feature fusion on the first spatial features and the second spatial features to obtain an initial fusion feature; using a set sliding spatio-temporal window to perform a sliding spatial convolution operation on the initial fusion feature to obtain a first spatio-temporal graph and a first adjacency matrix; performing a spatial convolution operation on the initial fusion feature based on the first adjacency matrix to obtain a first convolution graph; fusing the first spatio-temporal graph and the first convolution graph to obtain a pose feature, and completing arm pose detection.

6. A control device for operating an ultrasonic probe, characterized in that, Including: An acquisition unit, configured to use the detection device as described in claim 5 to acquire the patient's body position and the arm pose of the operating probe in real time; An orientation control unit for adjusting the movement orientation of the arm according to the patient's body position and the arm posture; wherein, adjusting the movement orientation of the arm according to the patient's body position and the arm posture includes: obtaining in real time a plurality of set position information of the area to be operated in the patient's body position, and determining the position information of the probe tip in the arm posture; determining the movement trajectory of the arm based on the plurality of set position information and the position information of the probe tip; adjusting the movement orientation of the arm based on the movement trajectory; wherein, during the process of adjusting the movement orientation of the arm, the contact pressure of the probe is obtained in real time, and the distance between the probe and the patient is adjusted according to the contact pressure and the set pressure.

7. An electronic device, characterized in that, Comprising: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the method for detecting the arm posture of the ultrasonic probe as described in any one of claims 1 to 3.

8. An electronic device, characterized in that, Comprising: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the control method as described in claim 4.

9. An electronic device, characterized in that, Comprising: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the method for detecting the arm posture of the ultrasonic probe as described in any one of claims 1 to 3 and the control method as described in claim 4.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, the method for detecting the arm posture of the ultrasonic probe as described in any one of claims 1 to 3 is implemented.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, the control method as described in claim 4 is implemented.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, the method for detecting the arm posture of the ultrasonic probe as described in any one of claims 1 to 3 and the control method as described in claim 4 are implemented.

13. An ultrasonic instrument, characterized in that, Comprising: The detection device as described in claim 5.

14. An ultrasonic instrument, characterized in that, Comprising: The control device as described in claim 6.

15. An ultrasonic instrument, characterized in that, Comprising: The detection device as described in claim 5 and the control device as described in claim 6.

16. An ultrasonic instrument, characterized in that, Comprising: The electronic device as described in any one of claims 7 - 9.

17. An ultrasonic instrument, characterized in that, Comprising: The computer-readable storage medium as described in any one of claims 10 - 12.

Images