Handheld ultrasonic control method and system based on multi-axis acceleration signals

Abstract

The invention relates to the technical field of handheld ultrasound, in particular to a handheld ultrasound control method and system based on a multi-axis acceleration signal, and the method comprises the steps: obtaining an ultrasonic echo sequence and a multi-axis acceleration signal sequence; searching the multi-axis acceleration signal sequence to determine a target time interval; comparing the ultrasonic echo sequences, and determining a problem echo interval; and marking the reconstructed image according to the problem echo interval so as to indicate the part needing to be scanned again. In order to solve the problem that a probe of the handheld ultrasonic equipment is deviated due to the knocking action so as to influence imaging, the multi-axis acceleration signal is analyzed, so that the time node when the user adjusts the holding posture and prepares to start knocking and the time point when the user returns to the stable holding posture after knocking is finished are determined, and the user experience is improved. The echo sequence is checked based on the time interval, and the interval in which the image quality may have influence is prompted to be scanned again, so that the influence of introduction of a knock control mode on imaging is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of handheld ultrasound, and in particular to a handheld ultrasound control method and system based on multi-axis acceleration signals. Background Art

[0002] A palmtop ultrasound device, also known as a handheld ultrasound, is a palm-sized, easy-to-control medical ultrasound device that combines a host and a probe into one. It is usually composed of a probe, a processor, a display, and a battery. It has the characteristics of small size, light weight, and low power consumption.

[0003] For example, Chinese patent CN201710560949.0 discloses a handheld three-dimensional ultrasound imaging system and method, including a handheld ultrasound probe for scanning and acquiring ultrasound images; a display, control and processing terminal connected to the handheld ultrasound probe via a wired or wireless connection. It also includes: a handheld three-dimensional spatial positioning system, which is connected to the handheld ultrasound probe and moves with the movement of the handheld ultrasound probe, and is connected to the display, control and processing terminal via a wired or wireless connection, for independently locating the three-dimensional position of the handheld ultrasound probe. By applying this handheld three-dimensional ultrasound imaging system and method, the huge spatial positioning system in the existing three-dimensional ultrasound imaging system is transformed into a portable and readily available spatial positioning system, so that handheld three-dimensional ultrasound imaging can be widely used.

[0004] However, during actual implementation, the inventors found that this type of technical solution usually uses buttons to input operating instructions during use. In some scenarios, the operation process is relatively simple and inconvenient to use. Summary of the Invention

[0005] In view of the above problems existing in the prior art, a handheld ultrasonic control method based on multi-axis acceleration signals is provided;

[0006] On the other hand, a handheld ultrasound control system for implementing the handheld ultrasound control method is also provided.

[0007] The specific technical solutions are as follows:

[0008] A handheld ultrasonic control method based on multi-axis acceleration signals, comprising:

[0009] Step S1: During the ultrasound scan, an ultrasound echo sequence is acquired and imaged, and a multi-axis accelerometer is used to detect a knocking motion. When a knocking motion occurs, the process proceeds to step S2;

[0010] The tapping action is used to control the handheld ultrasound device;

[0011] Step S2: searching the multi-axis acceleration signal sequence of the multi-axis accelerometer to determine the target time interval between when the user starts to make the tapping action and when the user ends the tapping action;

[0012] Step S3: comparing the ultrasonic echo sequence according to the target time interval to determine the problematic echo interval;

[0013] Step S4: marking the reconstructed image according to the problematic echo interval to indicate the part that needs to be re-scanned.

[0014] On the other hand, the step S1 includes:

[0015] Step S11: During the ultrasound scanning process, the ultrasound echo sequence and the multi-axis acceleration signal sequence are collected in the same time sequence;

[0016] Step S12: performing real-time image reconstruction based on the ultrasonic echo sequence to form the reconstructed image, and performing peak detection on the multi-axis acceleration signal sequence;

[0017] Step S13: When the multi-axis acceleration signal exceeds the signal amplitude threshold, it is considered that the tapping action occurs;

[0018] The signal amplitude threshold is a threshold value set for each axis in the multi-axis acceleration signal sequence;

[0019] Alternatively, a threshold is set for the vector sum of the multi-axis acceleration signal sequence.

[0020] On the other hand, the step S2 includes:

[0021] Step S21: performing multi-axis signal vector synthesis on the multi-axis acceleration signal sequence to form an original vector sequence;

[0022] The vector data of each frame in the original vector sequence respectively corresponds to the direction of the handheld ultrasound device at the current moment;

[0023] Step S22: Calculating the window vector mean of the original vector sequence using a sliding window to form a vector mean sequence;

[0024] Step S23: determining, based on the vector mean sequence, the segment nodes where the direction of the handheld ultrasound device changes during the scanning process;

[0025] Step S24: segmenting the multi-axis acceleration signal sequence using the segmentation nodes, and determining a to-be-identified interval and an adjacent interval associated with the to-be-identified interval in combination with the time point at which the tapping action is detected;

[0026] Step S25: extracting signal features according to the adjacent intervals and constructing a reference vector;

[0027] Step S26: intercepting the interval to be identified using a sliding window and extracting signal features to obtain a window vector, and comparing the window vector with the reference vector to determine an abnormal vector;

[0028] Step S27: Determine the time point corresponding to the abnormal vector as the target time interval.

[0029] On the other hand, step S3 includes:

[0030] Step S31: intercepting the ultrasonic echo sequence using the target time interval to determine an echo sequence to be identified;

[0031] Step S32: reconstructing the echo sequence to be identified to form an image sequence to be identified;

[0032] Step S33: performing frame-by-frame image quality assessment on the images in the image sequence to be identified to determine problematic image frames with affected image quality;

[0033] Step S34: determining the problematic echo sequence in the ultrasound echo sequence according to the problematic image frame.

[0034] On the other hand, the step S4 includes:

[0035] Step S41: constructing a panoramic image using the ultrasound echo sequence;

[0036] Step S42: adding a labeling box to the panoramic image according to the problematic echo sequence.

[0037] A handheld ultrasonic control system based on multi-axis acceleration signals, used to implement the above-mentioned handheld ultrasonic control method;

[0038] The handheld ultrasound control system includes:

[0039] An acquisition module, which acquires an ultrasonic echo sequence and performs imaging during an ultrasonic scan, and detects the tapping action using a multi-axis accelerometer;

[0040] The tapping action is used to control the handheld ultrasound device;

[0041] A search module, the search module is connected to the acquisition module;

[0042] The search module searches the multi-axis acceleration signal sequence of the multi-axis accelerometer to determine the target time interval between when the user starts to make the tapping action and when the tapping action ends;

[0043] A comparison module, the comparison module is connected to the search module;

[0044] The comparison module compares the ultrasonic echo sequence according to the target time interval to determine the problematic echo interval;

[0045] a marking module, the marking module being connected to the comparison module;

[0046] The marking module marks the reconstructed image according to the problematic echo interval to indicate the part that needs to be re-scanned.

[0047] On the other hand, the acquisition module includes:

[0048] a synchronous acquisition module, wherein the synchronous acquisition module acquires the ultrasonic echo sequence and the multi-axis acceleration signal sequence according to the same timing during the ultrasonic scanning process;

[0049] A real-time processing module, the real-time processing module is connected to the synchronous acquisition module;

[0050] The real-time processing module performs real-time image reconstruction based on the ultrasonic echo sequence to form the reconstructed image, and performs peak detection on the multi-axis acceleration signal sequence;

[0051] A threshold determination module, the threshold determination module being connected to the real-time processing module;

[0052] The threshold determination module determines that the tapping action occurs when the multi-axis acceleration signal exceeds the signal amplitude threshold;

[0053] The signal amplitude threshold is a threshold value set for each axis in the multi-axis acceleration signal sequence;

[0054] Alternatively, a threshold is set for the vector sum of the multi-axis acceleration signal sequence.

[0055] On the other hand, the search module includes:

[0056] a vector synthesis module, wherein the vector synthesis module performs multi-axis signal vector synthesis on the multi-axis acceleration signal sequence to form an original vector sequence;

[0057] The vector data of each frame in the original vector sequence respectively corresponds to the direction of the handheld ultrasound device at the current moment;

[0058] A vector calculation module, the vector calculation module is connected to the vector synthesis module;

[0059] The vector calculation module calculates the window vector mean of the original vector sequence using a sliding window to form a vector mean sequence;

[0060] a node determination module, the node determination module being connected to the vector calculation module;

[0061] The node determination module determines, based on the vector mean sequence, the segment nodes where the direction of the handheld ultrasound device changes during the scanning process;

[0062] A segmentation module, the segmentation module is connected to the node determination module;

[0063] The segmentation module segments the multi-axis acceleration signal sequence using the segmentation nodes, and determines a to-be-identified interval and an adjacent interval associated with the to-be-identified interval in combination with a time point at which the tapping action is detected;

[0064] a reference vector construction module connected to the segmentation module;

[0065] The reference vector construction module extracts signal features according to the adjacent intervals and constructs a reference vector;

[0066] a vector comparison module, the vector comparison module being connected to the reference vector construction module;

[0067] The vector comparison module intercepts the interval to be identified using a sliding window and extracts signal features to obtain a window vector, and compares the window vector with the reference vector to determine an abnormal vector;

[0068] an interval determination module, the interval determination module being connected to the vector comparison module;

[0069] The interval determination module determines the time point corresponding to the abnormal vector as the target time interval.

[0070] On the other hand, the comparison module includes:

[0071] an interval interception module, wherein the interval interception module intercepts the ultrasonic echo sequence using the target time interval to determine an echo sequence to be identified;

[0072] an interval reconstruction module, the interval reconstruction module being connected to the interval interception module;

[0073] The interval reconstruction module performs image reconstruction on the echo sequence to be identified to form an image sequence to be identified;

[0074] a quality evaluation module, the quality evaluation module being connected to the interval reconstruction module;

[0075] The quality assessment module performs frame-by-frame image quality assessment on the images in the image sequence to be identified to determine problematic image frames with affected image quality;

[0076] A sequence interception module, the sequence interception module is connected to the quality evaluation module;

[0077] The sequence interception module determines the problematic echo sequence in the ultrasound echo sequence according to the problematic image frame.

[0078] On the other hand, the marking module includes:

[0079] a panoramic image generation module, wherein the panoramic image generation module uses the ultrasonic echo sequence to construct a panoramic image;

[0080] a marking adding module, the marking adding module being connected to the panoramic image generating module;

[0081] The annotation adding module adds an annotation box to the panoramic image according to the problem echo sequence.

[0082] The above technical solution has the following advantages or beneficial effects:

[0083] 1. To address the problem that the operation method of handheld ultrasound devices in the existing technology is relatively simple, this solution integrates a multi-axis accelerometer in the handheld ultrasound device. The multi-axis accelerometer collects the acceleration of each axis to determine whether the user has made a tapping action, and then combines the number of taps to determine the tapping action input by the user, providing a new operation method.

[0084] 2. To address the problem that tapping actions may cause the probe of the handheld ultrasound device to shift, thereby affecting imaging, this solution introduces a process for detecting multi-axis acceleration signals. By analyzing the multi-axis acceleration signals, the time nodes when the user adjusts the grip and prepares to start tapping, and the time points when the user returns to a stable grip after tapping are determined. Based on the above time intervals, the echo sequence is checked, and the intervals where image quality may be affected are prompted to be re-scanned, avoiding the impact of the introduction of tapping control on imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] The embodiments of the present invention will be described more fully with reference to the accompanying drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0086] Figure 1 is an overall schematic diagram of an embodiment of the present invention;

[0087] Figure 2 This is a schematic diagram of step S1 in an embodiment of the present invention;

[0088] Figure 3 This is a schematic diagram of step S2 in an embodiment of the present invention;

[0089] Figure 4 This is a schematic diagram of step S3 in an embodiment of the present invention;

[0090] Figure 5 This is a schematic diagram of step S4 in an embodiment of the present invention;

[0091] Figure 6 A schematic diagram of a system in an embodiment of the present invention;

[0092] Figure 7 This is a schematic diagram of an acquisition module in an embodiment of the present invention;

[0093] Figure 8 A schematic diagram of a search module in an embodiment of the present invention;

[0094] Figure 9 Schematic diagram of a comparison module in an embodiment of the present invention;

[0095] Figure 10 Schematic diagram of a marking module in an embodiment of the present invention. DETAILED DESCRIPTION

[0096] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0097] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0098] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0099] The present invention comprises:

[0100] A handheld ultrasonic control method based on multi-axis acceleration signals, such as Figure 1 Shown, including:

[0101] Step S1: During the ultrasound scan, an ultrasound echo sequence is acquired and imaged, and a multi-axis accelerometer is used to detect a knocking motion. When a knocking motion occurs, the process proceeds to step S2;

[0102] The tapping action is used to control the handheld ultrasound device;

[0103] Step S2: searching the multi-axis acceleration signal sequence of the multi-axis accelerometer to determine the target time interval between when the user starts to make the tapping action and when the user ends the tapping action;

[0104] Step S3: comparing the ultrasonic echo sequence according to the target time interval to determine the problematic echo interval;

[0105] Step S4: Mark the reconstructed image according to the problematic echo interval to indicate the part that needs to be re-scanned.

[0106] Specifically, in response to the problem that the operation method of handheld ultrasound devices in the existing technology is relatively single, in this solution, a multi-axis accelerometer is integrated into the handheld ultrasound device. The multi-axis accelerometer collects the acceleration of each axis to determine whether the user has made a tapping action, and then combines the number of taps to determine the tapping action input by the user, providing a new operation method.

[0107] Furthermore, in order to address the problem that tapping actions may cause the probe of the handheld ultrasound device to shift, thereby affecting imaging, this solution introduces a process for detecting multi-axis acceleration signals. By analyzing the multi-axis acceleration signals, the time nodes when the user adjusts his grip and prepares to start tapping, and the time points when the user returns to a stable grip after tapping are determined. Based on the above time intervals, the echo sequence is checked, and the intervals where image quality may be affected are prompted to be re-scanned, avoiding the impact of the introduction of tapping control methods on imaging.

[0108] Specifically, the above-mentioned handheld ultrasound control method is mainly configured in the handheld ultrasound device as a software embodiment, and runs in parallel with other programs in the handheld ultrasound device to realize corresponding functions.

[0109] A handheld ultrasound device is an integrated device that integrates array element driving, echo acquisition, image reconstruction, image transmission and other functions. It can drive the array element according to the parameters set by the user, transmit ultrasound to the patient's predetermined position and collect the reflected signal of the tissue to form an echo sequence, and then reconstruct the image according to the echo sequence, and then display it on the display screen.

[0110] To achieve a simpler control process, this handheld ultrasound device also integrates a vibration control function. This function includes at least one set of multi-axis accelerometers, which can mainly collect acceleration signals in the three axes of X, Y, and Z. By calculating the vector sum and combining it with the corresponding judgment program, for example, the number of acceleration peaks that appear within a certain period of time is considered to be the number of tapping actions, thereby matching the corresponding operations, such as adjusting the brightness, display mode, ultrasound parameters, etc.

[0111] Because the driving of the array elements during operation of an ultrasound device generates certain harmonics, a higher threshold is often set for tapping to improve the accuracy of motion recognition. This causes the user to unconsciously change the way they hold the handheld ultrasound device during tapping to facilitate the application of force. This can cause the handheld ultrasound device to deviate slightly during scanning, leading to misalignment and reduced pressure at the coupling interface between the probe surface and the patient's body surface. This can degrade the ultrasound image during this process, causing problems in subsequent steps such as artificial intelligence recognition.

[0112] To address this situation, the above-mentioned control method is added to this application. First, a multi-axis accelerometer is used to detect the tapping action. When a tapping action occurs, two parallel branches are entered, including a branch for searching the signal sequence, and a branch for judging and controlling based on the tapping action itself.

[0113] Subsequent branches can be implemented based on the above-mentioned peak matching method or other equivalent technologies.

[0114] The signal sequence search process primarily searches for the multi-axis acceleration signal sequence of the multi-axis accelerometer. By evaluating the change in acceleration, we determine the time when the user adjusts their grip and begins tapping, and the time when the user ends tapping and returns to a stable grip. These two time points serve as the target time intervals.

[0115] The ultrasound echo sequence is then compared according to the target time interval to identify problematic echo intervals caused by ultrasound probe deflection. This comparison process can be performed to assess the image quality of the image captured in the echo, or it can be performed to measure the signal itself to determine whether the signal-to-noise ratio has decreased or whether the signal has experienced a sudden change. This allows the user's grip to be changed to identify intervals that affect actual imaging.

[0116] Finally, the reconstructed image is marked according to the problem echo interval, including adding annotation boxes, colored areas, etc. In some multimodal scanning scenarios, it also includes annotation transplantation of the surface optical image to indicate to the doctor the part that needs to be re-scanned. The doctor can make up for the image defects of that part by re-scanning.

[0117] In one embodiment, Figure 2 As shown, step S1 includes:

[0118] Step S11: During the ultrasound scanning process, an ultrasound echo sequence and a multi-axis acceleration signal sequence are collected in the same time sequence;

[0119] Step S12: performing real-time image reconstruction based on the ultrasonic echo sequence to form a reconstructed image, and performing peak detection on the multi-axis acceleration signal sequence;

[0120] Step S13: When a multi-axis acceleration signal exceeding a signal amplitude threshold appears, it is considered that a tapping action has occurred;

[0121] The signal amplitude threshold is a threshold set for each axis in the multi-axis acceleration signal sequence;

[0122] Alternatively, a threshold value is set for the vector sum of a multi-axis acceleration signal sequence.

[0123] Specifically, to implement the above processing, in this embodiment, a synchronous scanning process is first configured in the acquisition phase. Specifically, during the ultrasound scanning process, the ultrasound echo sequence and the multi-axis acceleration signal sequence are acquired in the same time sequence.

[0124] Among them, the ultrasound echo sequence is an echo sequence generated based on pre-configured scanning parameters and reflected by tissue. It can be used in the image reconstruction process to form a real-time reconstructed image and stitch previous images to form a complete panoramic image.

[0125] A multi-axis acceleration signal sequence is a signal sequence generated by collecting the acceleration data of the entire device through a multi-axis accelerometer. It usually has three signal channels, corresponding to the X, Y, and Z axes respectively.

[0126] Considering that the sampling frequency of the multi-axis accelerometer is often higher than that of the ultrasound probe, the original signal needs to be resampled during the acquisition process, including resetting the sampling frequency to the sampling frequency of the ultrasound probe and aligning the sampling points of the two.

[0127] The above resampling process can effectively align each image frame with the acceleration data of the three axes, making it easier to find the problem frame later.

[0128] After sampling, the ultrasound echo sequence and the multi-axis acceleration signal sequence enter two parallel branches, including image reconstruction and peak detection of the multi-axis acceleration signal sequence. When a multi-axis acceleration signal exceeds the signal amplitude threshold, it is considered a tapping action. The signal amplitude threshold is set for each axis in the multi-axis acceleration signal sequence, that is, the signal amplitude in any channel exceeds the limit; alternatively, the vector sum of the multi-axis acceleration signal sequence is first calculated, and then the threshold is set for the vector sum.

[0129] When it is determined that a knocking action occurs, the detection process of S2 is entered, and in parallel, the knocking action itself is judged.

[0130] In one embodiment, Figure 3 As shown, step S2 includes:

[0131] Step S21: performing multi-axis signal vector synthesis on the multi-axis acceleration signal sequence to form an original vector sequence;

[0132] The vector data of each frame in the original vector sequence corresponds to the acceleration direction of the handheld ultrasound device at the current moment;

[0133] Step S22: Calculating the window vector mean of the original vector sequence using a sliding window to form a vector mean sequence;

[0134] Step S23: determining, based on the vector mean sequence, the segment nodes where the direction of the handheld ultrasound device changes during the scanning process;

[0135] Step S24: segmenting the multi-axis acceleration signal sequence using segmentation nodes, and determining the interval to be identified and the adjacent intervals associated with the interval to be identified in combination with the time point when the tapping action is detected;

[0136] Step S25: extracting signal features according to adjacent intervals and constructing a reference vector;

[0137] Step S26: intercepting the interval to be identified using a sliding window and extracting signal features to obtain a window vector, and comparing the window vector with a reference vector to determine an abnormal vector;

[0138] Step S27: Determine the time point corresponding to the abnormal vector as the target time interval.

[0139] Specifically, after the user performs a tapping action, in order to effectively determine the time point when the user's gripping action changes, in this embodiment, the multi-axis acceleration signal sequence is first subjected to multi-axis signal vector synthesis, that is, the actual vector sum of the acceleration components in the three signal channels is calculated to form an original vector sequence; the vector data of each frame in the original vector sequence corresponds to the acceleration direction of the handheld ultrasound device at the current moment.

[0140] During a scan with a handheld ultrasound device, for example, during a scan of a patient's abdominal cavity, it can generally be assumed that the handheld ultrasound device scans in one direction, reaches a boundary, moves back to the other end, or turns back to scan another line in the opposite direction.

[0141] Therefore, during each line scan, the acceleration direction of the handheld ultrasound device should remain consistent when the positive and negative directions are omitted, and only obvious deflection will occur at the reflected position.

[0142] When the adjustment of the holding posture is introduced, an additional acceleration fluctuation direction will be introduced into the movement process that should have been in a steady state. Based on this direction, it can be determined that the adjustment of the holding action has affected the scanning imaging process.

[0143] In order to extract the action, in this embodiment, a sliding window is first used to calculate the window vector mean of the original vector sequence to form a vector mean sequence, thereby removing the influence of smaller acceleration fluctuations and performing comparison on a relatively long window.

[0144] For this series of windows, it is generally believed that the obvious peaks are caused by the scanning segment reaching the boundary and needing to be re-directed and scanned. Based on peak detection and signal feature comparison, the segment nodes where the direction of the handheld ultrasound device changed during the scanning process can be determined.

[0145] In the interval formed under each segmentation node, the segment with a consistent scanning direction is usually identified. At this time, the segment where the tapping action occurs can be determined according to the time point when the tapping action was previously detected, and thus used as the interval to be identified. Then, an adjacent interval is extracted from the interval to be identified as the adjacent interval.

[0146] For this adjacent interval, a pre-trained artificial intelligence model is used to extract its signal features and generate a reference vector.

[0147] For the interval to be identified, a sliding window is used to segment it, and the signal features are extracted for each segment and a window vector is constructed.

[0148] At this point, the window vector should have signal characteristics similar to the reference vector. If the window vector shows a large change, it indicates that the vector is abnormal. The abnormal vectors are selected, and the time enclosed by a series of abnormal vectors is determined as the target time interval.

[0149] In one embodiment, Figure 4 As shown, step S3 includes:

[0150] Step S31: intercepting the ultrasonic echo sequence using the target time interval to determine the echo sequence to be identified;

[0151] Step S32: reconstructing the echo sequence to be identified to form an image sequence to be identified;

[0152] Step S33: performing frame-by-frame image quality assessment on the images in the image sequence to be identified to determine problematic image frames with affected image quality;

[0153] Step S34: determining a problematic echo sequence in the ultrasound echo sequence according to the problematic image frame.

[0154] Specifically, in order to achieve a more accurate measurement of image quality, in this embodiment, after determining the target time interval, the ultrasound echo sequence is first intercepted to determine the echo sequence to be identified.

[0155] Then, image reconstruction is performed on each frame in the to-be-identified echo sequence to form an to-be-identified image sequence, which includes multiple image frames.

[0156] Subsequently, the images in the image sequence to be identified are evaluated frame by frame to determine the problem image frames whose image quality is affected. The evaluation process includes whether there are artifacts in the image, whether there is a significant decrease or defect in the sharpness of the image, etc., to determine the problem image frames whose image quality is affected.

[0157] Finally, the ultrasonic echo sequence is reversely searched according to the problem image frame to determine the problem echo sequence.

[0158] In one embodiment, Figure 5 As shown, step S4 includes:

[0159] Step S41: constructing a panoramic image using an ultrasonic echo sequence;

[0160] Step S42: adding a labeling box to the panoramic image according to the problematic echo sequence.

[0161] Specifically, to achieve better annotation results, this embodiment first constructs a panoramic image using a complete ultrasound echo sequence. The various parts of this panoramic image are typically spliced together in a time sequence. Therefore, annotations can be added to the panoramic image based on the acquisition time points corresponding to the problematic echo sequence, achieving better prompting results.

[0162] A handheld ultrasonic control system based on multi-axis acceleration signals, used to implement the above-mentioned handheld ultrasonic control method;

[0163] like Figure 6 As shown, the handheld ultrasound control system includes:

[0164] Acquisition module 1, during the ultrasonic scanning process, acquires an ultrasonic echo sequence and performs imaging, and uses a multi-axis accelerometer to detect the tapping action;

[0165] The tapping action is used to control the handheld ultrasound device;

[0166] Search module 2, search module 2 is connected to acquisition module 1;

[0167] Search module 2 searches the multi-axis acceleration signal sequence of the multi-axis accelerometer to determine the target time interval between the start and end of the user's tapping action;

[0168] Comparison module 3, comparison module 3 is connected to search module 2;

[0169] The comparison module 3 compares the ultrasonic echo sequence according to the target time interval to determine the problematic echo interval;

[0170] Marking module 4, marking module 4 is connected to comparison module 3;

[0171] The marking module 4 marks the reconstructed image according to the problematic echo interval to indicate the part that needs to be re-scanned.

[0172] Specifically, in order to address the problem in the prior art that the tapping action will cause the probe of the handheld ultrasound device to deviate, thereby affecting imaging, in this solution, first, the acquisition module 1 uses a multi-axis accelerometer to detect the tapping action. When the tapping action occurs, it enters two parallel branches, including a branch for searching the signal sequence, and a branch for judging and controlling based on the tapping action itself.

[0173] Subsequent branches can be implemented based on the above-mentioned peak matching method or other equivalent technologies.

[0174] Search module 2 searches for the signal sequence by searching the multi-axis acceleration signal sequence of the multi-axis accelerometer. By judging the change in acceleration, it determines the time when the user adjusts the grip and starts the tapping action, and the time when the user ends the tapping action and returns to a stable grip. These two time points are used as the target time interval.

[0175] Comparison module 3 then compares the ultrasound echo sequence according to the target time interval to identify problematic echo intervals where probe deviation has caused echo problems. This comparison process can be an image quality assessment of the image captured in the echo, or it can be a process that measures the signal itself to determine whether the signal-to-noise ratio has decreased or whether the signal has experienced a sudden change. This allows the user's grip to be changed to identify intervals that affect actual imaging.

[0176] Finally, the marking module 4 marks the reconstructed image according to the problem echo interval, including adding annotation boxes, colored areas, etc. In some multimodal scanning scenarios, it also includes annotation transplantation of the surface optical image to indicate to the doctor the parts that need to be re-scanned. The doctor can make up for the image defects of this part by re-scanning.

[0177] In one embodiment, Figure 7 As shown, the acquisition module 1 includes:

[0178] Synchronous acquisition module 11, which acquires ultrasonic echo sequence and multi-axis acceleration signal sequence according to the same timing during the ultrasonic scanning process;

[0179] A real-time processing module 12, the real-time processing module 12 is connected to the synchronous acquisition module 11;

[0180] The real-time processing module 12 performs real-time image reconstruction based on the ultrasonic echo sequence to form a reconstructed image, and performs peak detection on the multi-axis acceleration signal sequence;

[0181] A threshold determination module 13, the threshold determination module 13 is connected to the real-time processing module 12;

[0182] The threshold determination module 13 considers that a tapping action occurs when a multi-axis acceleration signal exceeding a signal amplitude threshold appears;

[0183] The signal amplitude threshold is a threshold set for each axis in the multi-axis acceleration signal sequence;

[0184] Alternatively, a threshold value is set for the vector sum of a multi-axis acceleration signal sequence.

[0185] Specifically, to implement the above-mentioned processing, in this embodiment, the synchronous acquisition module 11 first configures a synchronous scanning process during the acquisition phase. Specifically, during the ultrasonic scanning process, the ultrasonic echo sequence and the multi-axis acceleration signal sequence are acquired in the same time sequence.

[0186] Among them, the ultrasound echo sequence is an echo sequence generated based on pre-configured scanning parameters and reflected by tissue. It can be used in the image reconstruction process to form a real-time reconstructed image and stitch previous images to form a complete panoramic image.

[0187] A multi-axis acceleration signal sequence is a signal sequence generated by collecting the acceleration data of the entire device through a multi-axis accelerometer. It usually has three signal channels, corresponding to the X, Y, and Z axes respectively.

[0188] Considering that the sampling frequency of the multi-axis accelerometer is often higher than that of the ultrasound probe, the original signal needs to be resampled during the acquisition process, including resetting the sampling frequency to the sampling frequency of the ultrasound probe and aligning the sampling points of the two.

[0189] The above resampling process can effectively align each image frame with the acceleration data of the three axes, making it easier to find the problem frame later.

[0190] After the sampling is completed, the real-time processing module 12 enters two parallel branches for the ultrasonic echo sequence and the multi-axis acceleration signal sequence, respectively, including image reconstruction and peak detection of the multi-axis acceleration signal sequence. When a multi-axis acceleration signal exceeding the signal amplitude threshold appears, the threshold judgment module 13 considers that a tapping action has occurred.

[0191] In one embodiment, Figure 8 As shown, the search module 2 includes:

[0192] A vector synthesis module 21 performs multi-axis signal vector synthesis on the multi-axis acceleration signal sequence to form an original vector sequence;

[0193] The vector data of each frame in the original vector sequence corresponds to the direction of the handheld ultrasound device at the current moment;

[0194] A vector calculation module 22, the vector calculation module 22 is connected to the vector synthesis module 21;

[0195] The vector calculation module 22 calculates the window vector mean of the original vector sequence using a sliding window to form a vector mean sequence;

[0196] A node determination module 23, the node determination module 23 is connected to the vector calculation module 22;

[0197] The node determination module 23 determines, based on the vector mean sequence, the segment nodes where the direction of the handheld ultrasound device changes during the scanning process;

[0198] Segmentation module 24, segmentation module 24 is connected to node determination module 23;

[0199] The segmentation module 24 segments the multi-axis acceleration signal sequence using segmentation nodes, and determines the interval to be identified and the adjacent intervals associated with the interval to be identified in combination with the time point when the tapping action is detected;

[0200] A reference vector construction module 25, which is connected to the segmentation module 24;

[0201] The reference vector construction module 25 extracts signal features according to adjacent intervals and constructs a reference vector;

[0202] A vector comparison module 26, the vector comparison module 26 is connected to the reference vector construction module 25;

[0203] The vector comparison module 26 intercepts the interval to be identified using a sliding window and extracts signal features to obtain a window vector, and compares the window vector with a reference vector to determine an abnormal vector;

[0204] An interval determination module 27, the interval determination module 27 is connected to the vector comparison module 26;

[0205] The interval determination module 27 determines the time point corresponding to the abnormal vector as the target time interval.

[0206] Specifically, after the user performs a tapping action, in order to effectively determine the time point when the user's gripping action changes, in this embodiment, the vector synthesis module 21 first performs multi-axis signal vector synthesis on the multi-axis acceleration signal sequence, that is, the actual vector sum of the acceleration components in the three signal channels is calculated to form an original vector sequence; the vector data of each frame in the original vector sequence corresponds to the acceleration direction of the handheld ultrasound device at the current moment.

[0207] The vector calculation module 22 calculates the window vector mean of the original vector sequence using a sliding window to form a vector mean sequence, thereby removing the influence of smaller acceleration fluctuations and performing comparison on a relatively longer window.

[0208] For this series of windows, it is generally believed that the obvious peaks therein are caused by the scanning segment reaching the boundary and needing to be re-directed and scanned. The node determination module 23 can determine the segment nodes where the direction of the handheld ultrasound device changes during the scanning process based on peak detection and signal feature comparison.

[0209] In the interval formed under each segmentation node, it is usually determined to be a segment with a consistent scanning direction. At this time, the segmentation module 24 can determine the segment where the tapping action occurs according to the time point when the tapping action was previously detected, and thus use it as the interval to be identified. Then, an adjacent interval is extracted for the interval to be identified as the adjacent interval.

[0210] For the adjacent interval, the reference vector construction module 25 uses a pre-trained artificial intelligence model to extract its signal features and generate a reference vector.

[0211] For the interval to be identified, the vector comparison module 26 uses a sliding window to segment it, extracts signal features for each segment and constructs a window vector.

[0212] At this time, the window vector should have signal characteristics similar to the reference vector. If the window vector shows a large change, it indicates that the vector is abnormal. The interval determination module 27 selects the abnormal vectors and determines the time enclosed by a series of abnormal vectors as the target time interval.

[0213] In one embodiment, Figure 9 As shown, the comparison module 3 includes:

[0214] The interval interception module 31 intercepts the ultrasonic echo sequence using the target time interval to determine the echo sequence to be identified;

[0215] The interval reconstruction module 32 is connected to the interval interception module 31;

[0216] The interval reconstruction module 32 performs image reconstruction on the echo sequence to be identified to form an image sequence to be identified;

[0217] The quality evaluation module 33 is connected to the interval reconstruction module 32;

[0218] The quality assessment module 33 performs frame-by-frame image quality assessment on the images in the image sequence to be identified to determine problematic image frames with affected image quality;

[0219] A sequence interception module 34, the sequence interception module 34 is connected to the quality evaluation module 33;

[0220] The sequence interception module 34 determines a problematic echo sequence in the ultrasound echo sequence according to the problematic image frame.

[0221] Specifically, in order to achieve a more accurate measurement of image quality, in this embodiment, after determining the target time interval, the interval interception module 31 first intercepts the ultrasound echo sequence to determine the echo sequence to be identified.

[0222] Then, the interval reconstruction module 32 performs image reconstruction on each frame in the to-be-identified echo sequence to form an to-be-identified image sequence, which includes multiple image frames.

[0223] Subsequently, the quality assessment module 33 performs frame-by-frame image quality assessment on the images in the image sequence to be identified to determine the problem image frames whose image quality is affected. The assessment process includes whether artifacts appear in the image, whether the sharpness of the image is significantly reduced or defective, etc., to determine the problem image frames whose image quality is affected.

[0224] Finally, the sequence interception module 34 performs a reverse search on the ultrasound echo sequence according to the problematic image frame to determine the problematic echo sequence.

[0225] In one embodiment, Figure 10 As shown, the marking module 4 includes:

[0226] A panoramic image generation module 41 , which uses an ultrasonic echo sequence to construct a panoramic image;

[0227] Annotation adding module 42, the annotation adding module 42 is connected to the panoramic image generating module 41;

[0228] The annotation adding module 42 adds an annotation box in the panoramic image according to the problematic echo sequence.

[0229] Specifically, to achieve better annotation results, in this embodiment, the panoramic image generation module 41 first constructs a panoramic image using a complete ultrasound echo sequence. The various components of this panoramic image are typically spliced together in a time sequence. Therefore, the annotation adding module 42 adds annotations to the panoramic image based on the acquisition time points corresponding to the problematic echo sequence, achieving better prompting results.

[0230] The above are only preferred embodiments of the present invention and do not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A handheld ultrasonic control method based on multi-axis acceleration signals, characterized in that: include: Step S1: During the ultrasound scan, an ultrasound echo sequence is acquired and imaged, and a multi-axis accelerometer is used to detect a knocking motion. When a knocking motion occurs, the process proceeds to step S2; The tapping action is used to control the handheld ultrasound device; Step S2: searching the multi-axis acceleration signal sequence of the multi-axis accelerometer to determine the target time interval between when the user starts to make the tapping action and when the user ends the tapping action; Step S3: comparing the ultrasonic echo sequence according to the target time interval to determine the problematic echo interval; Step S4: marking the reconstructed image according to the problematic echo interval to indicate the part that needs to be re-scanned.

2. The handheld ultrasonic control method according to claim 1, characterized in that: The step S1 comprises: Step S11: During the ultrasound scanning process, the ultrasound echo sequence and the multi-axis acceleration signal sequence are collected in the same time sequence; Step S12: performing real-time image reconstruction based on the ultrasonic echo sequence to form the reconstructed image, and performing peak detection on the multi-axis acceleration signal sequence; Step S13: When the multi-axis acceleration signal exceeds the signal amplitude threshold, it is considered that the tapping action occurs; The signal amplitude threshold is a threshold value set for each axis in the multi-axis acceleration signal sequence; Alternatively, a threshold is set for the vector sum of the multi-axis acceleration signal sequence.

3. The handheld ultrasonic control method according to claim 1, characterized in that: The step S2 comprises: Step S21: performing multi-axis signal vector synthesis on the multi-axis acceleration signal sequence to form an original vector sequence; The vector data of each frame in the original vector sequence respectively corresponds to the acceleration direction of the handheld ultrasound device at the current moment; Step S22: Calculating the window vector mean of the original vector sequence using a sliding window to form a vector mean sequence; Step S23: determining, based on the vector mean sequence, the segment nodes where the direction of the handheld ultrasound device changes during the scanning process; Step S24: segmenting the multi-axis acceleration signal sequence using the segmentation nodes, and determining a to-be-identified interval and an adjacent interval associated with the to-be-identified interval in combination with the time point at which the tapping action is detected; Step S25: extracting signal features according to the adjacent intervals and constructing a reference vector; Step S26: intercepting the interval to be identified using a sliding window and extracting signal features to obtain a window vector, and comparing the window vector with the reference vector to determine an abnormal vector; Step S27: Determine the time point corresponding to the abnormal vector as the target time interval.

4. The handheld ultrasonic control method according to claim 1, characterized in that: The step S3 comprises: Step S31: intercepting the ultrasonic echo sequence using the target time interval to determine an echo sequence to be identified; Step S32: reconstructing the echo sequence to be identified to form an image sequence to be identified; Step S33: performing frame-by-frame image quality assessment on the images in the image sequence to be identified to determine problematic image frames with affected image quality; Step S34: determining the problematic echo sequence in the ultrasound echo sequence according to the problematic image frame.

5. The handheld ultrasonic control method according to claim 1, characterized in that: The step S4 comprises: Step S41: constructing a panoramic image using the ultrasound echo sequence; Step S42: adding a labeling box to the panoramic image according to the problematic echo sequence.

6. A handheld ultrasonic control system based on multi-axis acceleration signals, characterized in that: Used to implement the handheld ultrasonic control method according to any one of claims 1 to 5; The handheld ultrasound control system includes: An acquisition module, which acquires an ultrasonic echo sequence and performs imaging during an ultrasonic scan, and detects the tapping action using a multi-axis accelerometer; The tapping action is used to control the handheld ultrasound device; A search module, the search module is connected to the acquisition module; The search module searches the multi-axis acceleration signal sequence of the multi-axis accelerometer to determine the target time interval between when the user starts to make the tapping action and when the tapping action ends; A comparison module, the comparison module is connected to the search module; The comparison module compares the ultrasonic echo sequence according to the target time interval to determine the problematic echo interval; a marking module, the marking module being connected to the comparison module; The marking module marks the reconstructed image according to the problematic echo interval to indicate the part that needs to be re-scanned.

7. The handheld ultrasonic control system according to claim 6, characterized in that: The acquisition module includes: a synchronous acquisition module, wherein the synchronous acquisition module acquires the ultrasonic echo sequence and the multi-axis acceleration signal sequence according to the same timing during the ultrasonic scanning process; A real-time processing module, the real-time processing module is connected to the synchronous acquisition module; The real-time processing module performs real-time image reconstruction based on the ultrasonic echo sequence to form the reconstructed image, and performs peak detection on the multi-axis acceleration signal sequence; A threshold determination module, the threshold determination module being connected to the real-time processing module; The threshold determination module determines that the tapping action occurs when the multi-axis acceleration signal exceeds the signal amplitude threshold; The signal amplitude threshold is a threshold value set for each axis in the multi-axis acceleration signal sequence; Alternatively, a threshold is set for the vector sum of the multi-axis acceleration signal sequence.

8. The handheld ultrasonic control system according to claim 6, characterized in that: The search module includes: a vector synthesis module, wherein the vector synthesis module performs multi-axis signal vector synthesis on the multi-axis acceleration signal sequence to form an original vector sequence; The vector data of each frame in the original vector sequence respectively corresponds to the direction of the handheld ultrasound device at the current moment; A vector calculation module, the vector calculation module is connected to the vector synthesis module; The vector calculation module calculates the window vector mean of the original vector sequence using a sliding window to form a vector mean sequence; a node determination module, the node determination module being connected to the vector calculation module; The node determination module determines, based on the vector mean sequence, the segment nodes where the direction of the handheld ultrasound device changes during the scanning process; A segmentation module, the segmentation module is connected to the node determination module; The segmentation module segments the multi-axis acceleration signal sequence using the segmentation nodes, and determines a to-be-identified interval and an adjacent interval associated with the to-be-identified interval in combination with a time point at which the tapping action is detected; a reference vector construction module connected to the segmentation module; The reference vector construction module extracts signal features according to the adjacent intervals and constructs a reference vector; a vector comparison module, the vector comparison module being connected to the reference vector construction module; The vector comparison module intercepts the interval to be identified using a sliding window and extracts signal features to obtain a window vector, and compares the window vector with the reference vector to determine an abnormal vector; an interval determination module, the interval determination module being connected to the vector comparison module; The interval determination module determines the time point corresponding to the abnormal vector as the target time interval.

9. The handheld ultrasonic control system according to claim 6, characterized in that: The comparison module includes: an interval interception module, wherein the interval interception module intercepts the ultrasonic echo sequence using the target time interval to determine an echo sequence to be identified; an interval reconstruction module, the interval reconstruction module being connected to the interval interception module; The interval reconstruction module performs image reconstruction on the echo sequence to be identified to form an image sequence to be identified; a quality evaluation module, the quality evaluation module being connected to the interval reconstruction module; The quality assessment module performs frame-by-frame image quality assessment on the images in the image sequence to be identified to determine problematic image frames with affected image quality; A sequence interception module, the sequence interception module is connected to the quality evaluation module; The sequence interception module determines the problematic echo sequence in the ultrasound echo sequence according to the problematic image frame.

10. The handheld ultrasonic control system according to claim 6, characterized in that: The marking module includes: a panoramic image generation module, wherein the panoramic image generation module uses the ultrasonic echo sequence to construct a panoramic image; a annotation adding module, the annotation adding module being connected to the panoramic image generating module; The annotation adding module adds an annotation box to the panoramic image according to the problem echo sequence.

Images