Ultrasonic image and pose data synchronous acquisition method, device, equipment, medium and product

Abstract

The invention discloses an ultrasonic image and pose data synchronous acquisition method, device, equipment, medium and product, and relates to the field of ultrasonic image localization, the device comprises an acquisition system host, a pulse generator, a positioning unit and an ultrasonic machine; the pulse generator sends the synchronous pulse signal to the positioning unit and the acquisition system host at the same time; the ultrasonic probe collects an ultrasonic image; the positioning unit comprises a sensor and an electromagnetic signal generator; the sensor is arranged on the ultrasonic probe; the electromagnetic signal generator sends electromagnetic waves to the sensor and receives echo signals of the electromagnetic waves; the positioning unit is used for positioning real-time pose data of the ultrasonic probe in the moving process according to the echo signal on the basis of a motion tracking system; the collection system host records the kernel timestamp of the synchronous pulse signal and synchronously collects the ultrasonic image output by the ultrasonic probe and the pose signal output by the positioning unit, and the application can accurately collect the pose change in the probe moving process and synchronously collect the ultrasonic image and the pose signal.

Description

Technical Field

[0001] The present application relates to the field of ultrasonic image positioning, and in particular to a method, device, equipment, medium and product for synchronously collecting ultrasonic images and posture data. Background Art

[0002] In the medical field, ultrasound image acquisition is an effective way to observe human biology and the development of disease. In the field of cardiac ultrasound, cardiac ultrasound doctors need to make tiny probe movements for each section when performing ultrasound testing to locate posture changes during movement. However, the existing camera-based computer video positioning algorithm requires the installation of multiple cameras and the use of image feature matching algorithms to extract corresponding points in the image. The matching accuracy is low, and positioning may also fail due to camera obstruction. Therefore, during high-precision, small-range movement, it is impossible to accurately capture posture changes during probe movement, resulting in the inability to synchronously capture ultrasound images and posture data. Summary of the invention

[0003] The purpose of this application is to provide a method, device, equipment, medium and product for synchronously collecting ultrasound images and posture data, so as to solve the problem that the posture changes during the movement of the probe cannot be accurately collected, resulting in the inability to synchronously collect ultrasound images and posture data.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a device for synchronously collecting ultrasound images and posture data, comprising:

[0006] A collection system host, and a pulse generator, a positioning unit and an ultrasound machine connected to the collection system host;

[0007] The pulse generator is used to send a synchronous pulse signal to the positioning unit and the acquisition system host simultaneously;

[0008] The ultrasound machine includes an ultrasound probe; the ultrasound probe is used to collect ultrasound images;

[0009] The positioning unit includes a sensor and an electromagnetic signal generator; the sensor is arranged on the ultrasonic probe; the electromagnetic signal generator is used to send electromagnetic waves to the sensor and receive echo signals of the electromagnetic waves; the positioning unit is used to locate the real-time posture data of the ultrasonic probe during movement according to the echo signals based on a motion tracking system matched with the positioning unit;

[0010] The acquisition system host is used to record the kernel timestamp of the synchronization pulse signal, and based on the kernel timestamp, synchronously acquire the ultrasound image output by the ultrasound probe and the posture signal output by the positioning unit; the posture signal is the real-time posture data of the ultrasound probe during the spatial movement.

[0011] In a second aspect, a method for synchronously collecting ultrasound images and posture data includes:

[0012] Using a pulse generator, a synchronous pulse signal is sent to a positioning unit and a host of a collection system at the same time; the positioning unit includes a sensor and an electromagnetic signal generator;

[0013] Using an electromagnetic signal generator to send electromagnetic waves to a sensor located on the ultrasonic probe, and receiving an echo signal of the electromagnetic wave;

[0014] Using the positioning unit, based on a motion tracking system matched with the positioning unit, positioning the real-time position data of the ultrasonic probe during the movement according to the echo signal;

[0015] The acquisition system host is used to record the kernel timestamp of the synchronization pulse signal, and based on the kernel timestamp, the ultrasound image output by the ultrasound probe and the posture signal output by the positioning unit are synchronously acquired; the posture signal is the real-time posture data of the ultrasound probe during the spatial movement.

[0016] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned method for synchronously acquiring ultrasound images and posture data.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for synchronously acquiring ultrasound images and posture data.

[0018] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned method for synchronously acquiring ultrasound images and posture data.

[0019] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0020] The present application utilizes a pulse generator to synchronize the positioning unit and the kernel timestamp of the acquisition system host through a synchronous pulse signal, and sets the sensor of the positioning unit on the ultrasonic probe. Based on the built-in motion tracking signal of the positioning unit, electromagnetic waves are sent to the sensor according to the electromagnetic signal generator to locate the real-time posture data of the ultrasonic probe during movement, so as to accurately acquire the posture changes of the probe during movement; finally, based on the synchronized kernel timestamp, the ultrasonic image and the posture signal can be matched one by one, thereby realizing the synchronous acquisition of the ultrasonic image output by the ultrasonic probe and the posture signal output by the positioning unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 This is a schematic diagram of a device for synchronously collecting ultrasound images and posture data in one embodiment of the present application;

[0023] Figure 2 A flowchart of a method for synchronously collecting ultrasound images and posture data provided in one embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0026] The present application provides a synchronous acquisition device for ultrasonic image and posture data, comprising: an acquisition system host 1, and a pulse generator 2, a positioning unit 3 and an ultrasonic machine 4 connected to the acquisition system host; the pulse generator 2 is used to send a synchronous pulse signal to the positioning unit 3 and the acquisition system host at the same time; the ultrasonic machine 4 includes an ultrasonic probe; the ultrasonic probe is used to acquire ultrasonic images; the positioning unit 3 includes a sensor and an electromagnetic signal generator; the sensor is arranged on the ultrasonic probe; the electromagnetic signal generator is used to send electromagnetic waves to the sensor and receive echo signals of the electromagnetic waves; the positioning unit 3 is used to locate the real-time posture data of the ultrasonic probe during movement according to the echo signal based on a motion tracking system matched with the positioning unit 3; the acquisition system host is used to record the kernel timestamp of the synchronous pulse signal, and based on the kernel timestamp, synchronously acquire the ultrasonic image output by the ultrasonic probe and the posture signal output by the positioning unit 3; the posture signal is the real-time posture data of the ultrasonic probe during spatial movement.

[0027] In an exemplary embodiment, the sensor is fixed to the ultrasound probe via a 3D printed mounting adapter.

[0028] In an exemplary embodiment, the ultrasound probe samples the orientation quaternion at a rate of 240 Hz.

[0029] In practical applications, in order to achieve synchronization between the probe position and the ultrasound video, it is necessary to first build a data acquisition system based on electromagnetic positioning. It is planned to develop a motion tracking system equipped with a VIPER4 positioning unit, which can be the positioning unit 3 produced by Polhemus Corporation in the United States.

[0030] The sensor of the positioning unit 3 is connected to the probe through a 3D-printed mounting adapter. The probe direction quaternion is sampled at a rate of 240 Hz, which can synchronously collect the image output by the ultrasound machine 4 and the real-time posture of the ultrasound probe in space; wherein, the direction quaternion is a method of representing rotation in three-dimensional space, and the three-dimensional rotation of the ultrasound image collected by the ultrasound probe relative to the reference position is collected in the form of the direction quaternion.

[0031] In an exemplary embodiment, the main framework of the video acquisition in the acquisition system host is an open source gstreamer framework; the open source gstreamer framework is used to add the kernel timestamp to each video frame of the ultrasound image.

[0032] The main difficulty in implementing synchronous acquisition lies in the time synchronization problem between real-time video acquisition and acquisition frames. This application will use the kernel clock of the Linux operating system as the reference clock and the open source gstreamer framework as the main framework for video acquisition. In order to obtain accurate video frame timestamps, the video source component of gstreamer is modified, and the kernel timestamp obtained from the video acquisition logic device provided by the Linux operating system is added to each video frame; wherein, the gstreamer framework is a video processing framework, which contains multiple components including video source, video processing, video encoding storage, etc.; the gstreamer video source component is used in the acquisition system host.

[0033] The steps to add kernel timestamp can be:

[0034] 1) Modify the process of gstreamer reading video data from the video device. On the basis of the original reading action, additionally read the kernel timestamp corresponding to the video frame.

[0035] 2) For each captured video frame, add additional metadata and save the acquired kernel timestamp in the additional metadata.

[0036] In an exemplary embodiment, the acquisition system host drives the kernel timestamp through the Linux kernel, and records the time corresponding to the posture signal through a GPIO interrupt response function; the Linux kernel is a Linux operating system.

[0037] In an exemplary embodiment, the acquisition system host further includes: a central processing unit (CPU) core; the CPU core runs the GPIO interrupt response function separately and in isolation through CPU core hard isolation technology.

[0038] In actual applications, for the time synchronization method, this application uses an external pulse generator 2 to simultaneously connect the generated synchronization pulse signal to the positioning unit 3 and the acquisition system host. In the positioning system, the kernel timestamp of the pulse signal is recorded through the Linux kernel driver. In the kernel driver, the timing of the external synchronization pulse is realized through the GPIO interrupt response function, thereby recording the time corresponding to each positioning data of the positioning unit 3.

[0039] Considering that the Linux operating system is a non-real-time operating system, in order to improve the speed of interrupt response and reduce the interference of the Linux system on pulse timing, the GPIO interrupt response function is run separately on the isolated CPU core through the hard isolation technology of the central processing unit (CPU) core to ensure the stability and consistency of timing; Among them, the CPU core is a unit of the central processing unit of the computer, which can execute the instructions of the computer program; a CPU includes the hardware components required to perform basic operations and logical operations, such as arithmetic logic unit (ALU), registers, control unit and cache. The CPU can execute instructions independently, and the multi-core CPU (including dual-core, quad-core, etc.) is composed of multiple such cores, which can process multiple tasks in parallel, thereby improving computing efficiency.

[0040] In addition to the motion tracking system, acquisition software that can interact with doctors is also needed. The acquisition software is installed on the acquisition system host and is planned to be developed using the GTK4 graphical interface framework.

[0041] The OpenGL rendering environment is embedded through the GLArea component of GTK4. Based on the OpenGL rendering environment, the three_d 3D graphics rendering library is used to implement simple 3D rendering and lighting and shadow effects, which helps doctors intuitively understand the current position of the ultrasound probe and the predicted position of the target section. The acquisition software can interact with the Linux kernel module through the netlink protocol to collect the position of the ultrasound probe in real time.

[0042] The acquisition software divides the ultrasound images output by the ultrasound machine 4 into two identical paths, one of which is used for encoding and storage, and the other is integrated with the GStreamer framework through the GUI component of GTK4 to realize the real-time display of the image acquisition results.

[0043] Due to the limited acquisition time, in order to improve the acquisition efficiency, doctors rotate the probe very fast in actual operation, which makes the data on the ultrasound scanning path sparse. At the same time, due to the random distribution of path points, the successfully acquired path points are usually close to the starting point, so the space scanned on the trajectory is small.

[0044] In order to solve the above problems, this application proposes to adopt a new acquisition scheme. The new scheme fixes the position from the path point to the starting point, and renders the virtual probe movement trajectory on the three-dimensional rendering interface according to the expected path scanning time, guiding the doctor to scan at the expected speed. At the same time, in order to improve the enthusiasm of the doctor, the doctor is informed to reduce the time spent at the path point, and at the same time, the prompt time for the probe to stay on the acquisition software is reduced, thereby shortening the time spent at the path point.

[0045] The present application also provides a method for synchronously collecting ultrasound images and posture data, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, Figure 2 As shown, the method includes the following steps.

[0046] S1: Using the pulse generator 2, a synchronous pulse signal is sent to the positioning unit 3 and the acquisition system host at the same time; the positioning unit 3 includes a sensor and an electromagnetic signal generator.

[0047] S2: Using an electromagnetic signal generator to send electromagnetic waves to a sensor located on the ultrasonic probe, and receiving an echo signal of the electromagnetic wave.

[0048] S3: Using the positioning unit 3 and based on a motion tracking system matched with the positioning unit 3, the real-time position data of the ultrasonic probe during the movement is positioned according to the echo signal.

[0049] S4: Using the acquisition system host, record the kernel timestamp of the synchronization pulse signal, and based on the kernel timestamp, synchronously acquire the ultrasound image output by the ultrasound probe and the posture signal output by the positioning unit 3; the posture signal is the real-time posture data of the ultrasound probe during the spatial movement.

[0050] This application solves the problem of no posture data in the process of various ultrasound image data collection, and realizes the synchronous collection of ultrasound image and posture data

[0051] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store ultrasound image and posture data synchronous acquisition data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for synchronous acquisition of ultrasound image and posture data is implemented.

[0052] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method is implemented when the processor executes the computer program.

[0053] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above method when executed by a processor.

[0054] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above method when executed by a processor.

[0055] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnlyMemory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (Magnetoresistive RandomAccess Memory, MRAM), ferroelectric random access memory (Ferroelectric RandomAccess Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (RandomAccess Memory, RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0056] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0057] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0058] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0059] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A device for synchronously collecting ultrasonic images and posture data, characterized in that: include: A collection system host, and a pulse generator, a positioning unit and an ultrasound machine connected to the collection system host; The pulse generator is used to send a synchronous pulse signal to the positioning unit and the acquisition system host simultaneously; The ultrasound machine includes an ultrasound probe; the ultrasound probe is used to collect ultrasound images; The positioning unit includes a sensor and an electromagnetic signal generator; the sensor is arranged on the ultrasonic probe; the electromagnetic signal generator is used to send electromagnetic waves to the sensor and receive echo signals of the electromagnetic waves; the positioning unit is used to locate the real-time posture data of the ultrasonic probe during movement according to the echo signals based on a motion tracking system matched with the positioning unit; The acquisition system host is used to record the kernel timestamp of the synchronization pulse signal, and based on the kernel timestamp, synchronously acquire the ultrasound image output by the ultrasound probe and the posture signal output by the positioning unit; the posture signal is the real-time posture data of the ultrasound probe during the spatial movement.

2. The ultrasonic image and posture data synchronous acquisition device according to claim 1, characterized in that: The sensor is fixed to the ultrasound probe via a 3D printed mounting adapter.

3. The ultrasonic image and posture data synchronous acquisition device according to claim 1, characterized in that: The ultrasound probe samples the orientation quaternion at a rate of 240 Hz.

4. The ultrasonic image and posture data synchronous acquisition device according to claim 1, characterized in that: The main framework of the video acquisition in the acquisition system host is the open source gstreamer framework; the open source gstreamer framework is used to add the kernel timestamp to each video frame of the ultrasound image.

5. The ultrasonic image and posture data synchronous acquisition device according to claim 1, characterized in that: The acquisition system host drives the kernel timestamp through the Linux kernel, and records the time corresponding to the posture signal through the GPIO interrupt response function; the Linux kernel is the Linux operating system.

6. The ultrasonic image and posture data synchronous acquisition device according to claim 5, characterized in that: The acquisition system host also includes: a CPU core; The CPU core runs the GPIO interrupt response function separately and in isolation through the CPU core hard isolation technology.

7. A method for synchronously collecting ultrasonic images and posture data, characterized in that: The method for synchronously collecting ultrasonic images and posture data comprises: Using a pulse generator, a synchronous pulse signal is sent to a positioning unit and a host of a collection system at the same time; the positioning unit includes a sensor and an electromagnetic signal generator; Using an electromagnetic signal generator to send electromagnetic waves to a sensor located on the ultrasonic probe, and receiving an echo signal of the electromagnetic wave; Using the positioning unit, based on a motion tracking system matched with the positioning unit, positioning the real-time position data of the ultrasonic probe during the movement according to the echo signal; The acquisition system host is used to record the kernel timestamp of the synchronization pulse signal, and based on the kernel timestamp, the ultrasound image output by the ultrasound probe and the posture signal output by the positioning unit are synchronously acquired; the posture signal is the real-time posture data of the ultrasound probe during the spatial movement.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for synchronously acquiring ultrasound images and posture data as described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for synchronously collecting ultrasonic images and posture data described in claim 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for synchronously collecting ultrasonic images and posture data described in claim 7 is implemented.

Images