An ultrasonic image and pose data synchronous acquisition method, device, equipment, medium and product

Abstract

The application discloses an ultrasonic image and pose data synchronous acquisition method and device, equipment, medium and product, relates to the ultrasonic image positioning field, and the device comprises an acquisition system host, a pulse generator, a positioning unit and an ultrasonic machine; the pulse generator simultaneously sends a synchronization pulse signal to the positioning unit and the acquisition system host; the ultrasonic probe acquires ultrasonic images; the positioning unit comprises a sensor and an electromagnetic signal generator; the sensor is arranged on the ultrasonic probe; the electromagnetic signal generator sends electromagnetic waves to the sensor and receives echo signals of the electromagnetic waves; the positioning unit is based on a motion tracking system, and the real-time pose data of the ultrasonic probe in the moving process is positioned according to the echo signals; the acquisition system host records the kernel time stamp of the synchronization pulse signal, synchronously acquires the ultrasonic images output by the ultrasonic probe and the pose signals output by the positioning unit, the application can accurately acquire the pose change in the moving process of the probe, and synchronously acquires the ultrasonic images and the pose signals.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of ultrasound image positioning, and particularly relates to an ultrasound image and pose data synchronous acquisition method, device, equipment, medium and product. BACKGROUND

[0002] In the medical field, ultrasound image acquisition is an effective way to observe human biology and disease development. In the field of cardiac ultrasound, a cardiac ultrasound physician needs to make small probe movements on each section during ultrasound detection, and position the pose change during the movement. However, the existing camera-based computer video positioning algorithm needs to set up multiple cameras and use image feature matching algorithm to extract corresponding points in the image, which has low matching accuracy and may also cause positioning failure due to camera obstruction. Therefore, during high-precision and small-range movement, the real-time pose data of the ultrasound probe during movement cannot be accurately acquired, which leads to the failure of synchronous acquisition of ultrasound images and pose data. SUMMARY

[0003] The present application aims to provide an ultrasound image and pose data synchronous acquisition method, device, equipment, medium and product to solve the problem that the real-time pose data of the ultrasound probe during movement cannot be accurately acquired, which leads to the failure of synchronous acquisition of ultrasound images and pose data.

[0004] To achieve the above-mentioned purpose, the present application provides the following solutions.

[0005] In a first aspect, the present application provides an ultrasound image and pose data synchronous acquisition device, comprising:

[0006] an acquisition system host, a pulse generator, a positioning unit and an ultrasound machine connected to the acquisition system host;

[0007] The pulse generator is configured to send a synchronization pulse signal to the positioning unit and the acquisition system host simultaneously.

[0008] The ultrasound machine comprises an ultrasound probe, and the ultrasound probe is configured to acquire an ultrasound image.

[0009] The positioning unit comprises a sensor and an electromagnetic signal generator, the sensor is arranged on the ultrasound probe, the electromagnetic signal generator is configured to send an electromagnetic wave to the sensor and receive a return signal of the electromagnetic wave, and the positioning unit is configured to position real-time pose data of the ultrasound probe during movement based on a motion tracking system matched with the positioning unit according to the return signal.

[0010] The acquisition system host is configured to record a kernel timestamp of the synchronization pulse signal, and based on the kernel timestamp, synchronously acquire an ultrasound image output by the ultrasound probe and a pose signal output by the positioning unit; the pose signal is real-time pose data of the ultrasound probe during spatial movement.

[0011] In a second aspect, an ultrasound image and pose data synchronous acquisition method includes:

[0012] A pulse generator is used to simultaneously send a synchronization pulse signal to a positioning unit and an acquisition system host; the positioning unit includes a sensor and an electromagnetic signal generator.

[0013] The electromagnetic signal generator is used to send an electromagnetic wave to the sensor on the ultrasound probe, and receive a return signal of the electromagnetic wave.

[0014] The positioning unit is used to locate real-time pose data of the ultrasound probe during movement based on a motion tracking system matched with the positioning unit according to the return signal.

[0015] The acquisition system host is configured to record a kernel timestamp of the synchronization pulse signal, and based on the kernel timestamp, synchronously acquire an ultrasound image output by the ultrasound probe and a pose signal output by the positioning unit; the pose signal is real-time pose data of the ultrasound probe during spatial movement.

[0016] In a third aspect, the present application provides a computer device, including a memory, a processor, a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the above-mentioned ultrasound image and pose data synchronous acquisition method.

[0017] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above-mentioned ultrasound image and pose data synchronous acquisition method.

[0018] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the above-mentioned ultrasound image and pose data synchronous acquisition method.

[0019] According to the embodiments provided in the present application, the following technical effects are disclosed:

[0020] The application utilizes a pulse generator to synchronize the kernel time stamp of a positioning unit and a collection system host through a synchronous pulse signal, sets a sensor of the positioning unit on an ultrasonic probe, and transmits an electromagnetic wave to the sensor according to an electromagnetic signal generator based on a motion tracking signal built in the positioning unit, so as to locate real-time pose data of the ultrasonic probe in a moving process, thereby accurately collecting the pose change in the moving process of the probe. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only represent some of the embodiments of the present application, and other drawings can be obtained by those of ordinary skill in the art without any creative effort based on these drawings.

[0022] Figure 1 FIG. 1 is a schematic diagram of an ultrasonic image and pose data synchronous collection device according to an embodiment of the present application.

[0023] Figure 2 FIG. 2 is a flowchart of an ultrasonic image and pose data synchronous collection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only represent some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort fall within the scope of protection of the present application.

[0025] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0026] The application provides an ultrasound image and pose data synchronous acquisition device, which comprises an acquisition system host 1, a pulse generator 2, a positioning unit 3 and an ultrasound machine 4 connected with the acquisition system host; the pulse generator 2 is used for sending a synchronous pulse signal to the positioning unit 3 and the acquisition system host; the ultrasound machine 4 comprises an ultrasound probe; the ultrasound probe is used for acquiring an ultrasound image; the positioning unit 3 comprises a sensor and an electromagnetic signal generator; the sensor is arranged on the ultrasound probe; the electromagnetic signal generator is used for sending an electromagnetic wave to the sensor and receiving an echo signal of the electromagnetic wave; the positioning unit 3 is used for positioning real-time pose data of the ultrasound probe in a moving process according to the echo signal based on a motion tracking system matched with the positioning unit 3; the acquisition system host is used for recording a kernel timestamp of the synchronous pulse signal and synchronously acquiring the ultrasound image output by the ultrasound probe and the pose signal output by the positioning unit 3 based on the kernel timestamp; the pose signal is real-time pose data of the ultrasound probe in a spatial moving process.

[0027] In an exemplary embodiment, the sensor is fixed on the ultrasound probe through a 3D printed mounting adapter.

[0028] In an exemplary embodiment, the ultrasound probe samples the direction quaternion at a rate of 240 Hz.

[0029] In actual application, in order to realize the synchronization of the probe pose and the ultrasound video, it is necessary to first build an electromagnetic positioning-based data acquisition system, and a motion tracking system equipped with a VIPER4 positioning unit is developed, which can be a positioning unit 3 produced by the American Polhemus company.

[0030] The sensor of the positioning unit 3 is connected to the probe through a 3D printed mounting adapter, the probe direction quaternion is sampled at a rate of 240 Hz, and the ultrasound machine 4 output image and the real-time attitude of the ultrasound probe in space can be synchronously acquired; wherein the direction quaternion is a representation method of rotation in three-dimensional space, and the three-dimensional rotation of the ultrasound image acquired by the ultrasound probe relative to the reference position is acquired in the form of the direction quaternion.

[0031] In an exemplary embodiment, the main framework of the video acquisition in the acquisition system host is an open source gstreamer framework; the open source gstreamer framework is used for adding the kernel timestamp to each video frame of the ultrasound image.

[0032] The main difficulty in synchronous acquisition is the time synchronization between real-time video acquisition and acquisition frames. The kernel clock of the Linux operating system is used as the reference clock, and the open-source gstreamer framework is used as the main framework for video acquisition. In order to obtain accurate video frame timestamps, the video source component of gstreamer is modified, and the kernel timestamp obtained from the video acquisition logic device provided by the Linux operating system is added to each video frame. The gstreamer framework is a video processing framework that includes video source, video processing, video encoding and storage components. The gstreamer video source component is used in the acquisition system host.

[0033] The step of adding the kernel timestamp can be:

[0034] 1) Modify the gstreamer video data reading flow from the video device. On the basis of the original reading action, additional kernel timestamps corresponding to the video frame are read.

[0035] 2) Add additional metadata to each video frame acquired. The obtained kernel timestamp is saved in the additional metadata.

[0036] In an exemplary embodiment, the acquisition system host drives the kernel timestamp through the Linux kernel, and records the time corresponding to the pose signal through a GPIO interrupt response function. The Linux kernel is a Linux operating system.

[0037] In an exemplary embodiment, the acquisition system host further includes a central processing unit (CPU) core. The CPU core is separately isolated by CPU core hard isolation technology to run the GPIO interrupt response function.

[0038] In practical applications, for the time synchronization method, an external pulse generator 2 is used to simultaneously connect the synchronous pulse signal to the positioning unit 3 and the acquisition system host. In the positioning system, the kernel timestamp of the pulse signal is recorded by the Linux kernel driver. In the kernel driver, the timing of the external synchronization pulse is realized by the GPIO interrupt response function, thereby recording the time corresponding to each positioning data of the positioning unit 3.

[0039] Considering that the Linux operating system is a non-real-time operating system, in order to improve the speed of interrupt response and reduce the interference of the Linux system on pulse timing, the GPIO interrupt response function is separately placed on an isolated CPU core for running through the central processing unit (CPU) core hard isolation technology, so as to ensure the stability and consistency of timing. The CPU core is a unit of the central processing unit of a computer, which can execute instructions of a computer program. A CPU includes hardware components required for basic operation and logic operation, such as an arithmetic logic unit (ALU), a register, a control unit and a cache. The CPU can independently execute instructions, and a multi-core CPU (including dual-core, quad-core, etc.) is composed of multiple such cores, which can handle multiple tasks in parallel, thereby improving computing efficiency.

[0040] On the motion tracking system, a collection software capable of interacting with doctors is also needed, which is arranged on a collection system host. The collection software is developed by using a GTK4 graphical interface framework.

[0041] The embedding of an OpenGL rendering environment is realized through a GLArea component of GTK4. On the basis of the OpenGL rendering environment, a three-dimensional graphics rendering library three_d is used to realize simple three-dimensional rendering and shading effects, so as to facilitate doctors to intuitively understand the current pose of an ultrasonic probe and the pose of a predicted target section. The collection software can interact between the Linux kernel module and the collection software through a netlink protocol, and collect the pose of the ultrasonic probe in real time.

[0042] The collection software divides the ultrasonic image output by an ultrasonic machine 4 into two identical paths. One path of image is used for encoding and storage, and the other path is integrated with a GStreamer framework through a GUI component of GTK4, so as to realize real-time display of the image collection result.

[0043] Due to the limited collection time, in order to improve the collection efficiency, the speed of the probe rotation in the actual operation of doctors is very fast, which makes the data on the ultrasonic scanning path sparse. At the same time, due to the random distribution of the path points, the successfully collected path points are usually close to the starting point, so the space scanned on the trajectory is small.

[0044] In order to solve the above problems, the present application proposes a new collection scheme, which fixes the position of the path point to the starting point, and renders a virtual probe motion trajectory on a three-dimensional rendering interface according to the expected path scanning time, so as to guide doctors to scan at the expected speed. In order to improve the enthusiasm of doctors, the doctors are informed to reduce the time of staying at the path point, and the prompt time of the probe staying on the collection software is reduced, so as to shorten the time of staying at the path point.

[0045] The application further provides an ultrasound image and pose data synchronous acquisition method, which is executed by a computer device, specifically, can be executed by a terminal or a server or the like computer device alone, or can be executed by the terminal and the server together, and in the embodiment of the application, as shown in Figure 2 the method comprises the following steps.

[0046] S1: a pulse generator 2 is used to simultaneously send a synchronization pulse signal to a positioning unit 3 and an acquisition system host; the positioning unit 3 comprises a sensor and an electromagnetic signal generator.

[0047] S2: an electromagnetic signal generator is used to send an electromagnetic wave to the sensor located on an ultrasound probe and receive a return signal of the electromagnetic wave.

[0048] S3: the positioning unit 3 is used to position real-time pose data of the ultrasound probe in a moving process according to the return signal based on a motion tracking system matched with the positioning unit 3.

[0049] S4: the acquisition system host is used to record a kernel timestamp of the synchronization pulse signal, and based on the kernel timestamp, to synchronously acquire an ultrasound image output by the ultrasound probe and a pose signal output by the positioning unit 3; the pose signal is real-time pose data of the ultrasound probe in a spatial moving process.

[0050] The application solves the problem that there is no pose data in various ultrasound image data acquisition processes, and realizes synchronous acquisition of ultrasound image and pose data

[0051] In an exemplary embodiment, a computer device is provided, which can be a server or a terminal. The computer device comprises a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is used to store ultrasound image and pose data synchronous acquisition data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement an ultrasound image and pose data synchronous acquisition method.

[0052] In an exemplary embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0053] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the above method when executed by a processor.

[0054] In an exemplary embodiment, a computer program product is provided, comprising a computer program, and the computer program implementing the above method when executed by a processor.

[0055] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnly Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0056] In the present application, all actions of obtaining signals, information or data are performed in compliance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization of the owner of the corresponding device.

[0057] The database involved in each of the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, and the like, without being limited thereto. The processor involved in each of the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like, without being limited thereto.

[0058] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but it should be considered that any combination of the technical features is within the scope of the present disclosure, as long as there is no contradiction.

[0059] The principles and implementation manners of the present application are described by using specific examples herein, and the above embodiments are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A device for synchronous acquisition of ultrasound images and pose data, characterized in that, include: The acquisition system host, and a pulse generator, a positioning unit, and an ultrasonic machine connected to the acquisition system host; The pulse generator is used to simultaneously send synchronous pulse signals to the positioning unit and the acquisition system host. The ultrasound machine includes an ultrasound probe; the ultrasound probe is used to acquire ultrasound images. The positioning unit includes a sensor and an electromagnetic signal generator; the sensor is mounted on the ultrasonic probe; the electromagnetic signal generator is used to send electromagnetic waves to the sensor and receive the echo signals of the electromagnetic waves; the positioning unit is used to locate the real-time pose data of the ultrasonic probe during movement based on the echo signals using a motion tracking system matched with the positioning unit. The acquisition system host is used to record the kernel timestamp of the synchronization pulse signal, and based on the kernel timestamp, synchronously acquire the ultrasound image output by the ultrasound probe and the pose signal output by the positioning unit; the pose signal is the real-time pose data of the ultrasound probe during spatial movement. The main framework for video acquisition in the acquisition system host is the open-source GStreamer framework; the open-source GStreamer framework is used to add the kernel timestamp to each video frame of the ultrasound image. The kernel clock of the Linux operating system will be used as the reference clock, and the open-source gstreamer framework will be used as the main framework for video capture. The video source component of gstreamer will be modified to add the kernel timestamp obtained from the video capture logic device provided by the Linux operating system to each video frame; the gstreamer video source component will be used in the capture system host. The steps to add a kernel timestamp are as follows: 1) Modify the process of gstreamer reading video data from the video device, and in addition to the original reading action, read the kernel timestamp corresponding to the video frame; 2) Add additional metadata to each video frame acquired; save the acquired kernel timestamp in the additional metadata; The host of the acquisition system drives the kernel timestamp through the Linux kernel and records the time corresponding to the pose signal through the GPIO interrupt response function; The Linux kernel is the Linux operating system; The host of the acquisition system also includes a CPU core; the CPU core runs the GPIO interrupt response function in isolation through CPU core hard isolation technology; For the time synchronization method, an external pulse generator is used to simultaneously connect the generated synchronization pulse signal to the positioning unit and the acquisition system host. In the positioning system, the kernel timestamp of the pulse signal is recorded through the Linux kernel driver. In the kernel driver, the timing of the external synchronization pulse is implemented through the GPIO interrupt response function, and the time corresponding to each positioning data of the positioning unit is recorded. By using CPU core hard isolation technology, the GPIO interrupt response function is run separately on an isolated CPU core, ensuring the stability and consistency of timing; where the CPU core is a unit of the computer's central processing unit, capable of executing computer program instructions.

2. The ultrasonic image and pose data synchronous acquisition device according to claim 1, characterized in that, The sensor is fixed to the ultrasonic probe via a 3D-printed mounting adapter.

3. The ultrasound image and pose data synchronous acquisition device according to claim 1, characterized in that, The ultrasonic probe samples directional quaternions at a rate of 240 Hz.

4. A method for synchronous acquisition of ultrasound images and pose data, characterized in that, The method for synchronous acquisition of ultrasound images and pose data is applied to the ultrasound image and pose data synchronous acquisition device according to any one of claims 1-3, wherein the ultrasound image and pose data synchronous acquisition method includes: A pulse generator is used to simultaneously send synchronous pulse signals to the positioning unit and the acquisition system host; the positioning unit includes a sensor and an electromagnetic signal generator; An electromagnetic signal generator is used to send electromagnetic waves to a sensor located on an ultrasonic probe, and the echo signal of the electromagnetic waves is received. Using the positioning unit, and based on a motion tracking system matched with the positioning unit, the real-time pose data of the ultrasonic probe during its movement is located according to the echo signal; Using the host of the acquisition system, the kernel timestamp of the synchronization pulse signal is recorded, and based on the kernel timestamp, the ultrasound image output by the ultrasound probe and the pose signal output by the positioning unit are acquired synchronously; the pose signal is the real-time pose data of the ultrasound probe during spatial movement.

5. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method for synchronous acquisition of ultrasound images and pose data as described in claim 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for synchronous acquisition of ultrasound images and pose data as described in claim 4.

7. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the method for synchronous acquisition of ultrasound images and pose data as described in claim 4.

Images