Image synchronization cache display method, system, device and storage medium

Through asynchronous acquisition and synchronous cache technology, the problem that ultrasonic endoscopy and electronic endoscopy images cannot be played back simultaneously is solved, and the image synchronous display and data synchronization under different frame rate conditions are realized, which improves data loading efficiency.

CN114079753BActive Publication Date: 2024-12-03INNERMEDICAL CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111327559.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-10
Publication Date
2024-12-03
Estimated Expiration
2041-11-10

AI Technical Summary

Technical Problem

In ultrasound endoscopy, ultrasound endoscopic images and electronic endoscopic images cannot be played back simultaneously, which causes doctors to operate separately when analyzing and storing pictures, resulting in difficulty in finding data and image deviation problems.

Method used

Using asynchronous acquisition and real-time refresh image synchronization cache display method, images are collected asynchronously through the acquisition subthreads of each device, and image data is stored in the single-frame cache area and queue cache area to realize synchronous cache and real-time refresh of images, ensuring that Synchronous display of images under different frame rate conditions.

Benefits of technology

Synchronous playback display between devices with different image refresh frame rates is realized, which solves the problem of data search and image deviation, ensures data synchronization and real-timeness, and improves data loading efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114079753B_ABST
    Figure CN114079753B_ABST
Patent Text Reader

Abstract

The present invention provides a method, system, device and storage medium for synchronous image caching display, the method comprising: displaying the images of each device in real time; using each acquisition subthread to asynchronously acquire the images of each device respectively, and in each acquisition subthread, opening up a single frame buffer area and a queue buffer area respectively, wherein the single frame buffer area stores the latest image data; after each acquisition subthread completes a single acquisition, each device triggers the interface refresh of the corresponding image; using one of the devices completing a single acquisition action as a synchronization signal source, the latest image stored in the single frame buffer area of ​​each device is transferred to the respective queue buffer area, and the content of each queue buffer area maintains frame synchronization, thereby realizing image synchronous caching. The technical solution of the present invention enables the images of each device to be refreshed and displayed in real time at the maximum frame rate under the condition of different sampling frame rates, and can be played back and displayed synchronously.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to an image synchronization cache display method, system, device and storage medium. Background Art

[0002] In ultrasound endoscopy, due to space limitations, ultrasound endoscopes and electronic endoscopes need to share a display device. Usually, electronic endoscope video data is connected to the ultrasound endoscope system through a video capture card, and then connected from the ultrasound endoscope system to the display, so that ultrasound images and electronic endoscope images can be displayed simultaneously.

[0003] In existing endoscopic ultrasound examination scenarios, endoscopic ultrasound images and electronic endoscopic images can be displayed synchronously during real-time scanning. However, the two cannot be played back synchronously. The doctor needs to analyze the lesion area under the electronic endoscope and save the image; then scan the lesion area under the ultrasound endoscope, play it back, analyze, and save the image again; finally, when editing the report, the doctor needs to select the electronic endoscopic image and the endoscopic ultrasound image separately to make a medical record report. Since the electronic endoscope and ultrasound images are stored in two operations in succession, it is difficult to find the corresponding data, and there is an image deviation problem between the two images. Summary of the invention

[0004] In response to the above technical problems, the present invention discloses an image synchronous caching display method, system, device and storage medium, asynchronous acquisition and real-time refresh mode, so that the images of each device can be refreshed and displayed in real time at the maximum frame rate under conditions of different sampling frame rates, and can be played back and displayed synchronously.

[0005] To this end, the technical solution adopted by the present invention is:

[0006] An image synchronization cache display method, comprising:

[0007] Display the images of each device in real time;

[0008] Each device corresponds to a collection sub-thread, and each collection sub-thread is used to asynchronously collect images of each device. In each collection sub-thread, a single-frame buffer area and a queue buffer area are opened to store image data, wherein the single-frame buffer area stores the latest image data;

[0009] After each acquisition sub-thread completes a single acquisition, each device triggers the interface refresh of the corresponding image;

[0010] With one of the devices completing a single acquisition action as the synchronization signal source, the latest image stored in each device's single-frame buffer is transferred to its own queue buffer. The contents of each queue buffer maintain frame synchronization, thus achieving image synchronization caching.

[0011] When freezing playback, the image of the corresponding frame in each queue buffer is obtained according to the current playback frame, and the interface synchronously displays the image of each device.

[0012] The devices may be two devices, or three, four or more devices.

[0013] This technical solution is used to asynchronously collect data from multiple devices with different image refresh frame rates. When freezing the playback, the synchronized image can be obtained for display, realizing synchronized caching and real-time display. In addition, when saving images, editing reports, and exporting data, the images of each synchronized device can be directly used, which solves the problem of difficulty in finding the data corresponding to each device in the prior art, avoids the problem of inconsistent frame numbers caused by different acquisition frame rates, asynchronous acquisition, etc., and ensures data synchronization. Moreover, each acquisition sub-thread performs data acquisition separately without affecting the main thread.

[0014] Further preferably, the image refresh rates of the various devices are different. Of course, the above method can also be used if the image refresh rates of the various devices are the same, but if the image refresh rates of the various devices are the same, there are other methods in the prior art that can be implemented.

[0015] As a further improvement of the present invention, the queue buffer is a first-in-first-out queue buffer. Further, the queue buffer is set with a maximum number of cache frames. When the number of cache frames exceeds the set maximum number of cache frames, the image data that first enters the cache is removed, and then the new image data is placed in the first-in-first-out cache. Further, the maximum number of cache frames is 300 frames.

[0016] As a further improvement of the present invention, the image synchronization cache display method further includes: storing the matched images of each device as an image file. With this technical solution, the images of each synchronized device can be directly used when editing reports and exporting data. As a further improvement of the present invention, each device is an ultrasonic endoscope and an electronic endoscope, and the action of collecting ultrasonic endoscopic images is used as a synchronization signal source.

[0017] The present invention also discloses an image synchronization cache display system, which comprises:

[0018] A display module displays images of various devices in real time; the image refresh rates of the various devices are different;

[0019] In the acquisition module, each device corresponds to a collection sub-thread, and each collection sub-thread is used to asynchronously collect images of each device. In each collection sub-thread, a single-frame buffer area and a queue buffer area are opened to store image data, wherein the single-frame buffer area stores the latest image data;

[0020] Interface refresh module: after each acquisition sub-thread completes a single acquisition, each device triggers the interface refresh of the corresponding image;

[0021] The transfer module uses one of the devices completing a single acquisition action as the synchronization signal source, and transfers the latest image stored in each device's single-frame buffer to its own queue buffer. The content of each queue buffer maintains frame synchronization, thus realizing image synchronization caching.

[0022] The synchronous display module obtains the images corresponding to each queue buffer area according to the current playback frame during frozen playback, and the interface synchronously displays the images of each device.

[0023] As a further improvement of the present invention, the queue buffer is a first-in-first-out queue buffer; the queue buffer sets a maximum number of cache frames. When the number of cache frames exceeds the set maximum number of cache frames, the image data that first enters the cache is removed, and then the new image data is placed in the first-in-first-out cache.

[0024] As a further improvement of the present invention, the image synchronization cache display system further includes:

[0025] The matching storage module stores the images of each matched device as image files.

[0026] As a further improvement of the present invention, the various devices are ultrasonic endoscopes and electronic endoscopes, and the action of collecting ultrasonic endoscopic images is used as a synchronization signal source.

[0027] The present invention also discloses an image synchronization cache display device, which comprises a connected processor and a memory, wherein the processor is used to execute a computer program stored in the memory to execute any one of the above-mentioned image synchronization cache display methods.

[0028] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program. When the computer program is run on a computer, the method for synchronously caching and displaying an image as described in any one of the above items is executed.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] By adopting the technical solution of the present invention, the images of each device can be refreshed and displayed in real time at the maximum frame rate under the condition of different sampling frame rates; multiple sub-threads can collect data asynchronously without affecting other operations of the user; in the frozen playback state, the images of each matched device can be played back and displayed synchronously; the matched images can be stored as image files, and the synchronized images of each device can be quickly loaded, which reduces the time required for browsing data and improves efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a basic flow chart of image data processing of an image synchronization cache display method according to embodiment 1 of the present invention.

[0032] Figure 2 This is the basic process in the freeze playback of an image synchronous cache display method in embodiment 1 of the present invention.

[0033] Figure 3 This is a basic flow chart of image data processing in an image synchronization cache display method according to embodiment 2 of the present invention. DETAILED DESCRIPTION

[0034] The preferred embodiments of the present invention are described in further detail below.

[0035] Example 1

[0036] In this embodiment, the image processing of ultrasonic endoscope and electronic endoscope is taken as an example for description.

[0037] Ultrasonic endoscopy achieves ultrasonic imaging by transmitting and receiving returned ultrasonic signals through an ultrasonic transducer, and then subjecting the data to a series of algorithmic processing. The refresh frame rate of ultrasonic endoscopy images is approximately 10 frames per second. Electronic endoscopy images are acquired through a video capture card. According to user needs, the acquired electronic endoscopy images will also be cropped for the second time to achieve the best display effect. The refresh frame rate of electronic endoscopy images can reach 60 frames per second. Therefore, there are great differences in frame rate and data volume between ultrasonic endoscopy images and electronic endoscopy images. The method of the following embodiment performs asynchronous acquisition on the two images with different frame rates and data volumes mentioned above, and realizes synchronous caching and real-time display. Figure 1 and Figure 2 As shown, the specific steps include:

[0038] (1) Display ultrasonic endoscopic images and electronic endoscopic images with different frame rates in real time.

[0039] (2) Use the acquisition mode of two sub-threads to asynchronously acquire ultrasound endoscopic images and electronic endoscopic images.

[0040] (3) The ultrasonic endoscopic image and the electronic endoscopic image each trigger a refresh response without interfering with each other. There is no waiting logic between threads, so that the ultrasonic endoscopic image can reach a maximum refresh frame rate of 10 frames, and the electronic endoscopic image can reach a maximum refresh frame rate of 60 frames.

[0041] (4) Create a single-frame buffer and a first-in-first-out queue buffer to cache data.

[0042] (5) The action of acquiring the ultrasonic endoscopic image is used as the synchronization signal source to synchronously cache the asynchronously acquired ultrasonic endoscopic images and electronic endoscopic images, and transfer them from the single-frame buffer area to the first-in-first-out buffer area.

[0043] (6) A first-in-first-out buffer area is opened separately to cache the collected ultrasonic endoscopic images and electronic endoscopic images, and a maximum number of cache frames is set. When the number of cache frames exceeds the set maximum number of cache frames, the image data that first enters the cache area is removed, and then the new image data is placed in the first-in-first-out buffer area to ensure that the data does not overflow.

[0044] (7) During frozen playback, synchronized ultrasound endoscopic images and electronic endoscopic images are acquired and displayed from the first-in-first-out buffer.

[0045] (8) The matched ultrasound endoscopic image and electronic endoscopic image are stored as image files.

[0046] In the above steps, when ultrasound endoscopic images and electronic endoscopic images are collected, the refresh frame rates of the two are different, so each opens a sub-thread and uses asynchronous collection; a double buffer is opened to ensure that real-time display and frozen playback do not affect each other. The single-frame buffer is used to store the latest image, and the queue buffer is used to store synchronized images. When the number of caches in the queue buffer is greater than the threshold (for example, 300), a first-in-first-out mechanism will be used to remove the image data that first enters the buffer, and then put the new image data into the first-in-first-out buffer to ensure that the images in the queue buffer are the latest and the data will not overflow.

[0047] In an independent acquisition thread, a single-frame buffer is opened to store the latest image data, and the image data is written to the asynchronous single-frame image buffer. After the two acquisition sub-threads complete a single acquisition, they each trigger the interface refresh of the corresponding image without interfering with each other. There is no waiting logic between threads, so that the ultrasonic endoscopic image can reach a maximum refresh frame rate of 10 frames, and the electronic endoscopic image can reach a maximum refresh frame rate of 60 frames. At the same time, the acquisition of the ultrasonic endoscopic image is used as the synchronization signal source to transfer the latest images of the ultrasonic image and the electronic endoscopic image stored in the single-frame buffer to their respective queue buffers. The queue buffer can cache up to 300 frames of images and is designed as a first-in-first-out type, so that the actual contents of the two queue buffers maintain frame synchronization, avoiding the problem of inconsistent frame numbers caused by different acquisition frame rates and asynchronous acquisition, and ensuring data synchronization.

[0048] When entering frozen playback, the system calculates the current playback frame, and then fetches the corresponding images from two first-in-first-out buffers, and the interface will display the two images synchronously. During real-time acquisition, the system ensures the synchronization of the queue buffers, so in frozen playback, the ultrasound endoscopic images and electronic endoscopic images are also matched one by one. In addition, the matched ultrasound endoscopic images and electronic endoscopic images are stored as image files. When saving images, editing reports, and exporting data, the synchronized ultrasound endoscopic images and electronic endoscopic images can be used directly. The synchronized ultrasound endoscopic images and electronic endoscopic images can be quickly loaded, which reduces the time required to review data and improves efficiency.

[0049] This embodiment also discloses an image synchronization cache display system, which includes:

[0050] A display module, which displays the images of the ultrasonic endoscope and the electronic endoscope in real time;

[0051] In the acquisition module, the ultrasonic endoscope and the electronic endoscope each correspond to an acquisition sub-thread, and two acquisition sub-threads are used to asynchronously acquire images of the ultrasonic endoscope and the electronic endoscope respectively. In the two acquisition sub-threads, a single-frame buffer area and a queue buffer area are respectively opened to store image data, wherein the single-frame buffer area stores the latest image data; the queue buffer area is a first-in-first-out queue buffer area; the queue buffer area sets a maximum cache frame number. When the cache frame number exceeds the set maximum cache frame number, the image data that first enters the cache area is removed, and then the new image data is placed in the first-in-first-out cache area.

[0052] Interface refresh module: after each acquisition sub-thread completes a single acquisition, each device triggers the interface refresh of the corresponding image.

[0053] The transfer module uses one of the devices completing a single acquisition action as the synchronization signal source, and transfers the latest image stored in each device's single frame buffer to their respective queue buffers. The contents of each queue buffer maintain frame synchronization, thus realizing image synchronization caching.

[0054] The synchronous display module obtains the images corresponding to each queue buffer area according to the current playback frame during frozen playback, and the interface synchronously displays the images of each device.

[0055] The matching storage module stores the images of each matched device as image files.

[0056] The devices are an ultrasonic endoscope and an electronic endoscope, and the action of collecting ultrasonic endoscopic images is used as a synchronization signal source.

[0057] An embodiment of the present invention further discloses an image synchronization cache display device, which includes a connected processor and a memory, wherein the processor is used to execute a computer program stored in the memory to execute any one of the above-described image synchronization cache display methods.

[0058] The present invention also discloses a computer-readable storage medium in an embodiment, wherein the computer-readable storage medium includes a computer program. When the computer program is run on a computer, the method for synchronously caching and displaying an image as described above is executed.

[0059] Example 2

[0060] Based on Example 1, the device can be expanded to 3 or more devices for synchronous image caching and display. For example, one ultrasound endoscope and two electronic endoscopes can be used for asynchronous acquisition to achieve synchronous image caching and real-time display. The processing flow chart is as follows Figure 3 As shown, the specific steps include:

[0061] (1) Display the ultrasound endoscope images and the images of two electronic endoscopes in real time.

[0062] (2) Use the acquisition mode of three sub-threads to asynchronously acquire ultrasound endoscopic images and images of two electronic endoscopes.

[0063] (3) The ultrasound endoscopic image and the two electronic endoscopic images each trigger a refresh response without interfering with each other. There is no waiting logic between threads, so that the ultrasound endoscopic image can reach a maximum refresh frame rate of 10 frames, and the electronic endoscopic image can reach a maximum refresh frame rate of 60 frames.

[0064] (4) Create a single-frame buffer and a first-in-first-out queue buffer to cache data.

[0065] (5) The action of acquiring the ultrasonic endoscopic image is used as the synchronization signal source to synchronously cache the asynchronously acquired ultrasonic endoscopic images and electronic endoscopic images, and transfer them from the single-frame buffer area to the first-in-first-out buffer area.

[0066] (6) A first-in-first-out buffer area is opened separately to cache the collected ultrasonic endoscopic images and electronic endoscopic images, and a maximum number of cache frames is set. When the number of cache frames exceeds the set maximum number of cache frames, the image data that first enters the cache area is removed, and then the new image data is placed in the first-in-first-out buffer area to ensure that the data does not overflow.

[0067] (7) During frozen playback, synchronized ultrasound endoscopic images and electronic endoscopic images are acquired and displayed from the first-in-first-out buffer.

[0068] (8) The matched ultrasound endoscopic image and electronic endoscopic image are stored as image files.

[0069] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for synchronously caching and displaying an image, characterized in that: It includes: Display the images of ultrasound endoscope and electronic endoscope in real time; The ultrasound endoscope and the electronic endoscope each correspond to a collection sub-thread, and each collection sub-thread is used to asynchronously collect the images of the ultrasound endoscope and the electronic endoscope. In each collection sub-thread, a single-frame buffer area and a queue buffer area are opened to store image data, wherein the single-frame buffer area stores the latest image data; After each acquisition sub-thread completes a single acquisition, the ultrasound endoscope and electronic endoscope each trigger the interface refresh of the corresponding image; With one of the devices completing a single acquisition action as the synchronization signal source, the latest images of the ultrasound endoscope and electronic endoscope stored in their respective single-frame buffers are transferred to their respective queue buffers. The contents of each queue buffer maintain frame synchronization, thus achieving image synchronization caching. When the playback is frozen, the image of the corresponding frame in each queue buffer is obtained according to the current playback frame, and the interface synchronously displays the images of the ultrasonic endoscope and the electronic endoscope; The queue buffer is a first-in-first-out queue buffer; the queue buffer sets a maximum number of cache frames. When the number of cache frames exceeds the set maximum number of cache frames, the image data that first enters the cache is removed, and then the new image data is placed in the first-in-first-out cache.

2. The image synchronization cache display method according to claim 1, characterized in that: Also includes: The matched ultrasound endoscope and electronic endoscope images are stored as image files.

3. The image synchronization cache display method according to any one of claims 1 to 2, characterized in that: The acquisition of ultrasound endoscopic images to complete this action will serve as a synchronization signal source.

4. An image synchronization cache display system, characterized in that: It includes: A display module, which displays the images of the ultrasonic endoscope and the electronic endoscope in real time; In the acquisition module, the ultrasound endoscope and the electronic endoscope each correspond to an acquisition sub-thread, and each acquisition sub-thread is used to asynchronously acquire the images of the ultrasound endoscope and the electronic endoscope. In each acquisition sub-thread, a single-frame buffer area and a queue buffer area are opened to store image data, wherein the single-frame buffer area stores the latest image data; Interface refresh module: after each acquisition sub-thread completes a single acquisition, the ultrasound endoscope and electronic endoscope each trigger the interface refresh of the corresponding image; The transfer module uses one of the devices completing a single acquisition action as a synchronization signal source to transfer the latest images of the ultrasonic endoscope and electronic endoscope stored in their respective single-frame buffers to their respective queue buffers. The contents of each queue buffer maintain frame synchronization, thus achieving image synchronization caching. Synchronous display module, when freezing playback, obtains the images corresponding to each queue buffer area according to the current playback frame, and the interface synchronously displays the images of the ultrasonic endoscope and the electronic endoscope; The queue buffer is a first-in-first-out queue buffer; the queue buffer sets a maximum number of cache frames. When the number of cache frames exceeds the set maximum number of cache frames, the image data that first enters the cache is removed, and then the new image data is placed in the first-in-first-out cache.

5. The image synchronization cache display system according to claim 4, characterized in that: Also includes: The matching storage module stores the matched ultrasonic endoscope and electronic endoscope images as image files.

6. The image synchronization cache display system according to any one of claims 4 to 5, characterized in that: The acquisition of ultrasound endoscopic images to complete this action will serve as a synchronization signal source.

7. An image synchronization cache display device, comprising a connected processor and a memory, characterized in that: The processor is used to execute the computer program stored in the memory to perform the image synchronization cache display method according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program, and when the computer program is run on a computer, the image synchronization cache display method according to any one of claims 1 to 3 is executed.