System and method for viewing an image

By capturing and identifying the screen layout of the supplier's computing system in the visual computing system and loading advanced visual applications, the sharing problem between different supplier applications is solved, seamless image viewing and manipulation is achieved, and efficiency is improved.

CN113096777BActive Publication Date: 2024-12-06KONINKLIJKE PHILIPS NV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110536412.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2013-04-24
Filing Date
2014-04-23
Publication Date
2024-12-06
Estimated Expiration
2034-04-23

AI Technical Summary

Technical Problem

The prior art is difficult to achieve seamless sharing between image viewing applications of different vendors, resulting in users needing to select and load image data multiple times, resulting in lengthy and time-consuming image viewing and manipulation.

Method used

By capturing the screen layout of the vendor computing system in the visual computing system, identifying and loading advanced visual applications, seamless sharing between different vendor applications is achieved, avoiding the limitations of software integration and standard compliance.

Benefits of technology

It realizes seamless image viewing and manipulation between different supplier applications, reduces user operation time and improves image processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113096777B_ABST
    Figure CN113096777B_ABST
Patent Text Reader

Abstract

A system and / or method is described herein that facilitates sharing of image viewing contexts between vendor visualization applications without the need to integrate different software applications packaged from different vendors. In one example, a visualization computing system (102) includes a processor (116) that executes computer-readable instructions that: capture a visual context of an imaging study displayed via a basic visualization application running on a vendor computing system; identify the study based on the captured visual context; load the study on the visualization computing system; and enable an advanced visualization application that allows viewing and manipulation of the loaded study using advanced visualization tools that are not available with the basic visualization application.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application. The application date of the original application is April 23, 2014, the application number is 201480022953.6, and the name of the invention is “Image Visualization”. Technical Field

[0002] The following generally relates to viewing images, such as computed tomography (CT) images, X-ray images, magnetic resonance (MR) images, positron emission tomography (PET) images, single photon emission computed tomography (SPECT) images, and / or other images. Background Art

[0003] Images generated in electronic format by systems such as CT, X-ray, MR, PET, SPECT, and / or other imaging systems have been viewed by running visualization software on a computing system such as a computer. For example, such images have been viewed via a PACS (Picture Archiving and Communication System) or the like. Such systems have been provided by different vendors with basic generic viewing functionality, allowing similar viewing across systems. Where a particular vendor includes advanced visualization tools and / or customized information in images with their software, such tools and / or information may not be available and / or accessible via another vendor's application software.

[0004] One way to mitigate such unavailability and / or inaccessibility of advanced visualization tools and / or customized information is for vendors to provide other vendors with plug-in applications that run on the other vendors' systems, etc. However, this requires running two different applications from two different vendors in the same software environment without sharing any information. As a result, the user must select and load image data twice, once for each application / system, and may go back and forth between applications to view and / or manipulate the imaging data, making image viewing and / or manipulation tedious and time-consuming for physicians.

[0005] The CCOW (Clinical Context Object Workgroup) background sharing standard provides a limited solution to the above problem. Regarding health information, the CCOW standard is an HL7 standard protocol that allows different applications of suppliers to be synchronized in real time on the user interface level, and presents information on the desktop and / or portal level in a unified manner. This standard requires that the applications of both suppliers follow the same standard. Unfortunately, most suppliers have not yet implemented the CCOW standard. In addition, partly due to the conflicting business goals between suppliers, this may not change in the future. Summary of the invention

[0006] Aspects of the presently described invention address the above-referenced issues and others.

[0007] The following describes a system and / or method that facilitates sharing of image viewing contexts between various vendor visualization applications without the need to integrate different software applications packaged from different vendors, neither through software integration nor through compliance with standards. However, partial or complete integration of different software applications from different vendors is also contemplated herein.

[0008] In one aspect, a visualization computing system includes a processor that executes computer-readable instructions that: capture a visual context of an imaging study displayed via a basic visualization application running on a vendor computing system; identify the study based on the captured visual context; load the study on the visualization computing system; and enable an advanced visualization application that allows viewing and manipulation of the loaded study using advanced visualization tools that are not available with the basic visualization application.

[0009] In another aspect, a method includes: capturing a screen layout of a study loaded in conjunction with a basic visualization application running on a first computing system; identifying an identity of the loaded study based on the captured screen layout; and loading the identified study in conjunction with an advanced visualization application of a second, different computing system, the advanced visualization application including visualization tools other than those basic visualizations.

[0010] In another aspect, a computer-readable storage medium is encoded with one or more computer-executable instructions that, when executed by a processor of a computing system, cause the processor to: determine an identification of a study loaded in a basic visualization application running on a first computing system based on a screen capture of the loaded study and a layout template of the basic visualization application; and load the identified study in conjunction with an advanced visualization application of a second, different computing system, the advanced visualization application including visualization tools in addition to those basic visualizations.

[0011] The invention may take the form of various components and arrangements of components, and may take the form of various steps and arrangements of steps.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A sample visualization computing system including advanced visualization application(s) and a context recognizer is schematically illustrated.

[0013] Figure 2 An example of a background identifier is shown.

[0014] Figure 3An example of data displayed in a GUI for a basic visualization application running on a supplier computing system is illustrated.

[0015] Figure 4 An example of a visual computing system in conjunction with an external imaging device is illustrated.

[0016] Figure 5 Illustration of a sample screen layout template generator.

[0017] Figure 6 An example method for identifying and using a visualization context of a vendor computing system using a visualization computing system with an advanced visualization tool is illustrated.

[0018] Figure 7 Illustrate an example method for generating a screen layout template.

[0019] Figure 8 Text information for a first screenshot of an image having first text information and an object is illustrated.

[0020] Fig. 9 Text information for a second screenshot having second text information different from the first text information and an image of an object is illustrated.

[0021] Fig.10 The diagram shows that by Figure 8 and Fig. 9 The difference image is generated by subtracting the screen layout of DETAILED DESCRIPTION

[0022] First reference Figure 1 , a visualization computing system 102 is schematically illustrated in conjunction with a vendor computing system 104, imaging systems 106, 108, 110, and 112, and a data repository 114. In this example, the vendor computing system 104 includes basic visualization tools, while the visualization computing system 102 includes basic visualization tools and additional visualization tools such as vendor customized tools.

[0023] Visual computing system 102 includes processor(s) 116, such as a microprocessor, a central processing unit, a controller, etc. Visual computing system 102 also includes input / output (I / O) 118, which facilitates communication with output device(s) 120, such as a display monitor, a video player, etc., communication with input device(s) 122, such as a mouse, a keyboard, etc., communication with a network 124, etc.

[0024] The visualization computing system 102 also includes a computer-readable storage medium 126, which includes physical memory or other non-transitory storage. The processor(s) 116 execute computer-readable instructions 128 encoded or embedded in the computer-readable storage medium 126. The processor(s) 116 can also execute computer-readable instructions carried by signals, carrier waves, and other transient (non-computer-readable storage) media.

[0025] In the illustrated example, the instructions 128 include (one or more) basic visualization applications 130, which for this example include instructions for basic viewing functions that may be common across most vendor computing systems. The illustrated vendor computing system 104 also includes (one or more) basic visualization applications 130, as well as (one or more) processors, computer-readable storage media, I / O, input and output devices, which are not shown for the sake of brevity and clarity.

[0026] The instructions 128 also include advanced visualization application(s) 132, which for this example include additional instructions for image viewing and / or manipulation functionality that are not common to the vendor computing system 104 and / or portions of the base visualization application(s) 130. The instructions 128 also include a context identifier 134. As described in more detail below, the context identifier 134 identifies the visualization context (or screen layout) of the vendor computing system 104 and employs that context with the visualization computing system 102 to present the same study in conjunction with the advanced visualization application(s) 132.

[0027] In one example, this allows a user to seamlessly move from a vendor computing system 104 to a visualization computing system 102 when the user desires to use advanced visualization tools that are not available via the vendor computing system 104. Seamless means that the same studies and the same images rendered by the vendor computing system 104 are automatically recognized, loaded, and rendered by the visualization computing system 102. In one example, this alleviates having to integrate visualization applications of different vendors and / or conform to standard screen layouts utilized by multiple different vendors.

[0028] The illustrated visualization computing system 102 and / or the provider computing system 104 obtains imaging data from one or more of the imaging systems 106, 108, 110, and 112, a data repository 114, and / or other devices and / or storage devices. The imaging systems include a CT imaging system 106, an MR imaging system 108, a SPECT imaging system 110, and a PET imaging system 112. Other imaging systems are also contemplated by the present invention. The data repository 114 may include one or more of a radiology information system (RIS), a hospital information system (HIS), an electronic medical record (EMR), a server, a database, and the like.

[0029] The visualization computing system 102 can be activated to determine a visualization context for the vendor computing system 104 in response to a user activating the visualization computing system 102 to do so, such as when the user determines that they want to use the advanced visualization application(s) 132. In another example, the visualization computing system 102 determines the context and stores the context information when the basic visualization application is employed, and / or preloads studies on the visualization computing system 102.

[0030] The provider computing system 104 and / or the visualization computing system 102 can be a PACS and / or other computing systems.

[0031] Go to Figure 2 , an example of the background identifier 134 is illustrated.

[0032] The screen capture component 202 captures the current background or content displayed by the vendor computing system 104 in a display monitor or the like. Figure 3 An example of such content is depicted in Figure 3 A monitor 302 is shown having a display area 304 in which a basic visualization graphical user interface (GUI) 306 corresponding to the basic visualization application 105 is displayed. Figure 3 , a study has been loaded and visually presented in the GUI 306. The loaded data includes images of two-dimensional axial slices of the scanned object and various information corresponding to the patient, scan, axial slice, etc. In this example, the screen capture component 202 visually captures the GUI 306.

[0033] In one example, the screen capture component 202 includes a software module that is communicated to and executed by the vendor computing system 104. The executed software module captures the screen in an electronic data format and communicates the electronic data to the visualization computing system 102. In another example, the software module is communicated to and executed by the vendor computing system 104 in other ways, such as via a server over the network 124, from a portable memory (e.g., CD / DVD, etc.), etc. In yet another example, the screen capture component 202 is employed in conjunction with an external imaging device, such as an optical sensor, such as a camera, a video recorder, etc. This is shown in FIG. Figure 4 middle, Figure 4 An external imaging device 402 is included. In this case, a picture or video of the GUI 306 is sent to the visualization computing system 102.

[0034] Return to Figure 2 , the region identifier 204 employs (one or more) predetermined screen layout templates 206 from a template library 208 to identify one or more regions in the captured screenshot. The particular template(s) 206 correspond to the layout of the information in the captured screenshot and can be identified from a plurality of different templates 206 based on, for example, the name and / or unique identification of the vendor of the vendor computing system 104, the name of the underlying visualization software of the vendor computing system 104, a template selected by the user, an identification of a viewing facility at which the image is being viewed, and / or other information. Figure 3 In this example, the template 206 identifies, for example, by screen coordinates or otherwise, regions 310, 312, 314, and 316. The template 206 also identifies which information is displayed in each of the one or more regions.

[0035] Return to Figure 2 , the information extractor 210 extracts information from the identified one or more regions of the captured screenshot. Figure 3 In the example in FIG. 2 , template 206 identifies region 310 as displaying a string and / or value corresponding to a unique identification of a study (“Study ID”), region 412 as displaying a string and / or value corresponding to a sequence number (“Sequence #”), region 314 as displaying a string and / or value corresponding to an image slice number (“Image #”), and region 316 as displaying a string and / or value corresponding to a slice position (“Slice Position”). Information extractor 210 extracts the information such that it extracts “Study ID”, “Sequence #”, “Image #”, and “Slice Position”.

[0036] The character recognizer 212 interprets the extracted information to determine the meaning of the extracted information. Figure 3, the character recognizer 212 interprets the extracted information corresponding to "sequence #" 312 as "#2", the extracted information corresponding to "image #" as "#107", etc. The study retriever 214 retrieves the study, for example, based on the interpreted extracted information corresponding to the unique identification of the study. The study can be retrieved from the CT imaging system 106, the MR imaging system 108, the SPECT imaging system 110, the PET imaging system 112, and / or other imaging systems, the data repository 114, and / or other devices.

[0037] The advanced application enabler 216 enables the advanced visualization application(s) 132. The specific application to enable can be identified based on the interpreted extracted information. For example, where the interpreted extracted information includes information indicating a specific scan protocol, such as a cardiac scan, the advanced application enabler 216 can select from the advanced visualization application(s) 132 ( Figure 1 ) selects an advanced cardiac application. In another example, the user selects (one or more) advanced visualization applications of interest 132, such as via a GUI selection from a menu of available advanced applications. The menu can be presented in either or both of the vendor computing system 104 or the visualization computing system 102. In yet another example, a default advanced application is selected. The default application can be identified via a defaults file.

[0038] Figure 5 An example template generator 502 is illustrated that generates at least one of the templates 206 .

[0039] The template generator 502 obtains (e.g., retrieves, receives, etc.) at least two screenshots of images of the same object (e.g., a calibration phantom), the at least two screenshots having different but known text information, such as different "Study ID", "Sequence #", "Image #", "Slice Location", and / or other displayed information. The screenshots can be obtained via the screen capture component 202 and / or in other ways. This can include loading two studies and capturing the screen layout and / or receiving the screen layout.

[0040] The image difference generator 504 subtracts the at least two screenshots to generate a difference image. Since the objects are the same in the at least two screenshots, the objects therein cancel out. However, the information in the text information is different, and therefore the difference image will include areas with different information. This is shown in Figure 8 , Fig. 9 and Fig.10 middle. Figure 8 The text information for the first screenshot is shown. Fig. 9 shows different text information for the second screenshot, and Fig.10 Shows Figure 8 and Fig. 9 The difference between the text information in .

[0041] Region locator 506 records the coordinates for these regions. In this example, three regions are located, a first region 1002 corresponding to the examination identifier, a second region 1004 corresponding to the sequence identifier and a third region 1006 corresponding to the image identifier. In this example, the three regions are adjacent to each other. In other examples, the regions may be located in different regions of the image, such as in combination with 310, 312, 314 and 316 in Figure 3 In other examples, more or fewer regions are identified.

[0042] The string matcher 508 uses the coordinate information to match the known meaning of the text information in the original at least two images to locate the text information. Fig.10 , the string matcher 508 matches the position corresponding to 1002 with the string corresponding to the inspection identifier, matches the position corresponding to 1004 with the string corresponding to the sequence identifier, and matches the position corresponding to 1006 with the string corresponding to the image identifier.

[0043] The mapper 510 maps the identified string to a corresponding position to generate a screen layout template that identifies an area of ​​interest in a screen layout that includes text information of interest. The above can be repeated for multiple different vendors so that a screen layout template is generated for the screen layout of each of the multiple different vendors. In another example, a vendor provides a screen layout template for its screen layout. The template can be stored in the visualization computing system 102 (as shown) or externally thereto, such as at a server.

[0044] Figure 6 and Figure 7 A method according to the description herein is illustrated.

[0045] It should be appreciated that the order of the actions in the method is not restrictive. In this way, the present invention contemplates other orders. In addition, one or more actions may be omitted and / or one or more additional actions may be included.

[0046] Figure 6 An example method for employing the visualization computing system 102 is illustrated.

[0047] At 602, the study is loaded into a base visualization application running on a provider computing system.

[0048] At 604, an image displayed by the vendor computing system is captured.

[0049] At 606, text information in the region of interest identified from the template is extracted from the captured image.

[0050] At 608, the extracted text information is interpreted to identify studies loaded in a base visualization application.

[0051] At 610, the identified studies are retrieved.

[0052] At 612, an advanced visualization application is enabled on a visualization computing system different from the vendor computing system.

[0053] At 614, the identified studies are loaded in the advanced visualization application.

[0054] At 616 , an operator of the visualization computing system views and / or manipulates the loaded study via the advanced visualization application.

[0055] Figure 7 An example method for generating a template is illustrated.

[0056] At 702, for a base visualization application running on a vendor computing system, at least two screenshots of an image of the same object with different but known text information are obtained. This can be accomplished by loading a study and performing screen capture side by side or by receiving an already captured screen layout.

[0057] At 704, the at least two screenshots are subtracted to generate a difference image.

[0058] At 706, regions of the different images having different known text information are identified. Typically, regions in at least two images having the same information will cancel out so that the only regions in the difference image having text information are those regions that include different text information.

[0059] At 708, text information in the identified region is extracted.

[0060] At 710, the extracted text information is matched with known meanings.

[0061] At 712 , the meaning and the corresponding position of the extracted text information are mapped to generate a screen layout template for a screen layout of a basic visualization application running on a vendor computing system.

[0062] At 714, the mapping is stored.

[0063] At 716, the mapping is utilized by the visualization computing system 102 to present, via the advanced visualization application running on the visualization computing system 102, the studies loaded in the base visualization application running on the vendor computing system, with the studies presented in the same visual context as presented in the base visualization application running on the vendor computing system.

[0064] The above method can be implemented by computer-readable instructions encoded or embedded on a computer-readable storage medium, and when the computer-readable instructions are processed and executed by (one or more) computers, the (one or more) processors are caused to perform the described actions. Additionally or alternatively, at least one of the computer-readable instructions is carried by a signal, carrier wave or other transient medium.

[0065] In a variation, context recognition is performed through the use of accessibility application programming interfaces (APIs) for operating systems from Microsoft, , Apple, etc. The accessibility APIs allow external applications to introspect another application's user interface in a non-intrusive manner. These APIs are intended to enable accessibility tools such as screen readers to work with any application.

[0066] In addition, a screen background recognition (SCR) profile can be created based on the user interface (UI) hierarchy of the application and then used to retrieve information from various UI elements. Profile creation can follow a very similar technique to above, where an image with known data is displayed. The UI can then be "searched" and known markers are found to identify their matching UI elements.

[0067] The SCR system will need to be deployed in the target environment (PC) and be able to be very lightweight. A resident agent will have to be included, which runs constantly to capture and extract the context.

[0068] To further ease the integration of specialized applications that use context information (which is often complex on a PACS workstation), the SCR system can transmit context capture and changes to a server. The server will then be used by a thin client application such as the IntelliSpace portal client or a web-based zero-footprint application that can then react appropriately to context changes.

[0069] The present invention has been described with reference to preferred embodiments. Others may realize modifications and variations after reading and understanding the above detailed description. The present invention is intended to be constructed to include all such modifications and variations as long as they fall within the scope of the claims or their equivalents.

Claims

1. A system for viewing an image, comprising: A visualization computing system (102), comprising: A processor (116) that executes computer readable instructions that: capturing screen content of an imaging study displayed via a base visualization application running on a first computing system, the first computing system and the visualization computing system being separate and distinct computing systems; identifying the imaging study based on the captured screen content; loading the imaging study on the visualization computing system; and enabling an advanced visualization application that allows the loaded imaging study to be viewed and manipulated using advanced visualization tools not available with the basic visualization application, wherein the visualization computing system is configured to capture the screen content of the first computing system in the format of electronic data via a software module, the software module being executed by the first computing system and determining the electronic data by introspecting a user interface of the base visualization application using an accessibility API; wherein the executed computer readable instructions identify a study protocol based on the captured screen content, the study protocol indicating a scan type and enabling the advanced visualization application corresponding to the same scan type; and The first computing system includes the software module configured to determine the electronic data by introspecting the user interface of the base visualization application using the accessibility API and to send the electronic data to the visualization computing system.

2. The system according to claim 1, wherein: The study is identified based on a vendor-specific screen layout template corresponding to the first computing system.

3. The system according to claim 2, wherein: The template identifies the location of text information in the captured screen layout via screen coordinates, the text information identifying the study.

4. The system according to claim 3, wherein: The executed computer readable instructions also extract the text information from the location identified in the captured screen layout.

5. The system according to claim 4, wherein: The executed computer readable instructions employ a character recognition algorithm that interprets the extracted text information.

6. A system according to any one of claims 2 to 5, wherein: The executed computer readable instructions generate the template based on at least two calibration images of the supplier, wherein the calibration images include different text information.

7. The system according to claim 6, wherein: The executed computer readable instructions subtract the at least two calibration images from each other to generate a difference image indicating the location of different texts.

8. The system according to claim 7, wherein: The executed computer readable instructions map the locations to the data types at the locations, thereby creating the template.

9. A method for viewing an image, comprising: capturing screen content of a study loaded in conjunction with a base visualization application running on a first computing system, wherein the screen content of the first computing system is captured via a software module in the format of electronic data and sending the electronic data to a second computing system, the software module being executed by the first computing system and determining the electronic data by introspecting a user interface of the base visualization application using an accessibility API, the first computing system and the second computing system being separate and distinct computing systems; identifying, at the second computing system, an identification of the loaded study based on the captured screen content; and loading the identified study in conjunction with an advanced visualization application of the second computing system, the advanced visualization application including visualization tools other than those of basic visualization, wherein a study protocol indicative of a scan type is identified based on the captured screen content, and the advanced visualization application corresponding to the same scan type is enabled.

10. The method according to claim 9, further comprising: The identification of the loaded study is identified based on a screen layout template corresponding to a vendor of the first computing system.

11. The method according to claim 10, wherein: The template identifies the location of text information in the captured screen layout, the text information identifying the study.

12. The method according to claim 11, further comprising: extracting said text information from said location identified in the captured screen layout; interpreting the extracted text information; and A study protocol indicating the scan type is identified based on the captured screen layout.

13. The method according to claim 12, further comprising: The advanced visualization application is enabled, the advanced visualization application corresponding to the same scan type.