Visual interaction method, device and storage medium for multidisciplinary collaborative diagnosis and treatment

Abstract

The application relates to the field of intelligent medical technology and discloses a visual interaction method and device for multidisciplinary collaborative diagnosis and treatment and a storage medium. The method comprises the following steps: a graphical user interface is included, a first window for displaying a plurality of first partitions is included, a second window for displaying a time axis and a plurality of second partitions is included, the plurality of second partitions are associated with respective time nodes sequentially distributed along the time axis; in response to a first interaction operation on a target first partition, at least part of the time nodes are determined, a first page is generated based on the at least part of the time nodes and is updated to the second window, and the first page is used for adjusting display content of the second partition associated with the at least part of the time nodes; and in response to a second interaction operation on a target second partition, a second page is generated based on a target time node of the target second partition and is updated to the second window. The application simplifies the interaction logic of the visual interaction method for multidisciplinary collaborative diagnosis and treatment, so that information presentation is more convenient and intuitive.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical technology, and in particular to a visual interactive method, device and storage medium for multidisciplinary collaborative diagnosis and treatment. Background Technology

[0002] Multidisciplinary Team (MDT) is a core model of modern precision medicine. It requires experts from different departments (such as surgery, internal medicine, radiology, pathology, etc.) to jointly review medical records, analyze examination data, and discuss treatment plans for the same patient's complete medical history. An efficient MDT meeting relies heavily on the effective integration and intuitive presentation of the patient's full-cycle, multimodal medical information.

[0003] Currently, software systems or platforms supporting MDT typically adopt the following information organization and interaction modes: (1) Pagination / tabbed system: Different types of patient information (such as basic information, medical records, imaging reports, test results, pathological slides, etc.) are placed in different independent pages or tabs. Experts need to frequently switch, search, and compare between multiple pages during discussions. For example, when discussing the condition at a specific point in time, experts need to locate the corresponding record at the time point in the "medical record page", "imaging page", and "test page", which is cumbersome and easily interrupts the continuity of the discussion and the focus of thinking. (2) Simple timeline list system: The system arranges all medical events into a linear list in chronological order. Although this method has a temporal sequence, the information presentation format is singular (mostly text lists), and the space utilization rate is low. Data from different disciplines are mixed and arranged, lacking effective classification and juxtaposition comparison views. Experts find it difficult to quickly extract and associate data of specific disciplines at key time nodes, and cannot intuitively grasp the synergistic and evolutionary relationship of data from different disciplines in the time dimension. (3) Static dashboard system: Key indicators are presented in chart form on a single interface. This mode is suitable for overviews, but lacks the ability to drill down in depth and explore interactively throughout the entire diagnosis and treatment process. When it is necessary to trace the detailed, multidisciplinary reasons behind an abnormal indicator, the system often cannot provide seamless navigation from the macro timeline to the micro data details.

[0004] Based on the above-mentioned existing technologies, the current MDT visualization interaction system mainly suffers from the following technical problems, which limit the efficiency and depth of diagnosis and treatment collaboration: (1) Information silos and context fragmentation: Data of different disciplines and types are physically isolated on different pages or areas. Experts must perform a lot of manual memorization and "mental integration" when analyzing the data. They cannot conduct intuitive and spatial comparative analysis under a unified view with a clear time reference, which makes information fusion difficult and makes it easy to miss key connections. (2) The time sequence navigation is not intuitive and inefficient: The existing system lacks an interactive mechanism that can display the time structure of the entire diagnosis and treatment process globally and intuitively, and support the rapid location and screening of key time nodes. Experts find it difficult to quickly get an overview of the diagnosis and treatment stages, and cannot easily focus on a specific period (such as before and after surgery, chemotherapy cycle) for centralized review of multidisciplinary data. (3) The interaction path is lengthy and unnatural: From "selecting the subject of interest" to "locating the time range" and then to "viewing the detailed data of the subject within this time period," this operation path, which conforms to the core thinking logic of MDT, often requires multiple discrete operation steps in the existing system (such as multiple menu selections, page jumps, list scrolling, etc.), resulting in a choppy interaction and disrupting the continuity of multidisciplinary discussions. (4) The spatial layout is rigid and cannot support dynamic focusing: The interface layout is fixed and cannot dynamically reorganize the presentation density and related display of information according to the current focus of the discussion (such as a key subject or a key stage of diagnosis and treatment), which limits the information carrying efficiency and cognitive guidance ability of the interface. In summary, the existing MDT has problems with complex visualization interaction logic and insufficiently intuitive and convenient information display. Summary of the Invention

[0005] The main objective of this invention is to solve the technical problems of complex visualization and interaction logic and insufficient intuitiveness and convenience of information display in the prior art of MDT.

[0006] The first aspect of the present invention provides a visual interactive method for multidisciplinary collaborative diagnosis and treatment, including a graphical user interface (GUI) providing a first window and a second window. The first window is used to display a plurality of first partitions; the second window is used to display a timeline and a plurality of second partitions, wherein the timeline includes a plurality of sequentially distributed time nodes, and the plurality of second partitions are associated with each time node sequentially distributed along the timeline. The method includes: in response to a first interactive operation on a target first partition in the first window, determining at least a portion of time nodes, generating a first page based on the at least a portion of time nodes and updating it to the second window, wherein the first page is used to adjust the display content of the second partitions associated with the at least a portion of time nodes; and in response to a second interactive operation on a target second partition in the second window, generating a second page based on a target time node of the target second partition and updating it to the second window, wherein the second page is used to adjust the display content of the target second partition.

[0007] Optionally, the display content in the second window also includes the display content corresponding to the first treatment category in the plurality of second partitions. In response to a first interactive operation on the target first partition in the first window, determining at least some time nodes, generating a first page based on the at least some time nodes, and updating it to the second window includes: in response to a first press operation on the target first partition in the first window, determining the second treatment category of the target first partition, and selecting at least some time nodes from the plurality of time nodes, wherein the at least some time nodes are associated with the second treatment category; adjusting the display content corresponding to the first treatment category in the second partition associated with the at least some time nodes to the display content corresponding to the second treatment category, and generating the first page; during the continuous pressing process of the first press operation, updating the display content in the second window according to the first page.

[0008] Optionally, during the continuous pressing process of the first pressing operation, after updating the display content in the second window according to the first page, the method further includes: in response to the first drag operation on the target first partition, determining the first release position of the first drag operation, wherein the first pressing operation and the first drag operation are consecutive operations; if the first release position is in the first release area, then switching the second window from the first page to the display content before the first page update; if the first release position is in the second release area, then continuously displaying the first page in the second window.

[0009] Optionally, in response to a second interactive operation on a target second partition in the second window, generating a second page based on a target time node of the target second partition and updating it to the second window includes: in response to a second press operation on a target second partition in the second window, obtaining first medical information corresponding to a target time node associated with the target second partition; generating a second page based on the first medical information corresponding to the target time node; and updating the displayed content in the second window according to the second page during the continuous pressing process of the second press operation.

[0010] Optionally, during the continuous pressing process of the second pressing operation, after updating the display content in the second window according to the second page, the method further includes: in response to the second drag operation on the target second partition, determining the second release position of the second drag operation, wherein the second pressing operation and the second drag operation are consecutive operations, and during the drag operation, the second page remains unchanged in the second window; if the second release position is in the third release area, then the second window is switched from the second page to the display content before the second page update; if the second release position is in the fourth release area, then the second page is continuously displayed in the second window.

[0011] Optionally, the content displayed on the second page includes at least one optional content. In response to a second interactive operation on the target second partition in the second window, after generating the second page based on the target time node of the target second partition and updating it to the second window, the method further includes: in response to a selection operation on any optional content, obtaining reference content associated with any optional content from the second medical information corresponding to the reference node, wherein the reference node is a non-target time node among the plurality of time nodes; and updating the second page based on the reference content.

[0012] Optionally, the second page is further used to adjust the display style of the target second partition, wherein the display style includes at least one of the size of the target second partition and a highlight effect, and the size is greater than the size of the target second partition before adjustment and less than or equal to the size of the second window.

[0013] Optionally, the visualization interaction method further includes: in response to a third drag operation on any first partition, determining a third release position of the third drag operation; if the third release position is the position of any second partition, determining a third treatment category of the any first partition; and based on the third treatment category, adjusting the display style and display content of the any second partition and updating it to the second window.

[0014] Optionally, the diagnosis and treatment categories corresponding to the plurality of first partitions include: hospital category, department category, doctor category, diagnosis category, indicator category, and medical imaging category; the display content of the plurality of second partitions includes at least part of the diagnosis and treatment information corresponding to the associated time node that is related to one or more diagnosis and treatment categories.

[0015] Optionally, the content displayed on the second page includes a first medical image; the visualization interaction method further includes: in response to a first sliding operation on the first medical image along a preset first direction, determining a second medical image according to the preset first direction, and updating the display content in the second window according to the second medical image, wherein the second medical image is another medical image besides the first medical image among the plurality of medical images corresponding to the target time node; or, in response to a second sliding operation on the second medical image along a preset second direction, determining a third medical image according to the preset second direction, and updating the display content in the second window according to the third medical image, wherein the third medical image is a medical image corresponding to a non-target time node among the plurality of time nodes.

[0016] Optionally, the preset first direction and the preset second direction are the directions from the center region of the first medical image to the edge region; updating the display content in the second window according to the second medical image includes: switching the first medical image displayed in the second window to the second medical image; and updating the display content in the second window according to the third medical image includes: switching the first medical image displayed in the second window to the third medical image.

[0017] Optionally, the preset first direction and the preset second direction are the directions from the edge region of the first medical image toward the center region; updating the display content in the second window according to the second medical image includes: simultaneously displaying the first medical image and the second medical image in the second window; and updating the display content in the second window according to the third medical image includes: simultaneously displaying the first medical image and the third medical image in the second window.

[0018] A second aspect of the present invention provides a computer device comprising: a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to cause the computer device to execute the above-described visual interactive method for multidisciplinary collaborative diagnosis and treatment.

[0019] A third aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the above-described visual interactive method for multidisciplinary collaborative diagnosis and treatment. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of an embodiment of a visualization and interaction method for multidisciplinary collaborative diagnosis and treatment according to the present invention;

[0021] Figures 2-12 This is a schematic diagram of the graphical user interface of the visualization and interactive device for multidisciplinary collaborative diagnosis and treatment in an embodiment of the present invention;

[0022] Figure 13 This is a schematic diagram of one embodiment of the computer device in this invention. Detailed Implementation

[0023] To facilitate understanding of the present invention, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0024] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. For example, the term "a plurality of" includes two or more.

[0025] The specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the visualization interaction method for multidisciplinary collaborative diagnosis and treatment in this invention includes:

[0026] 110. In response to a first interactive operation on a target first partition in the first window, at least some time nodes are determined, a first page is generated based on the at least some time nodes and updated to the second window, wherein the first page is used to adjust the display content of the second partition associated with the at least some time nodes;

[0027] It is understood that the subject of this invention can be a visual interactive device for multidisciplinary collaborative diagnosis and treatment, including smart devices such as smartphones, tablets, laptops, and desktop computers, and is not specifically limited here. The execution flow of the visual interactive method for multidisciplinary collaborative diagnosis and treatment in an MDT application scenario will be described below.

[0028] In this embodiment, a graphical user interface is included, which provides a first window and a second window. The first window is used to display a plurality of first partitions; the second window is used to display a timeline and a plurality of second partitions, wherein the timeline includes a plurality of time nodes distributed sequentially, and the plurality of second partitions are associated with each time node distributed sequentially along the timeline.

[0029] In this embodiment, the first window and the second window are rectangular, rounded rectangles, circles, ellipses, or other shapes, and the size of the first window is smaller than the size of the second window. Examples of size ratios include 1:5, 1:6, 1:7, 1:8, 1:9, and 1:10. On the graphical user interface, the first and second windows can be displayed as rectangular or rounded rectangles arranged horizontally or vertically. One pair of adjacent opposite sides has the same length, while the other pair of opposite sides follows the aforementioned size ratio. In another embodiment, the first window is located within the second window, and both the first and second windows are identical shapes scaled down to the aforementioned size ratio.

[0030] In one implementation, the diagnosis and treatment categories corresponding to the plurality of first partitions include: hospital category, department category, doctor category, diagnosis category, indicator category, and medical imaging category; the display content of the plurality of second partitions includes at least a portion of the diagnosis and treatment information corresponding to the associated time node that is related to one or more diagnosis and treatment categories.

[0031] In this embodiment, each first partition of the first window is displayed in the first window as a rectangle, rounded rectangle, circle, ellipse or other shapes, and each first partition displays different diagnostic and treatment category entries. For example, these include entries for diagnostic and treatment categories such as hospital, department, doctor, diagnosis, indicator, and medical imaging.

[0032] In this embodiment, the timeline of the second window is used to guide the user to view the patient's past medical history along the timeline; multiple time nodes are distributed along the timeline to represent each time the patient seeks medical treatment; the second partition associated with each time node is used to display the patient's medical information at each time, including one or more of the relevant information of the diagnosis and treatment category such as the hospital, department, doctor, diagnosis results, examination indicators, and collected medical images at the time of the visit.

[0033] The medical information includes detailed medical information and key medical information, with the key information being a summary description of the detailed medical information. Each time point is associated with all of the patient's medical information, and detailed or key medical information for the corresponding medical time is obtained and output based on user interaction. In one implementation, the initial preset display page of the second window includes: each second section obtains and displays some or all key medical information for the associated medical categories based on the associated time point.

[0034] Specifically, machine learning models or neural network models are used to extract key medical information from detailed medical information based on automatic semantic recognition, or key medical information is extracted manually from detailed medical information.

[0035] In addition, multiple time points may be displayed or not displayed in the first window; the visualization includes graphic patterns arranged sequentially on the timeline, including circular patterns, square patterns, or other polygonal patterns; the visualization may also include time text content, such as time text content in the format of "xxxx year xx month xx day" or "xxxx year xx month".

[0036] Each time node distributed along the time axis is connected to its corresponding second partition by a line. For example, the line connection can include a straight line, a curve, or an arc. Each second partition is distributed according to the order of the time nodes on the time axis, in the same order. All second partitions are located on the same side of the connecting axis, or adjacent second partitions are distributed across each other on opposite sides of the time axis. Each second partition is displayed in the second window as a rectangle, rounded rectangle, circle, ellipse, etc., without specific limitations.

[0037] See Figure 2 An example graphical user interface is provided, including a first window (e.g., first example window 210) and a second window (e.g., second example window 220). The first window includes multiple first sections (e.g., first example section 211), and the second window includes multiple second sections (e.g., second example section 221) and a timeline 222. Time nodes are sequentially distributed on the timeline (e.g., time node 223). Each first section displays entries for various medical categories, including hospital categories, department categories, doctor categories, diagnosis categories, indicator categories, medical imaging categories, etc. Each second section displays medical information for the corresponding time node, including hospital, department, doctor, diagnosis results, examination items, indicators, etc. In addition, it may further include collected medical images.

[0038] In this embodiment, the first interactive operation specifically includes one or a combination of operations such as click, press, long press, drag, slide, and multi-finger zoom. Users can perform the first interactive operation on the graphical user interface through gestures, mouse, keyboard, touch, etc.

[0039] In this embodiment, each first partition corresponds to a different diagnosis and treatment category and is associated with each type of diagnosis and treatment information associated with a time node. A first interactive operation is performed on the target first partition in the first window to select a diagnosis and treatment category. Based on the selected diagnosis and treatment category, the corresponding diagnosis and treatment information for each time node is selected and output to the second partition for display.

[0040] Each time node's corresponding storage space stores some or all of the diagnosis and treatment information corresponding to the diagnosis and treatment categories. If no corresponding diagnosis and treatment information can be found for a time node based on the selected diagnosis and treatment category, the output of the diagnosis and treatment information corresponding to the selected diagnosis and treatment category will be empty, or the display content of the second partition will not need to be adjusted, and only the diagnosis and treatment information that can be found will be output; that is, only the display content of the second partition associated with some time nodes can be adjusted.

[0041] In this embodiment, a first page is generated based on medical information queried at least some time points, and is displayed in a fixed window or a floating window in a second window. The page displayed before the second window switches to the first page can be retained or directly replaced.

[0042] If the first page is displayed in a fixed position in the second window, when the first interactive operation ends, the page displayed before the second window switches to the first page is directly replaced by the first page; if the first page is displayed in a floating window in the second window, when the first interactive operation ends, the first page in the second window reverts to the reserved display page, or further replaces the reserved display page to convert it into a fixed display.

[0043] In one implementation, the second window displays an initial preset display page, specifically including key medical information of some or all medical categories in each second partition; in response to a first interactive operation on the target first partition in the first window, the system controls a time node to obtain detailed medical information of a medical category, adjusts the key medical information of some or all medical categories displayed in each second partition to the detailed medical information of that medical category, and generates the first page.

[0044] In another implementation, each second partition displays detailed medical information for a specific medical category corresponding to the target first partition; in response to a first interactive operation on the target first partition in the first window, the key medical information for all medical categories is obtained at a controlled time point, the detailed medical information for a specific medical category displayed in each second partition is adjusted to the key medical information for all medical categories, and a first page is generated.

[0045] 120. In response to a second interactive operation on a target second partition in the second window, a second page is generated based on a target time node of the target second partition and updated to the second window, wherein the second page is used to adjust the display content of the target second partition.

[0046] In this embodiment, the second interactive operation specifically includes one or a combination of operations such as click, press, long press, drag, swipe, and multi-finger zoom. Users can perform the second interactive operation on the graphical user interface through gestures, mouse, keyboard, touch, etc.

[0047] In this embodiment, each second partition is used to display medical information at different time points, and the display is based on the medical information output at each time point. A second interactive operation is performed on the target second partition in the second window to determine the time point to which the second interactive operation needs to be pointed, and the adjusted medical information is obtained based on the time point, thereby adjusting the display content of the target second partition.

[0048] Specifically, the adjustment of the target second partition includes: adjusting the key medical information currently displayed in the target second partition to detailed medical information, or combining it with a first partition, adjusting the key medical information or detailed medical information currently displayed in the target second partition to detailed medical information of the corresponding medical category of the combined first partition.

[0049] Additionally, the second page is also used to adjust the display style of the target second partition. The display style includes at least one of the size of the target second partition and a highlighting effect. The size is greater than the original size of the target second partition and less than or equal to the size of the second window. That is, the size of the second partition is enlarged, or it is highlighted, to emphasize the content displayed in the target second partition.

[0050] In this embodiment, a second page is generated based on the medical information queried at least some time points. The second page is either fixed in the second window or displayed in a floating window. The page displayed before switching to the second page in the second window can be retained or directly replaced.

[0051] Specifically, if the second page is displayed in a fixed position in the second window, when the second interactive operation ends, the page displayed before switching to the second page in the second window is directly replaced by the second page; if the second page is displayed in a floating window in the second window, when the second interactive operation ends, the second page in the second window reverts to the reserved display page, or further replaces the reserved display page to convert it into a fixed display.

[0052] In one embodiment, the display content in the second window further includes the display content corresponding to the first treatment category in the plurality of second partitions. In response to a first interactive operation on a target first partition in the first window, determining at least a portion of time nodes, generating a first page based on the at least a portion of time nodes, and updating it to the second window includes: in response to a first press operation on a target first partition in the first window, determining the second treatment category of the target first partition, and selecting at least a portion of time nodes from the plurality of time nodes, wherein the at least a portion of time nodes are associated with the second treatment category; adjusting the display content corresponding to the first treatment category in the second partition associated with the at least a portion of time nodes to the display content corresponding to the second treatment category, and generating the first page; during the continuous pressing process of the first press operation, updating the display content in the second window according to the first page.

[0053] In this embodiment, the first treatment category includes some or all treatment categories. The display content corresponding to the first treatment category in the plurality of second partitions includes key treatment information or detailed treatment information in the treatment information. Alternatively, the display content corresponding to the first treatment category in a part of the plurality of second partitions includes key treatment information, while the other part includes detailed treatment information.

[0054] In this embodiment, the target first partition, i.e. the first partition corresponding to the pressing area of ​​the first pressing operation, queries the corresponding medical information at each time node based on the second medical category. If the medical information corresponding to the second medical category cannot be found through the time node, the display content of the second partition associated with the time node is not updated. If the medical information corresponding to the second medical category is found through the time node, the display content of the second partition associated with at least some time nodes is updated. The first page is generated based on whether the medical information of the second medical category is updated or not in the second partition.

[0055] For example, such as Figure 3As shown, the current display page of the second window (such as the third example window 310) (such as the first example page 311) displays key diagnostic and treatment information for all diagnostic and treatment categories. That is, the first diagnostic and treatment category includes all diagnostic and treatment categories, and the displayed content is the corresponding key diagnostic and treatment information. In response to the first press operation on the target first partition (such as the third example partition 321) in the first window (such as the fourth example window 320), the second diagnostic and treatment category corresponding to the target first partition (such as the third example partition 321) is determined to be the diagnostic category. From 8 time nodes, 6 time nodes associated with the diagnostic category are determined. The key diagnostic and treatment information corresponding to all diagnostic and treatment categories in the 6 second partitions (such as the fourth example partition 312) associated with the 6 time nodes is adjusted to the detailed diagnostic and treatment information corresponding to the diagnostic category. The key diagnostic and treatment information of all diagnostic and treatment categories in the other 2 second partitions (such as the fifth example partition 313) remains unchanged, and the first page 314 is generated.

[0056] Specifically, while the first press operation is continuously pressed, the first page 314 remains displayed in the second window (e.g., the third example window 310); when the first press operation is canceled, the first page 314 displayed in the second window (e.g., the third example window 310) is switched back to the currently displayed page (e.g., the first example page 311). This enables rapid switching between the currently displayed page (e.g., the first example page 311) and the first page 314, facilitating the comparison of the patient's medical information during the joint diagnosis and treatment process.

[0057] In addition to using the continuous pressing and canceling of the first pressing operation to switch between the currently displayed page and the first page, the first interactive operation also includes a click operation. Executing the click operation switches the currently displayed page to the first page, and executing the click operation again switches the first page back to the currently displayed page, and so on.

[0058] In one embodiment, during the continuous pressing process of the first pressing operation, after updating the display content in the second window according to the first page, the method further includes: in response to the first drag operation on the target first partition, determining a first release position of the first drag operation, wherein the first pressing operation and the first drag operation are consecutive operations; if the first release position is in a first release area, then switching the second window from the first page to the display content before the first page update; if the first release position is in a second release area, then continuously displaying the first page in the second window.

[0059] In this embodiment, the first drag operation is performed during the continuous pressing process after the first press operation, and the content displayed in the second window will not change with the first drag operation; when the first drag operation is canceled, the first page or the content displayed before the first page is updated is determined according to the first release position of the first drag operation.

[0060] Specifically, the first release area and the second release area are not overlapping and can be set at any location in the graphical user interface. A first example includes both the first and second release areas being set in the first window or both being set in the second window; a second example includes the first release area being set in one of the first and second windows, and the second release area being set in the other; a third example includes one of the first and second release areas being set in either the first or second window, and the other being set in any other area besides the first release area; a fourth example includes one of the first and second release areas being set in the boundary area of ​​the graphical user interface, and the other being set in any other area besides the boundary area.

[0061] The first and second release areas can be displayed as optional areas or not in the graphical user interface.

[0062] For example, such as Figure 4 As shown, the second release area is set in the upper area of ​​the second window, and the first release area is any area in the graphical user interface other than the second release area. When the target first partition 411 in the first window (such as the fifth example window 410) is continuously pressed by the first press operation, the second window (such as the sixth example window 420) continuously displays the first page 421; then, a first drag operation is performed on the target first partition 411, and when the first release position of the first drag operation is the second release area 422, the first page 421 continues to be displayed in the second window (such as the sixth example window 420).

[0063] In one implementation, in response to a second interactive operation on a target second partition in the second window, generating a second page based on a target time node of the target second partition and updating it to the second window includes: in response to a second press operation on the target second partition in the second window, obtaining first medical information corresponding to a target time node associated with the target second partition; generating a second page based on the first medical information corresponding to the target time node; and updating the displayed content in the second window according to the second page during the continuous pressing process of the second press operation.

[0064] In this embodiment, the display content of each second partition of the second window includes key medical information or detailed medical information in the medical information, or the display content corresponding to the first medical category in a part of the multiple second partitions includes key medical information, and the other part includes detailed medical information.

[0065] In this embodiment, the target second partition is the second partition corresponding to the pressing area of ​​the second pressing operation. Based on the target second partition, the detailed medical information corresponding to the target time node is queried to obtain the first medical information, and the display content of the target second partition is updated to generate the second page.

[0066] Furthermore, the second page is also used to adjust the display style of the target second partition, wherein the display style includes at least one of the size of the target second partition and a highlighting effect, wherein the size is greater than the size of the target second partition before adjustment and less than or equal to the size of the second window.

[0067] The second page may include only the content of the target second partition, or include the content of all second partitions. In the former case, the size of the target second partition is adjusted to the size of the second window, while in the latter case, the size of the target second partition is enlarged without covering other second partitions, and / or a highlighting effect is added to the target second partition.

[0068] For example, the second page is also used to adjust the display style of the target second partition, including adjusting the size of the target second partition to the size of the second window. For example... Figure 5 As shown, the current display page of the second window (such as the seventh example window 510) (such as the second example page 511) displays key diagnostic and treatment information for all diagnostic and treatment categories; in response to the second press operation on the target second partition (such as the sixth example partition 512) in the second window, the first diagnostic and treatment information corresponding to the target time node associated with the target second partition (such as the sixth example partition 512) is obtained. The first diagnostic and treatment information includes detailed diagnostic and treatment information of the target time node, specifically including text content 513 and medical images (such as the first example medical image 514), wherein the medical image is the first sorted medical image among a plurality of medical images in a preset sort or a preset sorted medical image; a second page (such as the third example page 515) is generated based on the text content 513 and the first example medical image 514.

[0069] Specifically, while the second press operation is continuously pressed, the second page (e.g., third example page 515) remains displayed in the second window; when the first press operation is canceled, the second page (e.g., third example page 515) displayed in the second window is switched back to the currently displayed page (e.g., second example page 511). The second page (e.g., third example page 515) is only used to illustrate the display positions of text content 513 and the first example medical image 514, and may include a rectangular border to distinguish the two, or may not include a rectangular border.

[0070] Alternatively, the second page may consist of only text content 513 or only the first example medical image 514.

[0071] In addition to using the continuous pressing and canceling of the second pressing operation to switch between the currently displayed page and the second page, the second interactive operation also includes a click operation. Performing a click operation switches the currently displayed page to the second page, and performing a click operation again switches the second page back to the currently displayed page, and so on.

[0072] In one embodiment, during the continuous pressing process of the second pressing operation, after updating the display content in the second window according to the second page, the method further includes: in response to the second drag operation on the target second partition, determining the second release position of the second drag operation, wherein the second pressing operation and the second drag operation are consecutive operations, and during the drag operation, the second page remains unchanged in the second window; if the second release position is in a third release area, then the second window is switched from the second page to the display content before the second page update; if the second release position is in a fourth release area, then the second page is continuously displayed in the second window.

[0073] In this embodiment, the second drag operation is performed during the continuous pressing process after the second press operation, and the content displayed in the second window will not change with the second drag operation; when the second drag operation is canceled, the display of the second page or the content displayed before the second page is updated is determined according to the second release position of the second drag operation.

[0074] Specifically, the third and fourth release areas do not overlap and can be set at any location in the graphical user interface. The first example includes both the third and fourth release areas being set in the first window or both in the second window; the second example includes the third release area being set in one of the first and second windows, and the fourth release area being set in the other; the third example includes one of the third and fourth release areas being set in either the first or second window, and the other being set in any other area besides the third release area. The fourth example includes one of the third and fourth release areas being set in the boundary area of ​​the graphical user interface, and the other being set in any other area besides the boundary area.

[0075] The third and fourth release areas are either optional or not displayed in the graphical user interface.

[0076] For example, the fourth release area includes the first window (such as the eighth example window 620), at which point the first partition (such as the seventh example partition 621) is in an unselectable state. Figure 6As shown, when a second press operation is performed on the target second partition (e.g., the eighth example partition 611) in the second window (e.g., the fourth example page 612), the currently displayed page of the second window (e.g., the fourth example page 612) is switched to the second page (e.g., the fifth example page 613), and then a second drag operation is performed on the target second partition (e.g., the eighth example partition 611); when the second release position of the second drag operation is the third release area (e.g., the middle position 614), the second page displayed in the second window (e.g., the fifth example page 613) is switched back to the currently displayed page (e.g., the fourth example page 612); when the second drag operation passes the middle position 614 and further reaches the fourth release area for release, the second page (e.g., the fifth example page 613) continues to be displayed in the second window; wherein, the fourth release area is any other position in the graphical user interface outside the third release area.

[0077] In one implementation, the content displayed on the second page includes at least one optional content. After generating the second page based on the target time node of the target second partition and updating the second window in response to a second interactive operation on the target second partition in the second window, the method further includes: in response to a selection operation on any optional content, obtaining reference content associated with the optional content from the second medical information corresponding to the reference node, wherein the reference node is a non-target time node among the plurality of time nodes; and updating the second page based on the reference content.

[0078] In this embodiment, the optional content includes terms such as hospital name, department name, doctor's name, lesion location, disease name, treatment effect, examination item name, examination result parameters, and local lesion area in medical images. Reference content refers to terms of the same type from the aforementioned optional content.

[0079] Updating the second page based on the reference content includes adding the reference content to the second page. In response to the selection of any optional content, the selected term is determined. Based on the term, reference content is obtained by querying the second diagnostic information for terms of the same type through various reference nodes. The second page is then generated based on the same type of terms for both the target time node and non-target time nodes. The second page also includes time information for each time node to distinguish the terms for each time node.

[0080] For example, the time information is formatted as "xxxx year xx month xx day". The selected option points to the test result parameters, specifically including any one or more of the following: red blood cell count, hemoglobin, hematocrit, white blood cell count, white blood cell differential count, and platelet count. The updated content displayed on the second page includes:

[0081] On January MM1, YYY1: Red blood cell count A1 / L, hemoglobin B1 g / L, hematocrit C1%, white blood cell count E1 / L, differential white blood cell count F1%, platelet count G1 / L;

[0082] On January MM2, YYY2: Red blood cell count A2 / L, hemoglobin B2 g / L, hematocrit C2%, white blood cell count E2 / L, differential white blood cell count F2%, platelet count G2 / L;

[0083] On January MM3, YYY3: Red blood cell count A3 / L, hemoglobin B3 g / L, hematocrit C3%, white blood cell count E3 / L, differential white blood cell count F3%, platelet count G3 / L;

[0084] On January MM4, YYY4: Red blood cell count A4 / L, hemoglobin B4 g / L, hematocrit C4%, white blood cell count E4 / L, differential white blood cell count F4%, platelet count G4 / L.

[0085] Among them, "January MM1, YYY1" is the time information corresponding to the target time node, and "January MM2, YYY2", "January MM3, YYY3", and "January MM4, YYY4" are the time information corresponding to the reference nodes. "Red blood cell count A1 / L, hemoglobin B1 g / L, hematocrit C1%, white blood cell count E1 / L, differential white blood cell count F1%, platelet count G1 / L" is the selectable content pointed to by the selected operation, and "Red blood cell count A2 / L, hemoglobin B2 g / L, hematocrit C2%, white blood cell count E2 / L, differential white blood cell count F2%, platelet count G2 / L", "Red blood cell count A3 / L, hemoglobin B3 g / L, hematocrit C3%, white blood cell count E3 / L, differential white blood cell count F3%, platelet count G3 / L", and "Red blood cell count A4 / L, hemoglobin B4 g / L, hematocrit C4%, white blood cell count E4 / L, differential white blood cell count F4%, platelet count G4 / L" are the reference contents.

[0086] In one implementation, in response to a third drag operation on any one of the first partitions, determine the third release position of the third drag operation; if the third release position is the position of any one of the second partitions, determine the third diagnosis and treatment category of the any one of the first partitions; based on the third diagnosis and treatment category, adjust the display style and display content of the any one of the second partitions, and update them to the second window.

[0087] In this embodiment, the selected third treatment category is determined based on the third drag operation on any second partition, and the selected time node is determined based on any second partition where the third release position is located. The treatment information corresponding to the third treatment category is queried through the selected time node, and the display content of the selected second partition is replaced or added according to the queried treatment information. At the same time, the display style of the second partition is adjusted by highlighting and / or enlarging the size of the second partition.

[0088] For example, such as Figure 7 As shown, select the first partition (e.g., the ninth example partition 711) in the first window (e.g., the tenth example window 710) and drag it to the second partition (e.g., the tenth example partition 721) in the second window (e.g., the eleventh example window 720); update the display content of the second partition and enlarge the size of the second partition to obtain a new partition 722, and update the new partition 722 to the second window.

[0089] In one embodiment, the content displayed on the second page includes a first medical image; the visualization interaction method further includes: in response to a first sliding operation on the first medical image along a preset first direction, determining a second medical image according to the preset first direction, and updating the display content in the second window according to the second medical image, wherein the second medical image is another medical image besides the first medical image among a plurality of medical images corresponding to the target time node; or, in response to a second sliding operation on the second medical image along a preset second direction, determining a third medical image according to the preset second direction, and updating the display content in the second window according to the third medical image, wherein the third medical image is a medical image corresponding to a non-target time node among the plurality of time nodes.

[0090] The medical information includes multiple medical images in a preset order. When the second page only includes the first medical image, a first sliding operation is performed on the first medical image along a preset first direction. The medical image above or below the preset order is queried through the target time node to obtain the second medical image. When a second sliding operation is performed on the first medical image along a preset second direction, the previous or next sorting time node of the target time node is determined. The medical image in the first order of the preset order or other preset sorting order is queried through the previous or next sorting time node to obtain the third medical image.

[0091] In one embodiment, the preset first direction and the preset second direction are directions from the center region of the first medical image toward the edge region; updating the display content in the second window according to the second medical image includes: switching the first medical image displayed in the second window to the second medical image; and updating the display content in the second window according to the third medical image includes: switching the first medical image displayed in the second window to the third medical image.

[0092] In this embodiment, in one of the preset first directions and the preset second directions, the angle between the direction from the center region to the edge region and the vertical direction is smaller than the preset first angle, and the angle between the direction from the center region to the edge region and the horizontal direction is smaller than the preset second angle. The sum of the first angle and the second angle is further set to 90°, for example, the preset first angle is 30° and the preset second angle is 60°, or the preset first angle is 45° and the preset second angle is 45°.

[0093] For example, such as Figure 8 As shown, the preset first direction includes a first direction (such as the first example direction 831) and a second direction (such as the second example direction 832), and the preset second direction includes a third direction 821 and a fourth direction 822; the second window (such as the twelfth example window 810) displays a first medical image (such as the second example medical image 811); when a first sliding operation is performed on the first medical image along the first direction (such as the first example direction 831), the first medical image displayed in the second window is switched to the previous sorted medical image among a plurality of medical images in a preset order; when a first sliding operation is performed on the first medical image along the second direction (such as the second example direction 832), the first medical image displayed in the second window is switched to the next sorted medical image among a plurality of medical images in a preset order; when a second sliding operation is performed on the first medical image along the third direction 821, the first medical image displayed in the second window is switched to the medical image corresponding to the previous sorted time node; when a second sliding operation is performed on the first medical image along the second direction 822, the first medical image displayed in the second window is switched to the medical image corresponding to the next sorted time node.

[0094] In one embodiment, the preset first direction and the preset second direction are the directions from the edge region of the first medical image toward the center region; updating the display content in the second window according to the second medical image includes: simultaneously displaying the first medical image and the second medical image in the second window; and updating the display content in the second window according to the third medical image includes: simultaneously displaying the first medical image and the third medical image in the second window.

[0095] In this embodiment, in the preset first direction and the preset second direction, the angle between the direction from the edge region to the center region of one of them and the vertical direction is smaller than the preset first angle, and the angle between the direction from the edge region to the center region of the other and the horizontal direction is smaller than the preset second angle. The sum of the first angle and the second angle is further set to 90°, for example, the preset first angle is 30° and the preset second angle is 60°, or the preset first angle is 45° and the preset second angle is 45°.

[0096] For example, such as Figure 9 As shown, the preset first direction includes a first direction (such as the third example direction 921) and a second direction (such as the fourth example direction 922); the second window (such as the thirteenth example window 910) displays the first medical image (such as the third example medical image 911); when the second sliding operation is performed on the first medical image along the first direction (such as the third example direction 921), the second window displays the third example medical image 911 (displayed in the left area 931 of the first example) and the medical image corresponding to the previous sorting time node (displayed in the right area 932 of the first example); when the second sliding operation is performed on the first medical image along the second direction (such as the fourth example direction 922), the second window displays the third example medical image 911 (displayed in the left area 931 of the first example) and the medical image corresponding to the next sorting time node (displayed in the right area 932 of the first example).

[0097] The original display content 941 of the first window (such as the fourteenth example window 940) includes the diagnosis and treatment category entries displayed in each second partition. After updating the display content in the second window according to the third medical image, the original display content 941 of the first window is switched to the new display content 942. The new display content 942 includes the time information corresponding to the time node displayed in each second partition.

[0098] Simultaneously, the newly displayed content 942 includes a highlight effect for the second partition corresponding to the time nodes of the first and third medical images, to indicate the year information corresponding to the first and third medical images after the second window updates its display content. In response to interactive operations on other second partitions in the newly displayed content 942, the third medical image at the current time node is switched to a medical image at the time node corresponding to another second partition; wherein, the other second partitions are the second partitions corresponding to time nodes other than the time node corresponding to the third medical image.

[0099] Provide another example, such as Figure 10As shown, the preset first direction includes a first direction (such as the fifth example direction 1021) and a second direction (such as the sixth example direction 1022); the second window (such as the fifteenth example window 1010) displays a first medical image (such as the fourth example medical image 1011); when a first sliding operation is performed on the first medical image along the first direction (such as the fifth example direction 1021), the second window displays the fourth example medical image 1011 distributed horizontally (displayed in the left area 1031 of the second example) and the previous medical image in the preset sorted plurality of medical images (displayed in the right area 1032 of the second example); when a first sliding operation is performed on the first medical image along the second direction (such as the sixth example direction 1022), the second window displays the fourth example medical image 1011 distributed horizontally (displayed in the left area 1031 of the second example) and the next medical image in the preset sorted plurality of medical images (displayed in the right area 1032 of the second example).

[0100] In one implementation, the second page includes text content and medical images, and the visualization interaction method further includes: in response to a selection operation of the second partition corresponding to the medical image category in the first window, switching the display content of the second page to contain only medical images.

[0101] For example, such as Figure 11 As shown, the second window (such as the sixteenth example window 1110) displays text content 1111 and medical images (such as the fifth example medical image 1112). In response to the selection operation of the second partition (such as the eleventh example partition 1121) corresponding to the medical image category in the first window (such as the seventeenth example window 1120), the sixteenth example window 1110 only displays the fifth example medical image 1112.

[0102] In one implementation, the second window displays either the original page or the first page, including multiple second sections. The visualization interaction method further includes: responding to a drag operation on a target second section in the second window, and controlling the release position of the drag operation to the first section corresponding to the medical image category in the first window; querying medical images through the time node corresponding to the target second section; and generating a second page based on the queried medical images, i.e., the second page contains only medical images.

[0103] For example, such as Figure 12As shown, the second window (such as the eighteenth example window 1210) displays multiple second partitions. In response to the drag operation of the target second partition (such as the twelfth example partition 1211) in the second window, the release position of the drag operation is controlled to be the first partition (such as the thirteenth example partition 1221) corresponding to the medical image category in the first window (such as the nineteenth example window 1220). The medical image (such as the sixth example medical image 1212) is obtained by querying the time node corresponding to the target second partition (such as the twelfth example partition 1211). The second page (such as the sixth example page 1213) is generated based on the queried sixth example medical image 1212. That is, the second page (such as the sixth example page 1213) only contains the sixth example medical image 1212.

[0104] Figure 13 This is a schematic diagram of a computer device provided in an embodiment of the present invention, used to execute the aforementioned embodiments of a visualization and interactive method for multidisciplinary collaborative diagnosis and treatment. The computer device 1300 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1310 (e.g., one or more processors) and a memory 1320, and one or more storage media 1330 (e.g., one or more mass storage devices) storing application programs 1333 or data 1332. The memory 1320 and storage media 1330 can be temporary or persistent storage. The program stored in the storage media 1330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the computer device 1300. Furthermore, the processor 1310 may be configured to communicate with the storage media 1330 and execute the series of instruction operations in the storage media 1330 on the computer device 1300.

[0105] Computer device 1300 may also include one or more power supplies 1340, one or more wired or wireless network interfaces 1350, one or more input / output interfaces 1360, and / or one or more operating systems 1331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 13 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0106] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visual interactive method for multidisciplinary collaborative diagnosis and treatment, characterized in that, The system includes a graphical user interface (GUI) providing a first window and a second window. The first window displays multiple first sections; the second window displays a timeline and multiple second sections. The timeline includes multiple sequentially distributed time nodes, and the multiple second sections are associated with each sequentially distributed time node in the timeline. The multiple first sections correspond to different medical treatment categories, and the content displayed in the multiple second sections includes at least a portion of the medical information corresponding to the associated time nodes that is related to one or more medical treatment categories. The method includes: In response to a first interactive operation on a target first partition in the first window, at least some time nodes are determined, a first page is generated based on the at least some time nodes and updated to the second window, wherein the first page is used to adjust the display content of the second partition associated with the at least some time nodes; In response to a second interactive operation on a target second partition in the second window, a second page is generated based on the target time node of the target second partition and updated to the second window, wherein the second page is used to adjust the display content of the target second partition.

2. The visual interaction method according to claim 1, characterized in that, The display content of the plurality of second partitions includes display content related to the first diagnosis and treatment category. The display content of the second window also includes display content related to the first diagnosis and treatment category in the plurality of second partitions. In response to a first interactive operation on the target first partition in the first window, at least some time nodes are determined, and a first page is generated and updated to the second window based on the at least some time nodes, including: In response to a first press operation on a target first partition in the first window, a second treatment category of the target first partition is determined, and at least some time nodes are selected from the plurality of time nodes, wherein the at least some time nodes are associated with the second treatment category; Adjust the display content related to the first treatment category in the second partition associated with at least some time nodes to display content related to the second treatment category, and generate the first page; During the continuous pressing process of the first pressing operation, the displayed content in the second window is updated according to the first page.

3. The visual interaction method according to claim 2, characterized in that, During the continuous pressing process of the first pressing operation, after updating the displayed content in the second window according to the first page, the method further includes: In response to a first drag operation on the target first partition, a first release position of the first drag operation is determined, wherein the first press operation and the first drag operation are consecutive operations; If the first release position is within the first release area, then the second window will be switched from the first page to the content displayed before the first page was updated; If the first release position is within the second release area, the first page will continue to be displayed in the second window.

4. The visual interaction method according to claim 1, characterized in that, In response to a second interactive operation on a target second partition in the second window, a second page is generated based on the target time node of the target second partition and updated to the second window, including: In response to a second press operation on the target second partition in the second window, first diagnostic information corresponding to the target time node associated with the target second partition is obtained; A second page is generated based on the first medical information corresponding to the target time point; During the continuous pressing process of the second pressing operation, the displayed content in the second window is updated according to the second page.

5. The visual interaction method according to claim 4, characterized in that, During the sustained pressing process of the second pressing operation, after updating the displayed content in the second window according to the second page, the following is also included: In response to a second drag operation on the target second partition, a second release position of the second drag operation is determined, wherein the second press operation and the second drag operation are consecutive operations, and the second page remains unchanged in the second window during the drag operation; If the second release position is in the third release area, then the second window will be switched from the second page to the content displayed before the second page was updated; If the second release position is in the fourth release area, the second page will continue to be displayed in the second window.

6. The visual interaction method according to claim 4, characterized in that, The second page displays content including at least one optional element. In response to a second interactive operation on a target second partition within the second window, after generating the second page based on the target time node of the target second partition and updating it to the second window, the page further includes: In response to the selection operation of any optional content, the reference content associated with the optional content is obtained from the second diagnosis and treatment information corresponding to the reference node, wherein the reference node is a non-target time node among the plurality of time nodes; The second page is updated based on the reference content.

7. The visualization and interaction method according to any one of claims 1-6, characterized in that, The second page is also used to adjust the display style of the target second partition, wherein the display style includes at least one of the size of the target second partition and a highlight effect, and the size is greater than the size of the target second partition before adjustment and less than or equal to the size of the second window.

8. The visual interaction method according to claim 7, characterized in that, The visualization interaction method also includes: In response to a third drag operation on any of the first partitions, determine the third release position of the third drag operation; If the third release position is the location of any second partition, determine the third treatment category of any first partition; Based on the third diagnosis and treatment category, adjust the display style and content of any one of the second partitions and update it to the second window.

9. The visual interaction method according to claim 1, characterized in that, The diagnosis and treatment categories corresponding to the multiple first partitions include: hospital category, department category, doctor category, diagnosis category, indicator category, and medical imaging category.

10. The visual interaction method according to claim 9, characterized in that, The content displayed on the second page includes the first medical image; the visualization interaction method further includes: In response to a first sliding operation on the first medical image along a preset first direction, a second medical image is determined according to the preset first direction, and the display content in the second window is updated according to the second medical image, wherein the second medical image is another medical image besides the first medical image among multiple medical images corresponding to the target time node; or... In response to a second sliding operation on the first medical image along a preset second direction, a third medical image is determined according to the preset second direction, and the display content in the second window is updated according to the third medical image, wherein the third medical image is the medical image corresponding to a non-target time node among the plurality of time nodes.

11. The visual interaction method according to claim 10, characterized in that, The preset first direction and the preset second direction are the directions from the center region of the first medical image toward the edge region; Updating the display content in the second window based on the second medical image includes: switching the first medical image displayed in the second window to the second medical image; and... Updating the display content in the second window based on the third medical image includes: switching the first medical image displayed in the second window to the third medical image.

12. The visual interaction method according to claim 10, characterized in that, The preset first direction and the preset second direction are the directions from the edge region of the first medical image toward the center region; Updating the display content in the second window based on the second medical image includes: simultaneously displaying the first medical image and the second medical image in the second window; and, Updating the display content in the second window based on the third medical image includes simultaneously displaying the first medical image and the third medical image in the second window.

13. A computer device, characterized in that, The computer device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the computer device to perform the steps of the visualization interaction method for multidisciplinary collaborative diagnosis and treatment as described in any one of claims 1-12.

14. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the visualization interaction method for multidisciplinary collaborative diagnosis and treatment as described in any one of claims 1-12.

Images