Display system and display method

Abstract

To provide a display control system and display method that allows users to easily select image processing for appropriate analysis. [Solution] The display system according to this disclosure includes an acquisition unit 111 that acquires image data of a sample to be analyzed, an image processing unit 112 that performs image processing on the image data, and a display control unit 113 that displays an image display area divided into a first area and a second area by a boundary line. The display control unit 113 displays the boundary line so that it can be moved by user operation, displays the portion of the image data before image processing is performed that corresponds to the position of the first area in the first area, and displays the portion of the image data after image processing is performed that corresponds to the position of the second area in the second area.

Description

Technical Field

[0001] The present disclosure relates to a display system and a display method.

Background Art

[0002] Patent Document 1 describes an image display system. When processing information is associated with an image to be displayed, the image display system described in Patent Document 1 displays the image after image processing after displaying the image before image processing.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, image processing may be used as a preprocessing for performing sample analysis using an image. In this case, in order to appropriately perform sample analysis, the user needs to appropriately select an image processing method.

[0005] For example, when adopting a configuration in which images before and after image processing are displayed in time series as in the image display system described in Patent Document 1, it is difficult for the user to understand the changes due to image processing and to select appropriate image processing.

[0006] That is, in a scene where sample analysis using an image is performed, there is a problem that it is difficult for the user to easily select image processing for appropriately performing the analysis because it is difficult for the user to actually feel the effect of image processing as a preprocessing.

[0007] This disclosure was made to solve these problems and aims to provide a display control system and display method that allows users to easily select image processing to perform appropriate analysis. [Means for solving the problem]

[0008] The display system according to this disclosure comprises an acquisition unit that acquires image data of a sample to be analyzed, an image processing unit that performs image processing on the image data, and a display control unit that displays an image display area divided into a first area and a second area by a boundary line. The display control unit displays the boundary line so that it can be moved by user operation, displays the portion of the image data before image processing is performed that corresponds to the position of the first area in the first area, and displays the portion of the image data after image processing is performed that corresponds to the position of the second area in the second area.

[0009] With this configuration, the display system relating to this disclosure allows users to easily see changes in the image being analyzed through image processing. As a result, the display system relating to this disclosure allows users to easily select the appropriate image processing for performing the analysis.

[0010] In the display system relating to this disclosure, the image processing unit may sequentially execute multiple image processing operations on the image data. The display control unit may then display in the first region the portion of the image data corresponding to the position in the first region before the image processing operation selected by the user is performed, and in the second region the portion of the image data corresponding to the position in the second region after the image processing operation selected by the user is performed.

[0011] In the display system relating to this disclosure, the display control unit may display identification images corresponding to each of the multiple image processing operations in parallel, and the preprocessing corresponding to the identification image selected by the user may be considered as the image processing operation selected by the user.

[0012] In the display system according to this disclosure, the display control unit may further display a switching image near the image display area for switching the image data to be displayed in the image identification area.

[0013] The display method relating to this disclosure acquires image data of a sample to be analyzed, performs image processing on the image data, and displays an image display area that is divided into a first area and a second area by a boundary line. The image display area is displayed such that the boundary line can be moved by user operation, and the portion of the image data before image processing is performed that corresponds to the position of the first area is displayed in the first area, and the portion of the image data after image processing is performed that corresponds to the position of the second area is displayed in the second area. [Effects of the Invention]

[0014] This disclosure provides a display control system and display method that allow users to easily select image processing for appropriate analysis. [Brief explanation of the drawing]

[0015] [Figure 1] This is a block diagram showing the configuration of the analysis system according to the first embodiment. [Figure 2] Block diagram showing the hardware configuration of the server according to the first embodiment. [Figure 3] This is a block diagram showing the functional configuration of the server according to the first embodiment. [Figure 4] This is a schematic diagram showing the configuration of the display control unit according to the first embodiment. [Figure 5] This is a schematic diagram showing the configuration of the display control unit according to the first embodiment. [Figure 6] This is a schematic diagram showing the configuration of the display control unit according to the first embodiment. [Figure 7] This is a schematic diagram showing the configuration of the display control unit according to the first embodiment. [Figure 8] This is a flowchart showing the operation of the server according to the first embodiment.

Embodiment for Implementing the Invention

[0016] <First Embodiment> (Configuration of the Display System) Hereinafter, the first embodiment according to the present disclosure will be described in detail with reference to the drawings. First, the configuration of the display system according to this embodiment will be described in detail.

[0017] The display system according to this embodiment is a system configured as a part of the analysis system according to this embodiment, and is a system for displaying to the user the preprocessing process for the analysis system according to this embodiment to execute the analysis. FIG. 1 is a block diagram showing the configuration of the analysis system according to the first embodiment. As shown in FIG. 1, in the analysis system 1 according to this embodiment, a server 100 and a user terminal 200 are connected via a network N such as the Internet.

[0018] The analysis system 1 is a system for analyzing analysis target data. The analysis system 1 according to this embodiment is typically provided as a part of a data cloud type service used in sample development and research and development, and is used as a system for promoting research and development using so-called materials informatics (MI, Materials Informatics) and data science. In this case, the analysis system 1 stores, for example, various measurement data of a newly developed sample. Then, the analysis system 1 analyzes the stored measurement data based on an instruction from the user.

[0019] In the analysis system 1, the user terminal 200 transmits image data to be analyzed to the server 100, and the server 100 analyzes the received image data. Then, the server 100 transmits the analysis result to the user terminal 200, and the user terminal 200 displays the received analysis result.

[0020] The user terminal 200 according to this embodiment is a terminal operated by a user, and is typically a computer device having a display device. The user terminal 200 transmits the data to be analyzed to the server 100 via the network N. The user terminal 200 then receives the analysis results of the data from the server 100 via the network N.

[0021] In this embodiment, the server 100 receives data to be analyzed from the user terminal 200 and analyzes the received data. The server 100 then transmits the analysis results to the user terminal 200 via the network N.

[0022] Figure 2 is a block diagram showing the hardware configuration of the server according to the first embodiment. As shown in Figure 2, the server 100 includes a processor 110, memory 120, storage device 130, input / output interface 140, network interface 150, and internal bus 160.

[0023] The internal bus 160 is a data transmission path for the processor 110, memory 120, storage device 130, input / output interface 140, and network interface 150 to send and receive data to and from each other. However, the method of connecting the processor 110 and the other components to each other is not limited to bus connection.

[0024] Memory 120 is the main memory, implemented using RAM (Random Access Memory), etc. Furthermore, the storage device 130 is an auxiliary storage device implemented using a hard disk, SSD (Solid State Drive), memory card, or ROM (Read Only Memory), etc. The storage device 130 stores a program for realizing a desired function.

[0025] The processor 110 is a variety of processor, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an FPGA (field-programmable gate array). The processor 110 reads the program stored in the storage device 130 into the memory 120 and executes it, thereby performing the functions of each functional block as shown in Figures 3 and 4, which will be described later.

[0026] The input / output interface 140 is an interface for connecting the server 100 with input / output devices. For example, input devices such as keyboards and output devices such as display devices may be connected to the input / output interface 140. The network interface 150 is an interface for connecting the server 100 to the network.

[0027] The program, when loaded into a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored in a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include RAM, ROM, flash memory, SSD or other memory technologies, CD-ROM, DVD (digital versatile disc), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include, a temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically, or otherwise propagating signals.

[0028] In this embodiment, before performing image data analysis, the server 100 performs various image processing operations on the image data as preprocessing based on instructions from the user. The display system according to this embodiment is a system for the user to appropriately select the above-mentioned image processing, and is implemented as part of the functions of server 100.

[0029] Figure 3 is a block diagram showing the functions of the server according to the first embodiment. As shown in Figure 3, the server 100 according to this embodiment includes an acquisition unit 111, an image processing unit 112, and a display control unit 113 as functional blocks.

[0030] The acquisition unit 111 acquires image data to be analyzed. More specifically, in this embodiment, the acquisition unit 111 acquires image data from the user terminal 200 via the network N. The acquisition unit 111 outputs the acquired image data to the image processing unit 112.

[0031] Here, the acquisition unit 111 may store the acquired image data in the storage device 130 when it receives image data from the user terminal 200. Then, when it receives a request from the user for a specific image data, it may read the specified image data from the storage device 130 and output it to the image processing unit 112. In other words, the acquisition unit 111 may create a database storing the image data. The acquisition unit 111 may also be configured so that the user can appropriately select the image data to be analyzed from the created database.

[0032] The image processing unit 112 performs image processing on the image data. Specifically, the image processing unit 112 acquires image data from the acquisition unit 111 and receives the type of image processing to be performed from the user. The image processing unit 112 then performs the image processing received from the user on the image data. The image processing unit 112 outputs the processed image data to the display control unit 113.

[0033] The image processing unit 112 may perform multiple image processing operations on the image data. In this case, the image processing unit 112 receives the multiple image processing operations and the execution order of the multiple image processing operations from the user. The image processing unit 112 then executes the multiple image processing operations sequentially on the image data according to the execution order received from the user.

[0034] The display control unit 113 controls the display screen of the user terminal 200. More specifically, the display control unit 113 displays an image processing selection screen for the user to select an image processing method, and an image processing confirmation screen for the user to check the effect of the image processing on the user terminal 200.

[0035] Figures 4 and 5 are schematic diagrams showing the configuration of the display control unit according to the first embodiment. More specifically, Figure 4 is a schematic diagram showing the image processing selection screen P1 that the display control unit 113 according to this embodiment displays on the user terminal 200, and Figure 5 is a schematic diagram showing the image processing confirmation screen P2 that the display control unit 113 according to this embodiment displays on the user terminal 200. The image processing selection screen P1 and the image processing confirmation screen P2 will be explained below using Figures 4 and 5.

[0036] As shown in Figure 4, the display control unit 113 displays an image processing selection screen P1 in which an identification image P11 representing the image data before image processing and identification images P12a, P12b, and P12c corresponding to each of the multiple image processing processes are displayed in parallel.

[0037] Note that while Figure 4 shows three image processing steps as an example, the number of image processing steps performed is, of course, not limited to three. Furthermore, unless there is a particular need to distinguish them, in the following specification, the identification images P12a, P12b, and P12c will simply be referred to as identification image P12. The same applies to the identification name P121, image processing conditions P122, sample image P123, and switching image P226 described later.

[0038] The identification image P11 represents the image data before image processing. The identification image P11 displays the identification name P111 and image information P112 as text. Additionally, the sample image P113 is displayed on the identification image P11.

[0039] The identification name P111 is text indicating that the identified image P11 is associated with the image data before image processing. For example, in Figure 4, the text "Input Image" is displayed as the identification name P111 to indicate that the identified image P11 is associated with the image data before image processing. The identification name P111 may also be, for example, the specific name of the image data before image processing.

[0040] Image information P112 is text that represents information about the image data before image processing. Image information P112 may display various information, such as the data size of the image data, the size of the image, the date and time the user saved the image data, and user comments.

[0041] Sample image P113 displays the image before image processing. Sample image P113 may display a portion of the image before processing, or it may display the entire image before processing.

[0042] The image processing identification image P12 is an image that identifies the image processing to be performed on the image data. The identification image P12 displays the identification name P121 and the image processing conditions P122 as text. Additionally, the sample image P123 is displayed on the identification image P12.

[0043] The identification name P121 is text that indicates which image processing operation the identified image P12 is associated with. For example, in Figure 4, displaying the text "Image Processing 1" as the identification name P121 indicates that the identified image P12 is associated with the image processing operation named "Image Processing 1". The identification name P121 may also contain the specific name of the image processing operation, such as "Noise Reduction" or "Automatic Contrast Adjustment".

[0044] Image processing conditions P122 contains text indicating the conditions to be referenced when performing image processing. For example, the parameter values ​​required to perform image processing may be displayed in image processing conditions P122.

[0045] Sample image P123 displays the image after the corresponding image processing has been performed. Sample image P123 may display a portion of the processed image, or it may display the entire processed image.

[0046] Here, the identified images P12 are displayed in parallel according to the order in which the corresponding image processing is executed. In other words, in the case shown in Figure 4, the image processing corresponding to identified image P12a is executed first, the image processing corresponding to identified image P12b is executed second, and the image processing corresponding to identified image P12c is executed third.

[0047] Furthermore, the image processing selection screen P1 is configured to be editable by the user. For example, the image processing selection screen P1 is configured to allow the user to edit the type of image processing to be performed, the execution order of the image processing, etc. In this case, the user performs editing operations on the image processing selection screen P1 so that the identified images P12 associated with the desired image processing are displayed in the desired execution order. Then, the image processing unit 112 performs the image processing by referring to the image processing selection screen P1 edited by the user.

[0048] Here, if any of the identification images P12 are selected by the user, the display control unit 113 transitions the screen displayed on the user terminal 200 to the image processing confirmation screen P2. In the following description, assuming the user has selected the identified image P12b on the image processing selection screen P1 shown in Figure 4, the image processing confirmation screen P2 will be explained in detail.

[0049] As shown in Figure 5, the display control unit 113 displays a condition display area P21 that displays the image processing conditions and an image display area P22 that displays the images before and after image processing in parallel on the image processing confirmation screen P2.

[0050] The condition display area P21 contains text indicating the conditions to be referenced when performing image processing. The conditions listed in the condition display area P21 are the same as the conditions listed in the image processing conditions P122 shown in Figure 4. Here, the condition display area P21 is configured to be editable by the user. When the condition display area P21 is edited, the image processing unit 112 performs image processing based on the edited conditions.

[0051] The display control unit 113 displays images in the image display area P22 before and after the execution of the image processing selected by the user. In the example shown in Figure 5, the display control unit 113 displays images before and after the execution of "Image Processing 2," which corresponds to the identification image P12b described in Figure 4. In other words, the display control unit 113 considers "Image Processing 2," which corresponds to the identification image P12b selected by the user, to be the image processing selected by the user and displays the image in the image display area P22.

[0052] Here, the image display area P22 is divided into a first area P221 and a second area P222 by the boundary line P223. The display control unit 113 displays the image data before image processing in the first area P221. The display control unit 113 also displays the image data after image processing in the second area P222.

[0053] More specifically, the display control unit 113 displays the portion of the image data before image processing is performed that corresponds to the position in the first region P221 in the first region P221. Then, the display control unit 113 displays the portion of the image data after image processing that corresponds to the position in the second region P222 in the second region P222. Furthermore, in this embodiment, the display control unit 113 displays the boundary line P223 in such a way that it can be moved by user operation.

[0054] However, the "portion of the image data before image processing is performed that corresponds to the position of the first region P221" as used here refers to the portion of the image data that overlaps with the first region P221 when the "image data before image processing is performed" is enlarged or reduced to the same size as the image display region P22 and superimposed on the image display region P22. Furthermore, the term "the portion of the image data after image processing that corresponds to the position of the second region P222" refers to the portion of the image data that overlaps with the second region P222 when the "image data after image processing" is enlarged or reduced to the same size as the image display region P22 and superimposed on the image display region P22.

[0055] The following describes the change in the display when the boundary line P223 is moved, using Figures 6 and 7. Figures 6 and 7 show the change in the display when the boundary line P223 is moved from the position of boundary line P223b before the move to the position of boundary line P223a after the move. Figure 6 shows the display before moving boundary line P223, and Figure 7 shows the display after moving boundary line P223.

[0056] It should be noted that the direction in which the boundary line P223 is moved is merely an example and is not limited to the directions shown in Figures 6 and 7. The boundary line P223 can be freely moved within the image area P22 by user operation.

[0057] As shown in Figures 6 and 7, moving the boundary line P223 gradually changes the size of the first region P221 and the second region P222. In other words, as the boundary line P223 moves, the area that displayed the image data after image processing gradually changes to the area that displayed the image data before image processing.

[0058] With this configuration, users can visually and easily confirm the changes in image data due to image processing. As a result, the server 100 according to this embodiment allows the user to easily select the appropriate image processing for performing the analysis.

[0059] Let's return to the explanation of Figure 5. In this embodiment, the display control unit 113 further displays the marker images P224 and P225 in the vicinity of the image display area P22.

[0060] The labeling image P224 is used to label the image displayed in the first region P221 as the image before the image processing selected by the user is performed. In this embodiment, the text "Before Application (No. 2)" is displayed as the label image P224. The string "(No2)" in the text indicates that the image selected by the user before image processing is the image data after image processing corresponding to the identified image P12a in Figure 4.

[0061] The labeling image P225 is used to label the image displayed in the second region P222 as the image after performing the image processing selected by the user. In this embodiment, the text "After application" is displayed as the label image P225.

[0062] Furthermore, the display control unit 113 according to this embodiment displays a switching image P226 in the vicinity of the image display area P22 for switching the image data to be displayed in the image identification area P22. If the user selects the switching image P226, the display control unit 113 switches the image processing selected by the user to an image processing that is one step before or one step after the currently selected image processing in the execution order.

[0063] In this embodiment, when the user selects a switching image P226, the image data displayed in the first area P221 is switched, while the image data displayed in the second area P222 is not switched. More specifically, when the switching image P226 is selected, the display control unit 113 switches the image data to be displayed in the first area P221 between the image data corresponding to the identification image P11 and the image data after an image processing that has been performed earlier in the execution order than the image processing selected by the user.

[0064] In other words, as shown in Figure 5, when an image corresponding to the identification image P12b is displayed in the second region P222, the display control unit P113 can switch between displaying the input image P11 corresponding to the identification image P11 and the image data after the identification image P12a has been processed. More specifically, if the image displayed in the first area P221 is the input image corresponding to the identification image P11, and the user selects the switching image P226R, the image displayed in the first area P221 will switch to the image data after image processing corresponding to the identification image P12a has been performed. Furthermore, if the image displayed in the first area P221 is image data after performing image processing corresponding to the identification image P12a, and the user selects the switching image P226L, the image displayed in the first area P221 will switch to the input image corresponding to the identification image P11. Note that the image data after performing image processing corresponding to identification image P12a is identical to the image data before performing image processing corresponding to identification image P12b.

[0065] This configuration allows users to easily see the effects of combining multiple image processing techniques. As a result, users can more easily select the appropriate image processing technique.

[0066] (Display system operation) Next, the operation of the display system, that is, the display method according to the first embodiment, will be described in detail. Figure 8 is a flowchart showing the operation of the server according to the first embodiment.

[0067] In the processing procedure shown in Figure 8, the processor 110 of the server 100 functions as an acquisition unit 111, an image processing unit 112, and a display control unit 113 by reading the program stored in the storage device 130 into the memory 120 and executing it.

[0068] In the display method according to this embodiment, first, the processor 110 acquires image data (step ST1). That is, in step ST1, the processor 110 functions as an acquisition unit 111. For example, in step ST1, the processor 110 receives a selection of data to be analyzed from the user terminal 200. The processor 110 then retrieves the selected data from the storage device 130.

[0069] Next, the processor 110 displays the image processing selection screen P1 (step ST2). In other words, in step ST2, the processor 110 functions as the display control unit 113. More specifically, in step ST2, the processor 110 displays an image processing selection screen as shown in Figure 4. On the displayed image processing selection screen, the processor 110 accepts edits from the user regarding the type of image processing to be performed and the execution order of the image processing.

[0070] Next, the processor 110 performs image processing on the image data (step ST3). In other words, in step ST3, the processor 110 functions as an image processing unit 112. More specifically, in step ST3, the processor 110 performs image processing on the image data based on the type of image processing to be performed and the execution order of the image processing, which were edited in step ST2.

[0071] Steps ST2 and ST3 may be executed in parallel. That is, the processor 110 may perform image processing each time the user edits the image processing selection screen P1, and reflect the results of the image processing in the sample image P123 each time.

[0072] Next, the processor 110 determines whether the user has selected the image P12 for image processing identification (step ST4). In other words, in step ST4, the processor 110 is functioning as the display control unit 113. If the user has not selected an image P12 for image processing identification (step ST4 No), the processor 110 repeats step ST4. In other words, the processor 110 waits without executing the next step until the user selects an image P12 for image processing identification.

[0073] If the user selects the image identification image P12 for image processing (step ST4 YES), the processor 110 displays the image processing confirmation screen P2 (step ST5), and the server 100 terminates the series of operations. In step ST5, the processor 110 is functioning as the display control unit 113. More specifically, in step ST3, the processor 110 displays an image processing confirmation screen P2, which includes an image display area P22 divided into a first area P221 and a second area P222 at the boundary line P223. As shown in Figure 5, the image display area P22 is displayed such that the boundary line P223 can be moved by user operation. In addition, the first area P221 displays the portion of the image data before image processing is performed that corresponds to the position of the first area P221, and the second area P222 displays the portion of the image data after image processing is performed that corresponds to the position of the second area P222.

[0074] As described above, the server 100, i.e., the display system according to this embodiment, displays an image display area P22 which is divided into a first area P221 and a second area P222 at the boundary line P223. Furthermore, the first area P221 displays the portion of the image data before image processing is performed that corresponds to the position of the first area P221, and the second area P222 displays the portion of the image data after image processing is performed that corresponds to the position of the second area P222. With this configuration, users can visually and easily confirm the changes in image data due to image processing. As a result, the server 100 according to this embodiment allows the user to easily select the appropriate image processing for performing the analysis.

[0075] Although the present invention has been described above in reference to the embodiments described above, the present invention is not limited to the configuration of the embodiments described above, and of course includes various modifications, alterations, and combinations that can be made by a person skilled in the art within the scope of the claims of the present patent application. [Explanation of Symbols]

[0076] 1. Analysis System 100 servers 110 processors 120 memory 130 Storage Devices 140 Input / Output Interfaces 150 network interfaces 160 Internal Bus 111 Acquisition Department 112 Image Processing Unit 113 Display Control Unit 200 user terminals

Claims

1. An acquisition unit that acquires image data of the sample to be analyzed, An image processing unit that performs image processing on the aforementioned image data, The system includes a display control unit that displays an image display area divided into a first region and a second region at a boundary line, The display control unit, The aforementioned boundary line is displayed in such a way that it can be moved by user operation. Of the image data before the aforementioned image processing is performed, the portion corresponding to the position of the first region is displayed in the first region. Of the image data after the image processing described above has been performed, the portion corresponding to the position of the second region is displayed in the second region. Display system.

2. The image processing unit performs a plurality of image processing operations on the image data in sequence. The display control unit, Of the image data before the image processing selected by the user from the above-mentioned plurality of image processing processes is performed, the portion corresponding to the position of the first region is displayed in the first region. Of the image data obtained after performing the image processing selected by the user from the aforementioned plurality of image processing processes, the portion corresponding to the position of the second region is displayed in the second region. The display system according to claim 1.

3. The display control unit, The identification images corresponding to each of the multiple image processing steps are displayed in parallel. The image processing corresponding to the identification image selected by the user is considered to be the image processing selected by the user. The display system according to claim 2.

4. The display control unit further displays a switching image near the image display area for switching the image data to be displayed in the image identification area. The display system according to claim 2.

5. Obtain image data of the sample to be analyzed, Image processing is performed on the aforementioned image data. The image display area is divided into a first region and a second region at the boundary line. The aforementioned image display area is The aforementioned boundary line is displayed in such a way that it can be moved by user operation. Of the image data before the aforementioned image processing is performed, the portion corresponding to the position of the first region is displayed in the first region. Of the image data after the image processing described above, the portion corresponding to the position of the second region is displayed in the second region. Display method.

Images