Display system and display method

Abstract

A display system according to the present invention is provided with: an acquisition unit that acquires image data of a sample to be analyzed; an image processing unit that performs image processing on the image data; and a display control unit that displays an image display region divided into a first region and a second region at the boundary line. The display control unit displays the boundary line so that the boundary line can be moved by a user operation, displays a portion corresponding to the position of the first region in the image data before the image processing is executed in the first region, and displays a portion corresponding to the position of the second region in the image data after the image processing is executed in the second region.

Description

Technical Field

[0001] This invention relates to a display system and display method. Background Technology

[0002] Japanese Patent Application Publication No. 2015-210491 discloses an image display system. In this system, when processing information establishes a corresponding association with an image that becomes the object of display, the system displays an image after image processing, following the display of an image before image processing. Summary of the Invention

[0003] Image processing is sometimes used as a preprocessing step for performing sample analysis using images. In this case, the user needs to appropriately select the image processing method in order to perform the sample analysis properly.

[0004] For example, in the image display system described in Japanese Patent Application Publication No. 2015-210491, when the composition of the images before and after image processing is displayed in chronological order, it is difficult for the user to understand the changes caused by image processing, and thus it is difficult to select the appropriate image processing.

[0005] That is, in scenarios where sample analysis is performed using images, users may not be able to actually perceive the effect of image processing as preprocessing, thus creating a problem where users cannot select the appropriate image processing for performing the analysis.

[0006] This invention was made to solve this problem, and its purpose is to provide a display control system and display method that allows users to easily select for appropriate image processing to perform analysis.

[0007] The display system according to the present invention includes: an acquisition unit that acquires image data of a sample that is the object of analysis; an image processing unit that performs image processing on the image data; and a display control unit that displays an image display area divided into a first region and a second region at a boundary line.

[0008] The display control unit displays the boundary line in a way that allows the user to move the boundary line. The portion of the image data before image processing that corresponds to the position of the first region is displayed in the first region, and the portion of the image data after image processing that corresponds to the position of the second region is displayed in the second region.

[0009] Based on this configuration, the display system of the present invention enables users to easily confirm changes in an image that has become the object of analysis through image processing. As a result, the display system of the present invention enables users to easily select image processing methods for appropriately performing analysis.

[0010] In the display system of the present invention, the image processing unit can sequentially perform multiple image processing operations on the image data.

[0011] Furthermore, the display control unit can display the portion of the image data corresponding to the position of the first region in the first region before performing the image processing selected by the user from multiple image processing processes, and display the portion of the image data corresponding to the position of the second region in the second region after performing the image processing selected by the user from multiple image processing processes.

[0012] In the display system of the present invention, the display control unit can display recognition images corresponding to the plurality of image processing respectively in parallel, and regard the preprocessing corresponding to the recognition image selected by the user as the image processing selected by the user.

[0013] In the display system of the present invention, the display control unit may also display a switching image for switching image data to be displayed in the image recognition area near the image display area.

[0014] The display method of the present invention performs the following processing: acquiring image data of the sample to be analyzed; performing image processing on the image data; and displaying the image display area divided into a first region and a second region at the boundary line.

[0015] Furthermore, in the aforementioned image display area, the boundary lines are displayed in a manner that allows the user to move the boundary lines. The portion of the image data before image processing that corresponds to the position of the first region is displayed in the first region, and the portion of the image data after image processing that corresponds to the position of the second region is displayed in the second region.

[0016] This invention provides a display control system and display method that allows users to easily select an appropriate image processing method for performing analysis. Attached Figure Description

[0017] Hereinafter, with reference to the accompanying drawings, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described, in which the same reference numerals denote the same elements, and wherein:

[0018] Figure 1 This is a block diagram illustrating the configuration of the analysis system according to the first embodiment.

[0019] Figure 2 This is a block diagram illustrating the hardware configuration of the server according to the first embodiment.

[0020] Figure 3 This is a block diagram illustrating the functional configuration of the server according to the first embodiment.

[0021] Figure 4 This is a schematic diagram showing the configuration of the display control unit according to the first embodiment.

[0022] Figure 5 This is a schematic diagram showing the configuration of the display control unit according to the first embodiment.

[0023] Figure 6 This is a schematic diagram showing the configuration of the display control unit according to the first embodiment.

[0024] Figure 7 This is a schematic diagram showing the configuration of the display control unit according to the first embodiment.

[0025] Figure 8 This is a flowchart illustrating the operation of the server according to the first embodiment. Detailed Implementation

[0026] Implementation Method 1

[0027] Composition of a display system

[0028] Hereinafter, the first embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0029] First, the configuration of the display system involved in this embodiment will be described in detail.

[0030] The display system described in this embodiment is a system configured as part of the analysis system described in this embodiment, and is used to display the preprocessing process used by the analysis system described in this embodiment to perform analysis to the user.

[0031] Figure 1 This is a block diagram illustrating the configuration of the analysis system according to the first embodiment. For example... Figure 1 As shown, the analysis system 1 involved in this embodiment is connected to a server 100 and a user terminal 200 via a network N such as the Internet.

[0032] Analysis System 1 is a system used to analyze the data of the analysis object.

[0033] The analysis system 1 described in this embodiment is typically provided as part of a data cloud service used in sample development or research development. The analysis system 1 described in this embodiment serves as a system for so-called materials informatics (MI) or for advancing research and development using data science. In this case, the analysis system 1, for example, stores various measurement data of novel development samples. Furthermore, the analysis system 1 analyzes the stored measurement data according to instructions from the user.

[0034] In analysis system 1, user terminal 200 sends image data that is the object of analysis to server 100, and server 100 analyzes the received image data. Furthermore, server 100 sends the analysis results to user terminal 200, and user terminal 200 displays the received analysis results.

[0035] The user terminal 200 involved in this embodiment is a user-operated terminal, typically a computer device with a display device.

[0036] User terminal 200 sends analysis object data to server 100 via network N. Furthermore, user terminal 200 receives analysis results of the analysis object data from server 100 via network N.

[0037] In this embodiment, the server 100 receives analysis object data from the user terminal 200 and analyzes the received analysis object data. Furthermore, the server 100 sends the analysis results to the user terminal 200 via network N.

[0038] Figure 2 This is a block diagram illustrating the hardware configuration of the server according to the first embodiment.

[0039] like Figure 2 As shown, server 100 includes processor 110, memory 120, storage device 130, input / output interface 140, network interface 150 and internal bus 160.

[0040] The internal bus 160 is a data transmission path for the processor 110, memory 120, storage device 130, input / output interface 140, and network interface 150 to send and receive data with each other. The method of connecting the processor 110, etc., is not limited to bus connection.

[0041] The memory 120 is a main storage device implemented using random access memory (RAM) or the like.

[0042] Furthermore, the storage device 130 is an auxiliary storage device implemented using a hard disk, solid-state drive (SSD), memory card, or read-only memory (ROM). The storage device 130 stores programs for implementing the desired functions.

[0043] Processor 110 is a central processing unit (CPU), graphics processing unit (GPU), or field-programmable gate array (FPGA), among other processors. Processor 110 executes the program described later by loading it from storage device 130 into memory 120 and executing it. Figure 3 and Figure 4 The functions of each functional block are shown in the diagram.

[0044] Input / output interface 140 is an interface used to connect server 100 and input / output devices. For example, input devices such as keyboards or output devices such as displays can be connected to input / output interface 140.

[0045] Network interface 150 is an interface used to connect server 100 to a network.

[0046] Additionally, when a program is read into a computer, it includes a set of commands (or software code) for causing the computer to perform one or more functions described in the implementation. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of limitation, computer-readable media or tangible storage media include RAM, ROM, flash memory, SSD or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray disc (registered trademark) or other optical disc storage, magnetic tape, magnetic disk storage, or other magnetic storage devices. The program may be transmitted on a temporary computer-readable medium or a communication medium. By way of limitation, temporary computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagation signals.

[0047] Here, the server 100 involved in this embodiment performs various image processing on the image data as preprocessing according to instructions from the user before performing image data analysis.

[0048] The display system involved in this embodiment is a system for users to appropriately select the above-mentioned image processing, and is implemented as part of the functions of the server 100.

[0049] Figure 3 This is a block diagram illustrating the functions of the server according to the first embodiment.

[0050] like Figure 3 As shown, the server 100 according to this embodiment includes an acquisition unit 111, an image processing unit 112, and a display control unit 113 as functional blocks.

[0051] The acquisition unit 111 acquires image data that is the object of analysis. More specifically, the acquisition unit 111 according to this embodiment acquires image data from the user terminal 200 via network N. The acquisition unit 111 outputs the acquired image data to the image processing unit 112.

[0052] Here, when the acquisition unit 111 receives image data from the user terminal 200, it can store the acquired image data in the storage device 130. Furthermore, when a specified image data is received from the user, the specified image data can be read from the storage device 130 and output to the image processing unit 112.

[0053] That is, the acquisition unit 111 can create a database storing image data. Furthermore, the acquisition unit 111 can be configured such that a user can appropriately select image data from the created database as the object of analysis.

[0054] The image processing unit 112 performs image processing on the image data.

[0055] Specifically, the image processing unit 112 acquires image data from the acquisition unit 111 and receives the type of image processing to be performed from the user. Furthermore, the image processing unit 112 performs the image processing received from the user on the image data. The image processing unit 112 then outputs the processed image data to the display control unit 113.

[0056] Furthermore, the image processing unit 112 can perform multiple image processing operations on the image data. In this case, the image processing unit 112 receives multiple image processing operations and the execution order of the multiple image processing operations from the user. The image processing unit 112 then sequentially performs the multiple image processing operations on the image data according to the execution order received from the user.

[0057] The display control unit 113 controls the display screen of the user terminal 200. More specifically, the display control unit 113 displays an image processing selection screen for the user to select image processing and an image processing confirmation screen for the user to confirm the effect of image processing on the user terminal 200.

[0058] Figure 4 and Figure 5 This is a schematic diagram showing the configuration of the display control unit according to the first embodiment.

[0059] In more detail, Figure 4 This is a schematic diagram showing the image processing selection screen P1 displayed on the user terminal 200 by the display control unit 113 according to this embodiment. Figure 5This is a schematic diagram showing the image processing confirmation screen P2 displayed on the user terminal 200 by the display control unit 113 according to this embodiment.

[0060] The following uses Figure 4 and Figure 5 The image processing selection screen P1 and the image processing confirmation screen P2 are explained.

[0061] like Figure 4 As shown, the display control unit 113 displays an image processing selection screen P1, which displays the recognition image P11 representing the image data before image processing, and recognition images P12a, P12b and P12c corresponding to multiple image processing operations respectively.

[0062] In addition, Figure 4 In this example, three image processing steps are described, but of course, the number of image processing steps performed is not limited to three.

[0063] Furthermore, unless there is a specific need for differentiation, in the following instructions, the recognition images P12a, P12b, and P12c will be referred to simply as recognition image P12. The same applies to the recognition name P121, image processing condition P122, sample image P123, and switching image P226, which will be described later.

[0064] Image P11, the image data to be recognized, represents the image data before image processing. Image P11 displays the recognition name P111 and image information P112 as text. Furthermore, image P11 also displays a sample image P113.

[0065] The identification name P111 is text that establishes a corresponding association between the identified image P11 and the image data before image processing. For example, in Figure 4 In this process, the text "Input Image" is displayed as the recognition name P111, indicating that the recognition image P11 is associated with the image data before image processing. The recognition name P111 can be, for example, the specific name of the image data before image processing.

[0066] Image information P112 is text representing information related to the image data before image processing. For example, image information P112 can display various information such as the data size of the image data, the size of the image, the date and time the user saved the image data, and user comments.

[0067] The sample image P113 displays the image before image processing. The sample image P113 can display a portion of the image before image processing, or it can display all of the image before image processing.

[0068] Image processing recognition image P12 is an image that identifies the image processing performed on the image data. Recognition image P12 displays the recognition name P121 and image processing conditions P122 as text. Furthermore, sample image P123 is also displayed in recognition image P12.

[0069] The identification name P121 is text indicating which image processing process is associated with the identified image P12. For example, in Figure 4 In this context, the text "Image Processing 1" is displayed as the identification name P121, indicating that the identified image P12 is associated with the image processing described in "Image Processing 1". For example, the identification name P121 can contain specific names of image processing operations such as "noise reduction" or "automatic contrast adjustment".

[0070] Image processing condition P122 contains text indicating the conditions referenced when performing image processing. For example, image processing condition P122 may display parameter values ​​required for performing image processing.

[0071] Sample image P123 displays the image after the corresponding image processing. Sample image P123 can display a portion of the processed image, or it can display all of the processed image.

[0072] Here, the identified images P12 are displayed side-by-side according to the execution order of their corresponding image processing. That is, in the order shown... Figure 4 In the case shown, the image processing corresponding to the recognition image P12a is executed first, the image processing corresponding to the recognition image P12b is executed second, and the image processing corresponding to the recognition image P12c is executed third.

[0073] Furthermore, the image processing selection screen P1 is configured to be editable by the user. For example, the image processing selection screen P1 is configured to allow the user to edit the type of image processing to be performed, the execution order of the image processing, etc.

[0074] In this case, the user performs an editing operation on the image processing selection screen P1 so that the recognition image P12, which corresponds to the desired image processing, is displayed in the desired execution order. Furthermore, the image processing unit 112 performs image processing with reference to the image processing selection screen P1 edited by the user.

[0075] Here, if the user selects any of the recognition images P12, the display control unit 113 will transfer the screen displayed on the user terminal 200 to the image processing confirmation screen P2.

[0076] The following is as follows: Figure 4The image processing selection screen P1 shows the user selecting to recognize image P12b. The image processing confirmation screen P2 is explained in detail.

[0077] like Figure 5 As shown, the display control unit 113 displays in parallel a condition display area P21 that represents the image processing conditions and an image display area P22 that displays the images before and after image processing on the image processing confirmation screen P2.

[0078] The condition display area P21 contains text indicating the conditions referenced during image processing. Furthermore, the conditions displayed in the condition display area P21 are related to... Figure 4 The image processing conditions shown are the same as those described in P122.

[0079] Here, the condition display area P21 is configured to be editable by the user. When the condition display area P21 is edited, the image processing unit 112 performs image processing according to the edited conditions.

[0080] The display control unit 113 displays images before and after the execution of the image processing selected by the user in the image display area P22.

[0081] In addition, Figure 5 In the example shown, the display control unit 113 displays and Figure 4 The image displayed is the image before and after the execution of the image processing "Image Processing 2" corresponding to the recognition image P12b described in the document. That is, the display control unit 113 treats the image processing "Image Processing 2" corresponding to the recognition image P12b selected by the user as the image processing selected by the user, and displays the image in the image display area P22.

[0082] Here, the image display area P22 is divided into a first area P221 and a second area P222 at the boundary line P223. The display control unit 113 displays the image data before image processing in the first area P221. And, the display control unit 113 displays the image data after image processing in the second area P222.

[0083] More specifically, the display control unit 113 displays the portion of the image data before image processing that corresponds to the position of the first region P221 in the first region P221.

[0084] Furthermore, the display control unit 113 displays the portion of the image data after image processing that corresponds to the position of the second region P222 in the second region P222.

[0085] Furthermore, in this embodiment, the display control unit 113 is displayed in a manner that allows the user to move the boundary line P223.

[0086] The "part of the image data before image processing that corresponds to the position of the first region P221" mentioned here refers to the part of the image data that overlaps with the first region P221 when the "image data before image processing" is enlarged or reduced to the same size as the image display area P22 and overlaps with the image display area P22.

[0087] Furthermore, the "part of the image data after image processing that corresponds to the position of the second region P222" mentioned here refers to the portion of the image data that overlaps with the second region P222 when the "image data after image processing" is enlarged or reduced to the same size as the image display area P22 and overlaps with the image display area P22.

[0088] The following uses Figure 6 and Figure 7 The changes in display mode when the boundary line P223 is moved are explained. Figure 6 and Figure 7 This is a diagram showing the change in display when boundary line P223 is moved from its original position P223b to its current position P223a. Figure 6 This indicates the display method before moving the boundary line P223. Figure 7 This indicates the display method after moving the boundary line P223.

[0089] Furthermore, the direction in which the boundary line P223 is moved is merely an example and is not limited to... Figure 6 and Figure 7 The direction is shown. Boundary line P223 can be moved freely within image area P22 by user operation.

[0090] like Figure 6 and Figure 7 As shown, by moving the boundary line P223, the sizes of the first region P221 and the second region P222 gradually change. That is, as the boundary line P223 moves, the area displaying the image data after image processing gradually changes to the area displaying the image data before image processing.

[0091] Based on this configuration, users can easily visually confirm changes in image data processed by image processing. As a result, the server 100 according to this embodiment enables users to easily select image processing methods for appropriate analysis.

[0092] return Figure 5 Explanation.

[0093] The display control unit 113 in this embodiment also displays marker images P224 and P225 near the image display area P22.

[0094] The marked image P224 is used to identify the image displayed in region 1 P221 as the image before the image processing selected by the user is performed.

[0095] In this embodiment, the label image P224 displays text such as "Applicable Before (No.2)".

[0096] Additionally, the string "(No2)" in the text indicates that the image before performing the image processing selected by the user was processed. Figure 4 The image data after image processing corresponding to the identified image P12a in the image.

[0097] Image P225 is used to identify the image displayed in region 2 P222 as the image after image processing selected by the user.

[0098] In this embodiment, as a marker image P225, text such as "After application" is displayed.

[0099] Furthermore, the display control unit 113 of this embodiment displays a switching image P226 near the image display area P22 for switching image data to be displayed in the image recognition area P22.

[0100] When the user selects to switch image P226, the display control unit 113 switches the image processing selected by the user to an image processing whose execution order is one or one step earlier or later than the selected image processing.

[0101] In this embodiment, the image data displayed in the first area P221 is switched when the user selects to switch the image P226, while the image data displayed in the second area P222 is not switched.

[0102] More specifically, when the switching image P226 is selected, the display control unit 113 switches the image data displayed in the first area P221 between the image data corresponding to the identified image P11 and the image data after the image processing that was performed in a higher order than the image processing selected by the user.

[0103] That is, such as Figure 5 As shown, when an image corresponding to the recognition image P12b is displayed in the second region P222, the display control unit P113 can switch between displaying the input image P11 corresponding to the recognition image P11 and the image data after the recognition image P12a is executed.

[0104] More specifically, if the image displayed in the first area P221 is the input image corresponding to the recognition image P11, and the user selects to switch the image P226R, then the image displayed in the first area P221 is switched to the image data after performing the image processing corresponding to the recognition image P12a.

[0105] Furthermore, if the image displayed in the first area P221 is the image data after performing image processing corresponding to the recognition image P12a, and the user selects to switch the image P226L, then the image displayed in the first area P221 will be switched to the input image corresponding to the recognition image P11.

[0106] Furthermore, the image data after performing image processing corresponding to the image P12a is the same as the image data before performing image processing corresponding to the image P12b.

[0107] Based on this configuration, users can easily verify the effect of combining multiple image processing methods. As a result, users can more easily select the appropriate image processing method.

[0108] Display system actions

[0109] Next, the operation of the display system, i.e., the display method according to the first embodiment, will be described in detail. Figure 8 This is a flowchart illustrating the operation of the server according to the first embodiment.

[0110] exist Figure 8 In the processing sequence, the processor 110 of the server 100 reads the program stored in the storage device 130 into the memory 120 and executes it, thus functioning as the acquisition unit 111, the image processing unit 112, and the display control unit 113.

[0111] In the display method according to this embodiment, firstly, the processor 110 acquires image data (ST1). That is, in ST1, the processor 110 functions as an acquisition unit 111.

[0112] For example, in ST1, processor 110 receives a selection of analysis object data from user terminal 200 for performing analysis. Furthermore, processor 110 retrieves the selected analysis object data from storage device 130.

[0113] Next, the processor 110 displays the image processing selection screen P1 (ST2). That is, in ST2, the processor 110 functions as the display control unit 113.

[0114] More specifically, in ST2, processor 110 displays as follows: Figure 4The image processing selection screen shown. In the displayed image processing selection screen, the processor 110 receives from the user the edits of the type of image processing to be performed and the execution order of the image processing.

[0115] Next, the processor 110 performs image processing (ST3) on the image data. That is, in ST3, the processor 110 functions as the image processing unit 112.

[0116] More specifically, in ST3, processor 110 performs image processing on the image data according to the category of image processing to be performed and the execution order of image processing edited in ST2.

[0117] Furthermore, ST2 and ST3 can be steps executed in parallel. That is, the processor 110 can perform image processing each time the user edits the image processing selection screen P1, and each time the result of the image processing is reflected in the sample image P123.

[0118] Next, the processor 110 determines whether the user has selected the image processing recognition image P12 (ST4). That is, in ST4, the processor 110 functions as the display control unit 113.

[0119] If the user does not select the recognition image P12 for image processing (ST4 is "No"), the processor 110 executes ST4 again. That is, the processor 110 remains in standby mode without executing the next step until the user selects the recognition image P12 for image processing.

[0120] If the user selects the image to be recognized, P12 ("Yes" in ST4), the processor 110 displays the image processing confirmation screen P2 (ST5), and the server 100 completes a series of actions. In ST5, the processor 110 functions as the display control unit 113.

[0121] More specifically, in ST3, the processor 110 displays an image processing confirmation screen P2, which includes an image display area P22 divided into a first region P221 and a second region P222 at the boundary line P223.

[0122] like Figure 5 As shown, the image display area P22 is displayed in a manner that allows the user to move the boundary line P223. Furthermore, in the first area P221, the portion of the image data before image processing is performed, corresponding to the position in the first area P221, is displayed. In the second area P222, the portion of the image data after image processing is performed, corresponding to the position in the second area P222, is displayed.

[0123] As explained above, the server 100 in this embodiment displays the image display area P22 of the display system, which is divided into a first area P221 and a second area P222 at the boundary line P223. Furthermore, the first area P221 displays the portion of the image data before image processing that corresponds to the position in the first area P221. The second area P222 displays the portion of the image data after image processing that corresponds to the position in the second area P222.

[0124] Based on this configuration, users can easily visually confirm changes in image data processed by image processing. As a result, the server 100 according to this embodiment enables users to easily select image processing methods for appropriate analysis.

[0125] The present invention has been described above in correspondence with the above embodiments. However, the present invention is not limited to the above embodiments. Of course, it includes various modifications, alterations and combinations that can be made by those skilled in the art within the scope of the invention as claimed in this application.

Claims

1. A display system, characterized in that, have: The acquisition unit acquires image data of the sample that is the object of analysis. An image processing unit performs image processing on the image data; and The display control unit displays an image display area divided into a first region and a second region at the boundary line. The display control unit displays the boundary line in a manner that allows the user to move the boundary line. The portion of the image data corresponding to the position of the first region in the image data prior to the image processing will be displayed in the first region. The portion of the image data after the image processing that corresponds to the position of the second region will be displayed in the second region.

2. The display system according to claim 1, characterized in that, The image processing unit sequentially performs multiple image processing operations on the image data. The display control unit will display the portion of the image data corresponding to the position of the first region in the first region before performing the image processing selected by the user from the plurality of image processing, and will display the portion of the image data corresponding to the position of the second region in the second region after performing the image processing selected by the user from the plurality of image processing.

3. The display system according to claim 2, characterized in that, The display control unit displays recognition images corresponding to the plurality of image processes in parallel, and regards the image process corresponding to the recognition image selected by the user as the image process selected by the user.

4. The display system according to claim 2, characterized in that, The display control unit also displays a switching image near the image display area for switching the image data to be displayed in the image recognition area.

5. A display method, characterized in that, Perform the following processing: Acquire image data of the sample to be analyzed; Perform image processing on the image data; and The image display area is divided into region 1 and region 2 at the boundary line. In the image display area, the boundary line is displayed in a manner that allows the user to move the boundary line. The portion of the image data before the image processing is performed that corresponds to the position of the first region is displayed in the first region, and the portion of the image data after the image processing is performed that corresponds to the position of the second region is displayed in the second region.

Images