Pathology work support system and AR equipment

Abstract

To provide a technique that allows pathological work to be done efficiently.SOLUTION: A pathological work support system according to an embodiment of the present disclosure is a pathological work support system for supporting pathological diagnosis work, and includes an AR apparatus and a server device. The AR apparatus includes: a camera for imaging a real space; and a display unit for overlapping images in the real space and displaying the images. The server device controls the AR apparatus. At least one of the server device and the AR apparatus has a tag information acquisition unit, a sample information acquisition unit, and a sample information display unit. The tag information acquisition unit is in the vision of the camera and acquires, from the camera, tag information which is printed on a member used when a sample to be subjected to a pathological diagnosis is pre-treated. The sample information acquisition unit acquires sample information on the sample attached to the acquired tag information. The sample information display unit overlaps the sample information on the member with the tag information printed thereon for display in the display unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a pathology work support system and an AR device. [Background technology]

[0002] Conventionally, a technology has been known that allows a pathology department that is requested by an endoscopic examination department or the like to efficiently perform such pathology diagnosis work (hereinafter also referred to as "pathology work") for a patient (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-5313 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned technology leaves room for further improvement in terms of increasing the efficiency of pathology work.

[0005] The present disclosure has been made in view of the above, and provides a technology that can improve the efficiency of pathology work. [Means for solving the problem]

[0006] A pathology work support system according to one aspect of the present disclosure is a pathology work support system that supports pathology diagnosis work, and includes an AR device and a server device. The AR device includes a camera that captures images of real space and a display unit that displays an image superimposed on the real space. The server device controls the AR device. At least one of the server device and the AR device includes a tag information acquisition unit, a specimen information acquisition unit, and a specimen information display unit. The tag information acquisition unit acquires, from the camera, tag information that is marked on a member that is within the field of view of the camera and is used in preprocessing a specimen that is the subject of pathological diagnosis. The specimen information acquisition unit acquires specimen information related to the specimen that is associated with the acquired tag information. The specimen information display unit displays the specimen information on the display unit by superimposing the specimen information on the member on which the tag information is marked. [Effects of the Invention]

[0007] According to the present disclosure, pathology work can be made more efficient. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining an overview of a pathology work support method according to an embodiment. [Figure 2] FIG. 2 is a functional block diagram showing the configuration of the pathology work support system according to the embodiment. [Figure 3] FIG. 3 is a functional block diagram showing the configuration of the AR device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a specimen information storage unit according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of an imaging data storage unit according to the embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of a procedure for pre-treating a specimen according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a pathology work support process according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a pathology work support process according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the pathology work support process according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of the pathology work support process according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of the pathology work support process according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of the pathology work support process according to the embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of the pathology work support process according to the embodiment. [Figure 14] FIG. 14 is a functional block diagram showing the configuration of an AR device according to another embodiment. [Figure 15] FIG. 15 is a flowchart showing an example of the procedure of the pathology work support process executed by the pathology work support system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of a pathology work support system and an AR device disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present disclosure is not limited to the embodiments described below.

[0010] <Outline of pathology work support methods> First, an overview of a pathology work support method according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining an overview of a pathology work support method according to an embodiment.

[0011] The pathology work support method according to the embodiment includes the following steps S01 to S04. As shown in FIG. 1 , the pathology work support method supports an operator X who wears an AR (Augmented Reality) device 50 when performing pathology work.

[0012] The AR device 50 is a device capable of AR display. In the present disclosure, "AR display" refers to displaying various images in a way that makes the user perceive them as if they were real objects existing in real space.

[0013] In the pathology work support method according to the embodiment, first, the server device 10 acquires tag information printed on a member (hereinafter also referred to as a "pre-processing member") used in pre-processing of a specimen S (see FIG. 11) using the camera 55 (see FIG. 3) of the AR device 50 worn by the operator X (step S01).

[0014] Here, the pre-processing components include, for example, a pathology test request form for a pathological diagnosis of the specimen S, a container 61 for containing the specimen S (see Figure 7), a cassette 62 for pre-processing the specimen S (see Figure 10), and tweezers 63 for pre-processing the specimen S (see Figure 10).

[0015] The preprocessing members according to the embodiment are not limited to the pathology test request form, the container 61, the cassette 62, and the tweezers 63. The tag information printed on the preprocessing member is, for example, a barcode such as a two-dimensional barcode, and includes an individual ID and the like.

[0016] The process of step S01 is performed when the worker X visually checks the pre-processing member, causing the pre-processing member to enter the field of view of the camera 55. That is, in the embodiment, by using the camera 55 mounted on the AR device 50, the process of step S01 can be performed without the worker X being aware of it.

[0017] Next, the server device 10 acquires information about the sample S (hereinafter also referred to as "sample information") associated with the acquired tag information (step S02). The sample information is linked to the tag information and pre-stored in the storage unit 30 (see FIG. 2) of the server device 10. Furthermore, the sample information according to the embodiment includes, for example, the ID and name of the patient from whom the sample S was collected.

[0018] Next, the server device 10 displays the sample information obtained in the processing of step S02 superimposed on the pre-processing member for which tag information was obtained in the processing of step S01 on the display unit 54 of the AR device 50 (see FIG. 3) (step S03).

[0019] For example, in the example of Figure 1, when the information "ID:XXX-YYYY ○○ Taro" is obtained from the memory unit 30 as specimen information accompanying the tag information to be printed on the pathology test request form, the specimen information is superimposed and displayed near the pathology test request form.

[0020] This allows the operator X wearing the AR device 50 to check the specimen information associated with the preprocessing member simply by visually checking the preprocessing member. Therefore, according to the embodiment, it is possible to eliminate the need to check the specimen information for each preprocessing member using a barcode reader, a personal computer, or the like, thereby improving the efficiency of pathology work.

[0021] Furthermore, in the embodiment, by using the head-mounted AR device 50, it is possible to check the specimen information linked to the preprocessing member without touching a barcode reader, keyboard, mouse, etc. Therefore, according to the embodiment, it is possible to prevent the specimen S from contaminating the equipment during the preprocessing of the specimen S.

[0022] It should be noted that the AR device 50 of the present disclosure is not limited to a head-mounted AR device, and may be a smartphone, a tablet terminal, etc. This also makes it possible to improve the efficiency of pathology work.

[0023] Next, the server device 10 records the state of the real space captured by the camera 55 in accordance with a signal from the worker X (step S04). For example, in the example of FIG. 1, when the worker X utters a preset keyword (e.g., "start recording"), the server device 10 records the state of the workbench as seen by the worker X as a video.

[0024] In this case, the server device 10 may terminate the recording when the worker X utters another preset keyword (for example, "end recording").

[0025] In this way, in the embodiment, by recording the state of the real space according to a signal from the operator X, it is possible to simply and accurately record the work that is highly necessary to be preserved as evidence (for example, the work of taking out the specimen S from the container 61 or the like and performing various processes thereon). Therefore, according to the embodiment, it is possible to improve the efficiency of pathology work.

[0026] Furthermore, in the embodiment, the state of the real space is recorded using the head-mounted AR device 50, so that the viewpoint during work can be matched with the field of view of the camera 55. Therefore, according to the embodiment, it is possible to leave evidence recorded from an optimal viewpoint.

[0027] Furthermore, in the embodiment, by using the head-mounted AR device 50, the state of the work can be recorded without touching a switch, etc. Therefore, according to the embodiment, contamination of devices, etc. by the sample S during the pre-processing of the sample S can be suppressed.

[0028] Furthermore, in the embodiment, when the state of the pre-processing work is recorded as evidence, the specimen information has already been acquired in the processing of step S02, so the evidence record can be linked to the specimen information without the operator X being aware of it. Therefore, according to the embodiment, the efficiency of pathology work can be further improved.

[0029] Furthermore, in the embodiment, the amount of equipment required for pathology work can be reduced by using the AR device 50 to check sample information and record work. Therefore, according to the embodiment, the space required for pathology work can be saved.

[0030] 1 shows an example in which the state of the workbench as viewed by the worker X is recorded as a video, but the present disclosure is not limited to such an example. For example, the server device 10 may record the state of the workbench as viewed by the worker X as a still image by the worker X uttering yet another preset keyword (for example, "photograph").

[0031] 1 shows an example in which the state of the work table as viewed by the worker X is recorded by the worker X uttering a predetermined keyword, but the present disclosure is not limited to such an example. For example, the server device 10 may record the state of the work table as viewed by the worker X as a video or still image, using a predetermined gesture performed by the worker X as a signal. This also makes it possible to improve the efficiency of pathology work.

[0032] <Example of the configuration of a pathology work support system> Next, an example of the configuration of the pathology work support system 1 will be described with reference to Fig. 2 to Fig. 6. Fig. 2 is a functional block diagram showing the configuration of the pathology work support system 1 according to the embodiment.

[0033] 2, the pathology work support system 1 includes a server device 10 and an AR device 50. Note that, although Fig. 2 shows a case where the pathology work support system 1 includes two AR devices 50, the number of AR devices 50 included in the pathology work support system 1 is not limited to the example shown in the figure, and any number of AR devices 50 may be included.

[0034] The server device 10 and the AR device 50 are connected to each other so that they can communicate with each other via a network N. The network N may be wired or wireless, and may be any type of communication network, such as the Internet, a LAN (Local Area Network), or a VPN (Virtual Private Network).

[0035] The server device 10 is a computer that provides the above-mentioned pathology work support service to the AR device 50. The AR device 50 is a device that receives the above-mentioned pathology work support service from the server device 10. Fig. 3 is a functional block diagram showing the configuration of the AR device 50 according to the embodiment.

[0036] 3, the AR device 50 includes a communication unit 51, a control unit 52, a storage unit 53, a display unit 54, a camera 55, a microphone 56, and a speaker 57. The camera 55 and the microphone 56 are examples of a receiving unit, and the speaker 57 is an example of an audio output unit.

[0037] The communication unit 51 controls communication with other computers such as the server device 10, regardless of whether the communication is wired or wireless. The communication unit 51 is, for example, a communication interface such as a NIC (Network Interface Card).

[0038] The control unit 52 is a processing unit that controls the overall processing of the AR device 50. The control unit 52 is realized, for example, by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), or the like, by executing a program stored in an internal storage device using RAM as a working area.

[0039] The control unit 52 may also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0040] The storage unit 53 stores various data such as programs executed by the control unit 52. The storage unit 53 corresponds to a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), or a flash memory, or a storage device such as an HDD (Hard Disk Drive).

[0041] The display unit 54 displays an image superimposed on real space. The display unit 54 may be a so-called optical see-through display device or a so-called video see-through display device.

[0042] The optical see-through type is a system in which an electronic image is superimposed on the actual scenery seen through a lens using an optical system such as a prism or half mirror, while the video see-through type is a system in which an electronic image is superimposed on the scenery image captured in real time by the camera 55.

[0043] The camera 55 faces in approximately the same direction as the user wearing the AR device 50, and captures an image of real space that is approximately the same as the user's field of view. The camera 55 also captures an image of a gesture made by the worker X. The microphone 56 acquires audio, such as speech, from the user wearing the AR device 50. The speaker 57 outputs audio to the user wearing the AR device 50.

[0044] 2, the functional configuration of the server device 10 in the embodiment will be described. The server device 10 includes a communication unit 20, a storage unit 30, and a control unit 40.

[0045] The communication unit 20 controls communication with the AR device 50 and other computers, regardless of whether the communication is wired or wireless. The communication unit 20 is, for example, a communication interface such as a NIC.

[0046] The storage unit 30 stores various data such as programs executed by the control unit 40. The storage unit 30 corresponds to a semiconductor memory element such as a RAM, a ROM, or a flash memory, or a storage device such as a HDD. The storage unit 30 also includes a specimen information storage unit 31 and an imaging data storage unit 32.

[0047] The specimen information storage unit 31 stores the management history of the specimen S that is the subject of pathological diagnosis. Fig. 4 is a diagram showing an example of the specimen information storage unit 31 according to the embodiment. The specimen information storage unit 31 shown in Fig. 4 has items such as "tag information" and "specimen information".

[0048] "Tag information" refers to tag information that is individually assigned to each specimen S for which a pathological diagnosis has been requested. "Specimen information" refers to various information related to the specimen S for which a pathological diagnosis has been requested. Such specimen information includes, for example, the ID and name of the patient from whom the specimen S was collected.

[0049] The imaging data storage unit 32 stores a management history of imaging data recorded when the sample S is pre-processed. Fig. 5 is a diagram showing an example of the imaging data storage unit 32 according to the embodiment. The imaging data storage unit 32 shown in Fig. 5 has items such as "tag information" and "imaging data information".

[0050] "Tag information" refers to tag information that is individually assigned to each specimen S for which a pathological diagnosis has been requested. "Imaging data information" refers to information related to data that records the state of pre-processing of the specimen S (for example, the file name of the imaging data).

[0051] An example of the procedure of pre-processing performed on the sample S for pathological diagnosis will now be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the procedure of pre-processing the sample S according to the embodiment.

[0052] In the pre-processing of the specimen S, first, a reception process for the specimen S and a pathology test request is performed (step S11). In this reception process, for example, tag information printed on a formalin-containing container 61 (see FIG. 7) containing the specimen S and tag information printed on a pathology test request for a pathology diagnosis of the specimen S are read by a barcode reader or the like.

[0053] In addition, if a handwritten label rather than a barcode is attached to the container 61, in the processing of step S11, the tag information written on the pathology test request form is read and compared with the number of containers 61 written on the pathology test request form and the information on the handwritten label.

[0054] Next, the specimen S is soaked in formalin for a predetermined time (for example, several hours) to perform a fixation process to prevent corrosion of the specimen S (step S12). This fixation process is performed inside a container 61 containing formalin in which the specimen S is accommodated.

[0055] Next, an excision process is performed to excise a portion of the specimen S that is to be subjected to pathological diagnosis (step S13). In this excision process, for example, the specimen S is taken out from the container 61 onto a workbench, and after the portion that is to be subjected to pathological diagnosis is excised, the excised portion is stored inside the cassette 62 (see FIG. 10). Note that in this excision process, since the specimen S is taken out from the container 61, there is a risk of mix-up even at this point.

[0056] Next, the specimen S accommodated inside the cassette 62 is subjected to a treatment process to prevent it from crumbling even when thinly sliced (step S14). In this treatment process, for example, decalcification of the specimen S is performed inside the cassette 62.

[0057] Next, an embedding process is performed in which the treated specimen S is solidified with wax (step S15). In this embedding process, for example, the specimen S is taken out from the cassette 62 onto a workbench, and the specimen S is solidified with wax. Note that, since the specimen S is taken out from the cassette 62 in this embedding process, there is a risk of mix-up even at this point.

[0058] Next, in order to make the specimen S observable under a microscope or the like, a thin-slicing process is performed in which the embedded specimen S is thinly sliced (step S16). Note that, since the specimen S is thinly sliced in this thin-slicing process, there is a risk of mix-up even at this stage.

[0059] Next, in order to make the specimen S easily visible under a microscope, a staining process is performed to stain the specimen S after the thin sectioning process (step S17). Finally, a sealing process is performed in which the stained specimen S is sandwiched between a cover glass (step S18), thereby completing preparation of the specimen for pathological diagnosis.

[0060] As explained above, when the specimen S is pre-processed (i.e., when processing steps S11 to S18), the specimen S needs to be removed from the container 61 or cassette 62 and subjected to various processes, so it is very important to prevent the specimen S from being mixed up during pre-processing.

[0061] Returning to the explanation of Fig. 2, the control unit 40 is a processing unit that controls the overall processing of the server device 10. The control unit 40 is realized, for example, by a CPU, MPU, GPU, or the like executing a program stored in an internal storage device using RAM as a work area. The control unit 40 may also be realized, for example, by an integrated circuit such as an ASIC or FPGA.

[0062] The control unit 40 has a member detection unit 41, a tag information acquisition unit 42, a specimen information acquisition unit 43, a specimen information display unit 44, a reading unit 45, a notification unit 46, and a recording unit 47, and realizes or executes the functions and actions of the control processing described below. Note that the internal configuration of the control unit 40 is not limited to the configuration shown in Fig. 2, and may have any other configuration as long as it is capable of performing the control processing described below.

[0063] The component detection unit 41 detects pre-processing components and their positions within the field of view of the camera 55 in the AR device 50. The component detection unit 41 detects, for example, the positions of a pathology test request form for a pathological diagnosis of the specimen S, a container 61 containing the specimen S, a cassette 62 for pre-processing the specimen S, and tweezers 63 (see FIG. 10 ) for handling the specimen S during pre-processing, as coordinate data in the augmented reality space.

[0064] The component detection unit 41 detects the preprocessing component and acquires its coordinates using an object detection technique such as R-CNN (Region Convolutional Neural Network), YOLO (You Only Look Once), or SSD (Single Shot multibox Detector).

[0065] The tag information acquisition unit 42 acquires tag information marked on a preprocessing member that is present within the field of view of the camera 55. For example, the tag information acquisition unit 42 determines whether or not tag information is marked on a preprocessing member within the field of view detected by the member detection unit 41, and acquires the tag information if tag information is marked on the preprocessing member.

[0066] The specimen information acquisition unit 43 acquires specimen information associated with the tag information acquired by the tag information acquisition unit 42 from the specimen information storage unit 31 of the storage unit 30.

[0067] The specimen information display unit 44 displays the specimen information acquired by the specimen information acquisition unit 43 on the display unit 54 of the AR device 50, superimposed on a pre-processing member on which tag information associated with the specimen information is printed.

[0068] When the reading unit 45 displays the specimen information on the display unit 54 in a manner superimposed on the preprocessing member, the reading unit 45 outputs the specimen information from the speaker 57.

[0069] If the specimen information of a preprocessing member on which tag information is printed does not match the specimen information of another preprocessing member on which tag information is printed, the notification unit 46 notifies the operator X that the specimen information does not match.

[0070] The recording unit 47 records the state of the real space where the pre-processing of the specimen S is performed, which is captured by the camera 55 of the AR device 50. The recording unit 47 also stores the recorded captured image data in the storage unit 30.

[0071] <Examples of pathology work support processing> Next, a specific example of the pathology work support process according to the embodiment will be described with reference to Fig. 7 to Fig. 13. Fig. 7 to Fig. 13 are diagrams showing an example of the pathology work support process according to the embodiment. Note that, hereinafter, a series of the pathology work support process will be described while illustrating the display state of the display unit 54 as viewed by the worker X wearing the AR device 50.

[0072] First, as shown in Fig. 7, an operator X (see Fig. 1) who performs pre-processing of a sample S visually checks a container 61, which is a pre-processing member. Then, a member detection unit 41 (see Fig. 2) of a control unit 40 (see Fig. 2) detects the container 61 captured by a camera 55 (see Fig. 3) of an AR device 50 (see Fig. 3).

[0073] Then, the member detection unit 41 displays on the display unit 54 that the preprocessing member has been detected. For example, as shown in Fig. 7, the member detection unit 41 displays on the display unit 54 that the preprocessing member has been detected by outlining the outline of the preprocessing member with a thick dashed line or the like. This allows the worker X to easily recognize that the preprocessing member has been detected.

[0074] Next, the tag information acquisition unit 42 (see FIG. 2) determines whether or not tag information is printed on the detected pre-processing member, and if tag information is printed on the pre-processing member, acquires the tag information.

[0075] 7, tag information is marked on the detected container 61. In this case, the tag information acquisition unit 42 acquires the tag information marked on the container 61.

[0076] 10, among the detected container 61, cassette 62, and tweezers 63, tag information is printed only on the container 61. In this case, the tag information acquisition unit 42 acquires the tag information printed on the container 61.

[0077] In the present disclosure, if tag information is not attached to the container 61, a process of attaching tag information to the container 61 is required. In this case, for example, after the tag information attached to the pathology test request form is recognized by the AR device 50, the tag information can be attached to the container 61 by a predetermined operation by the user.

[0078] Furthermore, as shown in FIG. 10, when the display unit 54 contains pre-processing members with and without tag information printed thereon, the presence or absence of such tag information may be indicated by differences in the outline display of the pre-processing members.

[0079] For example, the component detection unit 41 may display the border of a pre-processing component on which tag information is printed in a different color from the border of a pre-processing component on which tag information is not printed, or may display the border in a different type of border line (for example, a dashed line and a dotted line, etc.) as shown in Figure 10.

[0080] Returning to the explanation of Figure 7, next, the specimen information acquisition unit 43 (see Figure 2) acquires specimen information associated with the tag information acquired by the tag information acquisition unit 42 from the specimen information storage unit 31 (see Figure 2) of the storage unit 30 (see Figure 2). For example, in the example of Figure 7, the information "ID:XXX-YYYY XX Taro" is acquired as specimen information associated with the acquired tag information of the container 61.

[0081] Next, the sample information display unit 44 (see Figure 2) displays the sample information acquired by the sample information acquisition unit 43 on the display unit 54 of the AR device 50, superimposed on a pre-processing member on which tag information associated with the sample information is printed.

[0082] For example, in the example of FIG. 7, the specimen information display unit 44 displays specimen information "ID:XXX-YYYY XX Taro" as superimposed information A1 to be superimposed on the container 61.

[0083] As a result, the worker X wearing the AR device 50 can visually confirm the specimen information linked to the preprocessing member simply by looking at the preprocessing member.

[0084] Therefore, according to the embodiment, it is possible to eliminate the need to check specimen information for each preprocessing member using a barcode reader, a personal computer, etc., thereby improving the efficiency of pathology work.

[0085] Furthermore, in the embodiment, by using the head-mounted AR device 50, it is possible to check the specimen information linked to the preprocessing member without touching a barcode reader, keyboard, mouse, etc. Therefore, according to the embodiment, it is possible to prevent the specimen S from contaminating the equipment during the preprocessing of the specimen S.

[0086] When displaying multiple pieces of sample information associated with multiple preprocessing members, the sample information display unit 44 may display all of the sample information or may allow the operator X to select the sample information to display.

[0087] For example, the operator X may select a pre-processing member for displaying sample information based on vocalization, gestures, line of sight, etc., and the sample information display unit 44 may display the sample information superimposed on the selected pre-processing member.

[0088] Next, when the reading unit 45 (see FIG. 2) displays the specimen information superimposed on the preprocessing member on the display unit 54, the reading unit 45 outputs the specimen information from the speaker 57 (see FIG. 3). For example, in the example of FIG. 7, the reading unit 45 outputs a message M1, "ID:XXX-YYYY XX Taro," from the speaker 57 and reads it out to the operator X.

[0089] This allows the operator X wearing the AR device 50 to reconfirm the sample information associated with the preprocessing member by voice when visually checking the sample information. Therefore, according to the embodiment, it is possible to prevent the mix-up of the sample S when preprocessing the sample S.

[0090] In addition, in the embodiment, various specimen information associated with tag information printed on a preprocessing member can be displayed in AR, as shown in Fig. 8. For example, the operator X instructs the server device 10 on specimen information to be displayed (e.g., a pathology test request form, images, and medical records related to the pathological diagnosis of the specimen S) based on vocalization, gesture, line of sight, etc. In response to this instruction, the server device 10 displays various specimen information as superimposed information A2 to A4 on the display unit 54.

[0091] Furthermore, in an embodiment, as shown in FIG. 9, worker X may select superimposed information A2 to A4 to be enlarged and displayed based on speech, gestures, line of sight, etc., and server device 10 may enlarge and display the selected superimposed information A2 to A4 (superimposed information A4 in the figure) on display unit 54.

[0092] In this way, in the embodiment, by displaying various specimen information accompanying the tag information printed on the pre-processing member in AR, it is possible to make pathology work more efficient and to reduce the space required for pathology work.

[0093] 10, in an excision process for excising a region to be subjected to pathological diagnosis, the specimen S in the container 61 is transferred to a cassette 62. At this time, the member detection unit 41 detects the container 61, cassette 62, and tweezers 63, which are visually observed by the operator X and photographed by the camera 55, as pre-processing members.

[0094] Then, the component detection unit 41 displays, as a border, on the display unit 54, an indication that the container 61, the cassette 62, and the tweezers 63 have been detected. Furthermore, the specimen information display unit 44 displays the specimen information associated with the tag information printed on the container 61, superimposed on the container 61.

[0095] In the example of FIG. 10, the cassette 62 and the tweezers 63 are not marked with tag information, and therefore the specimen information that accompanies the tag information is not displayed.

[0096] Also, as shown in Figure 10, when the specimen S is removed from the container 61 or cassette 62 during pre-processing of the specimen S, the recording unit 47 (see Figure 2) records the state of the real space captured by the camera 55 in accordance with a signal from the operator X.

[0097] This allows for easy and accurate recording of tasks that need to be kept as evidence, thereby improving the efficiency of pathology work.

[0098] Furthermore, in the embodiment, the state of the real space is recorded using the head-mounted AR device 50, so that the viewpoint during work can be matched with the field of view of the camera 55. Therefore, according to the embodiment, it is possible to leave evidence recorded from an optimal viewpoint.

[0099] Furthermore, in the embodiment, when the state of the pre-processing work is recorded as evidence, the specimen information has already been acquired, so the evidence record can be linked to the specimen information without the operator X being aware of it. Therefore, according to the embodiment, the efficiency of pathology work can be further improved.

[0100] 11, in pre-processing of the specimen S, tweezers 63 are brought close to the container 61 in order to remove the specimen S from the container 61. At this time, the specimen information display unit 44 displays the specimen information of the pre-processing member on which tag information is printed, superimposed on the pre-processing member on which tag information is not printed.

[0101] For example, when tweezers 63 without tag information are brought within a certain distance of a container 61 with tag information for a certain period of time, the specimen information display unit 44 associates the specimen information associated with the tag information of the container 61 with the tweezers 63. Then, the specimen information display unit 44 displays the associated specimen information as superimposed information A5 on the tweezers 63.

[0102] This makes it possible to visually display that the same information as the specimen information associated with the container 61 is associated with the tweezers 63. Therefore, according to the embodiment, the operator X can be sure that the specimen S picked up with the tweezers 63 has been taken out from the container 61.

[0103] For example, when tweezers 63 without tag information are brought close to a container 61 with tag information, the specimen information display unit 44 may notify the operator X of this approach. For example, when tweezers 63 without tag information are brought close to a container 61 with tag information, the specimen information display unit 44 may notify the operator X of this approach by flashing the outlines of the container 61 and the tweezers 63.

[0104] Then, after notifying the operator X that the container 61 and the tweezers 63 have come close to each other, if the pre-processing components are maintained within a certain distance for a certain period of time, the specimen information display unit 44 may superimpose and display the specimen information of the container 61, on which tag information is printed, on the tweezers 63, on which tag information is not printed.

[0105] This makes it easier to visually recognize that the same information as the specimen information associated with the container 61 is about to be associated with the tweezers 63. Note that the method of notifying the operator X of the approach is not limited to flashing the outline of the target pre-processing member, and for example, the color of the outline of the target pre-processing member may be changed.

[0106] 12, in the pre-processing of the specimen S, the specimen S picked up with the tweezers 63 is placed in the cassette 62 by bringing the tweezers 63 close to the cassette 62. At this time, the specimen information display unit 44 displays the specimen information of the pre-processing member to which specimen information has been attached superimposed on the pre-processing member to which tag information has been printed.

[0107] For example, when tweezers 63 with specimen information attached thereto approach within a certain distance for a certain period of time to a cassette 62 without tag information printed thereon, the specimen information display unit 44 links the specimen information attached to the tweezers 63 to the cassette 62. Then, the specimen information display unit 44 displays the linked specimen information as superimposed information A6 on the cassette 62.

[0108] This makes it possible to visually display that the same information as the specimen information associated with the container 61 is associated with the cassette 62. Therefore, according to the embodiment, the operator X can be sure that the specimen S contained in the cassette 62 has been taken out from the container 61.

[0109] In the example of Figure 12, as in the example of Figure 11 described above, when tweezers 63 with tag information printed on them approach a cassette 62 without tag information printed on it, the specimen information display unit 44 may notify the operator X that such approach has occurred.

[0110] Then, after notifying the operator X that the cassette 62 and the tweezers 63 have come close to each other, if the pre-processing components are maintained within a certain distance for a certain period of time, the specimen information display unit 44 may superimpose and display the specimen information of the tweezers 63, on which tag information is printed, on the cassette 62, on which tag information is not printed.

[0111] This makes it easy to visually recognize that the same specimen information linked to the tweezers 63 is about to be linked to the cassette 62.

[0112] On the other hand, as shown in FIG. 13, if the specimen information of the tweezers 63 applied from the container 61 does not match the specimen information of the cassette 62, the notification unit 46 notifies the operator X that the specimen information does not match.

[0113] For example, when a cassette 62 and tweezers 63 with different specimen information attached thereto are brought within a certain distance for a certain period of time, the notification unit 46 outputs a message M2 from the speaker 57 to notify the operator X that the specimen information does not match. This makes it possible to prevent the specimen S from being mixed up during pre-processing.

[0114] In addition, the present disclosure is not limited to the case where the notification unit 46 notifies the operator X that the sample information does not match by audio information, but the notification unit 46 may also display visual information on the sample information display unit 44 that the sample information does not match.

[0115] Furthermore, in the pathology work support processing shown in Figures 7 to 13, the server device 10 may transmit the state of pre-processing captured by the camera 55 of the AR device 50 to a terminal located at another location based on instructions from the worker X, so that the worker X can receive advice from the user of that terminal through the AR device 50.

[0116] This allows worker X to easily receive advice from experts in different locations, thereby improving the efficiency of pathology work.

[0117] In addition, the server device 10 may be able to obtain advice from a user of a terminal located in another location through audio information output from the speaker 57 of the AR device 50, or through visual information displayed on the display unit 54 of the AR device 50.

[0118] <Another embodiment> In the embodiment described above, an example has been shown in which the server device 10 performs the main processing of the pathology work support processing, but the present disclosure is not limited to such an example. For example, at least a part of the processing performed by the control unit 40 of the server device 10 may be performed by the control unit 52 of the AR device 50 instead.

[0119] Furthermore, in the present disclosure, all of the processes of the pathology work support process according to the embodiment may be executed by the AR device 50. Fig. 14 is a functional block diagram showing the configuration of the AR device 50 according to another embodiment. The AR device 50 of this another embodiment is, for example, a standalone AR device that is not connected to the network N (see Fig. 2).

[0120] 14, an AR device 50 according to another embodiment includes a control unit 52, a storage unit 53, a display unit 54, a camera 55, a microphone 56, and a speaker 57. Note that the display unit 54, the camera 55, the microphone 56, and the speaker 57 according to the another embodiment are similar to those in the above embodiment (see FIG. 3), and therefore detailed description thereof will be omitted.

[0121] The storage unit 53 of the AR device 50 includes a specimen information storage unit 53a and an imaging data storage unit 53b.

[0122] The specimen information storage unit 53a stores the management history of the specimen S that is the subject of pathological diagnosis. Note that the specimen information storage unit 53a according to another embodiment has the same configuration as the specimen information storage unit 31 according to the embodiment shown in FIG. 4, and therefore detailed description thereof will be omitted.

[0123] The imaging data storage unit 53b stores a management history of imaging data recorded when the sample S is pre-processed. Note that the imaging data storage unit 53b according to another embodiment has the same configuration as the imaging data storage unit 32 according to the embodiment shown in Fig. 5, and therefore detailed description thereof will be omitted.

[0124] The control unit 52 of the AR device 50 includes a member detection unit 52a, a tag information acquisition unit 52b, a specimen information acquisition unit 52c, a specimen information display unit 52d, a reading unit 52e, a notification unit 52f, and a recording unit 52g.

[0125] The member detection unit 52a detects pre-processing members and their positions within the field of view of the camera 55 of the AR device 50. Note that the member detection unit 52a according to another embodiment has the same function as the member detection unit 41 (see FIG. 2) shown in the above embodiment, and therefore detailed description thereof will be omitted.

[0126] The tag information acquisition unit 52b acquires tag information printed on a pre-processing member present within the field of view of the camera 55. Note that the tag information acquisition unit 52b according to another embodiment has the same function as the tag information acquisition unit 42 (see FIG. 2) shown in the embodiment above, and therefore detailed description thereof will be omitted.

[0127] The specimen information acquiring unit 52c acquires specimen information associated with the tag information acquired by the tag information acquiring unit 52b from the specimen information storage unit 53a of the storage unit 30. Note that the specimen information acquiring unit 52c according to another embodiment has the same function as the specimen information acquiring unit 43 (see FIG. 2) shown in the embodiment above, and therefore detailed description thereof will be omitted.

[0128] The specimen information display unit 52d displays the specimen information acquired by the specimen information acquisition unit 52c by superimposing it on a pre-processing member on which tag information associated with the specimen information is printed, on the display unit 54 of the AR device 50. Note that the specimen information display unit 52d according to another embodiment has the same function as the specimen information display unit 44 (see FIG. 2) shown in the above embodiment, and therefore detailed description thereof will be omitted.

[0129] When the sample information is displayed on the display unit 54 in a manner superimposed on the preprocessing member, the reading unit 52e outputs the sample information from the speaker 57. Note that the reading unit 52e according to another embodiment has the same functions as the reading unit 45 (see FIG. 2) shown in the above embodiment, and therefore a detailed description thereof will be omitted.

[0130] If the sample information of a preprocessing member on which tag information is printed does not match the sample information of another preprocessing member on which tag information is printed, the notification unit 52f notifies the operator X that the sample information does not match. Note that the notification unit 52f according to another embodiment has the same function as the notification unit 46 (see FIG. 2) shown in the embodiment above, and therefore detailed description thereof will be omitted.

[0131] The recording unit 52g records the state of the real space where the pre-processing of the specimen S is performed, which is captured by the camera 55 of the AR device 50. The recording unit 52g also stores the recorded captured image data in the storage unit 53. Note that the recording unit 52g according to another embodiment has the same function as the recording unit 47 (see FIG. 2) shown in the above embodiment, and therefore detailed description thereof will be omitted.

[0132] As described above, in another embodiment, in a stand-alone AR device 50, the control unit 52 and the storage unit 53 have the same configurations as the control unit 40 and the storage unit 30 according to the above embodiment.

[0133] As a result, the AR device 50 according to the other embodiment can execute the various pathology work support processes described in the above embodiment. Therefore, according to the other embodiment, the efficiency of pathology work can be improved.

[0134] In the example of Figure 14, an example is shown in which a sample information storage unit is provided in the AR device 50, but the present disclosure is not limited to such an example, and for example, the sample information storage unit may be provided in the storage unit 30 (see Figure 2) of the server device 10 (see Figure 2).

[0135] <Procedure for pathology work support method> Next, a processing procedure of the pathology work support method according to the embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the procedure of the pathology work support processing executed by the pathology work support system 1 according to the embodiment.

[0136] In the pathology work support process according to the embodiment, first, the server device 10 detects a pre-processing member present within the field of view of the camera 55 of the AR device 50 (step S101). Then, the server device 10 acquires, from the camera 55, tag information to be printed on the pre-processing member present within the field of view of the camera 55 (step S102).

[0137] Next, the server device 10 acquires the specimen information associated with the tag information acquired in the process of step S102 from the storage unit 30 (step S103). Then, the server device 10 displays the specimen information acquired in the process of step S103 on the display unit 54 of the AR device 50, superimposed on the pre-processing member on which the tag information associated with the specimen information is printed (step S104).

[0138] Next, the server device 10 determines whether a preprocessing member without tag information printed thereon is approaching or in contact with a preprocessing member with tag information printed thereon (step S105).

[0139] Then, if a pre-processing member that does not have tag information printed thereon is approaching or in contact with the sample (step S105, Yes), the server device 10 superimposes and displays the specimen information of the pre-processing member that is approaching or in contact with the pre-processing member that does not have tag information printed thereon (step S106).

[0140] On the other hand, if a preprocessing member without tag information printed thereon is not approaching or contacting the object (step S105, No), the process of step S106 described above is skipped and the process proceeds to the next step S107.

[0141] Next, the server device 10 determines whether the preprocessing members to which different specimen information is assigned are close to or in contact with each other (step S107).

[0142] If the pre-processing components to which different specimen information is assigned are close to or in contact with each other (step S107, Yes), the server device 10 notifies the operator X that the specimen information of the pre-processing components does not match (step S108).

[0143] On the other hand, if the preprocessing members to which different specimen information is assigned are not close to or in contact with each other (No at step S107), the process at step S108 described above is skipped and the process proceeds to the next step S109.

[0144] Next, the server device 10 determines whether the pre-processing of the sample S has been completed (step S109). If the pre-processing of the sample S has been completed (step S109, Yes), the series of pathology work support processes is terminated. On the other hand, if the pre-processing of the sample S has not been completed (step S109, No), the process returns to step S101.

[0145] In addition, in parallel with the processing of steps S101 to S108 described above, the server device 10 records the state of the real space where the pre-processing of the specimen S is being performed, which is captured by the camera 55 of the AR device 50, in accordance with a signal from the operator X (step S110).

[0146] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present invention.

[0147] As described above, the pathology work support system 1 according to the embodiment is a pathology work support system that supports pathology diagnosis work, and includes an AR device 50 and a server device 10. The AR device 50 includes a camera 55 that captures images of real space and a display unit 54 that displays an image superimposed on the real space. The server device 10 controls the AR device 50. At least one of the server device 10 and the AR device 50 includes a tag information acquisition unit 42, a specimen information acquisition unit 43, and a specimen information display unit 44. The tag information acquisition unit 42 is located within the field of view of the camera 55 and acquires, from the camera 55, tag information that is printed on a member (preprocessing member) used in preprocessing a specimen S to be subjected to pathological diagnosis. The specimen information acquisition unit 43 acquires specimen information related to the specimen S that is associated with the acquired tag information. The specimen information display unit 44 displays the specimen information on the display unit 54 by superimposing it on the member (preprocessing member) on which the tag information is printed. This improves the efficiency of pathology work.

[0148] Furthermore, in the pathology work support system 1 according to the embodiment, the AR device 50 further includes an audio output unit (speaker 57) that outputs audio. At least one of the server device 10 and the AR device 50 further includes a reading unit 45. The reading unit 45 outputs the sample information from the audio output unit (speaker 57) when superimposing the sample information on the member (preprocessing member) on which the tag information is printed. This makes it possible to prevent mix-ups of the samples S when the samples S are preprocessed.

[0149] Furthermore, in the pathology work support system 1 according to the embodiment, at least one of the server device 10 and the AR device 50 further includes a notification unit 46. When the specimen information of a member (preprocessing member) on which tag information is printed does not match the specimen information of another member (preprocessing member) on which tag information is printed, the notification unit 46 notifies the operator X that the specimen information does not match. This makes it possible to prevent mix-ups of specimens S during preprocessing.

[0150] Furthermore, in the pathology work support system 1 according to the embodiment, when a member (preprocessing member) on which tag information is printed and a member (preprocessing member) on which tag information is not printed approach or come into contact within the field of view of the camera 55, the specimen information display unit 44 displays the specimen information of the member (preprocessing member) on which tag information is printed superimposed on the member (preprocessing member) on which tag information is not printed. This allows the operator X to be certain that the specimen S contained in the cassette 62 has been taken out from the container 61.

[0151] Furthermore, in the pathology work support system 1 according to the embodiment, when a member (preprocessing member) on which tag information is printed and a member (preprocessing member) on which tag information is not printed come close to or come into contact within the field of view of the camera 55, the specimen information display unit 44 notifies the camera 55 that the member (preprocessing member) on which tag information is printed and the member (preprocessing member) on which tag information is not printed have come close to or come into contact within the field of view of the camera 55, and then displays the specimen information of the member (preprocessing member) on which tag information is printed, superimposed on the member (preprocessing member) on which tag information is not printed. This makes it easy to visually recognize that the same information as the specimen information linked to a preprocessing member is about to be linked to another preprocessing member.

[0152] Furthermore, in the pathology work support system 1 according to the embodiment, the components (preprocessing components) are at least one of a pathology test request form for a pathological diagnosis of the specimen S, a container 61 for accommodating the specimen S, a cassette 62 for preprocessing the specimen S, and tweezers 63 for preprocessing the specimen S. This makes it possible to prevent mix-ups of specimens S during preprocessing.

[0153] In the pathology work support system 1 according to the embodiment, the tag information is a barcode printed on the member (preprocessing member), which allows tag information to be printed on the preprocessing member easily.

[0154] In the pathology work support system 1 according to the embodiment, the AR device 50 is a head-mounted AR device. This makes it possible to prevent the sample S from contaminating the devices during the pre-processing of the sample S.

[0155] The AR device 50 according to the embodiment includes a camera 55 that captures an image of real space, a display unit 54 that displays an image superimposed on the real space, and a control unit 52 that controls each unit. The control unit 52 also includes a tag information acquisition unit 52b, a specimen information acquisition unit 52c, and a specimen information display unit 52d. The tag information acquisition unit 52b acquires, from the camera 55, tag information that is marked on a member (preprocessing member) that is within the field of view of the camera 55 and is used in preprocessing a specimen S that is the target of pathological diagnosis. The specimen information acquisition unit 52c acquires specimen information about the specimen S that is associated with the acquired tag information. The specimen information display unit 52d displays the specimen information on the display unit 54, superimposed on the member (preprocessing member) on which the tag information is marked. This can improve the efficiency of pathological work.

[0156] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0157] 1. Pathology work support system 10 Server device 20 Communications Department 30 Storage section 31 Sample information storage unit 32 Imaging data storage unit 40 Control Unit 41 Component detection unit 42 Tag information acquisition unit 43 Specimen Information Acquisition Unit 44 Sample information display section 45 Reading section 46 Information Department 47 Recording Section 50 AR equipment 52 Control section 52a Component detection unit 52b Tag information acquisition section 52c Specimen Information Acquisition Unit 52d Sample information display section 52e reading section 52F Notification Department 52g Recording section 53 Storage section 53a Sample information storage unit 53b Imaging data storage unit 54 Display section 55 Camera (an example of a receiving unit) 56 Microphone (an example of a receiving section) 57 Speaker (an example of an audio output unit) 61 Container (example of component) 62 Cassette (example of component) 63 Tweezers (example of component) S specimen X Worker

Claims

1. In a pathology work support system that supports pathology diagnostic work, an AR device having a camera that captures an image of a real space and a display unit that displays an image superimposed on the real space; a server device that controls the AR device; Equipped with At least one of the server device and the AR device a tag information acquisition unit that acquires, from the camera, tag information that is printed on a member that is within the field of view of the camera and that is used in pre-processing a specimen that is the subject of pathological diagnosis; a specimen information acquiring unit that acquires specimen information related to the specimen that is associated with the acquired tag information; a specimen information display unit in the display unit that displays the specimen information superimposed on the member on which the tag information is printed; a notification unit that notifies an operator that the specimen information does not match when the specimen information of the member on which the tag information is marked does not match the specimen information of another member on which the tag information is marked; A pathology work support system with

2. A pathology work support system that supports pathology diagnostic work, an AR device having a camera that captures an image of a real space and a display unit that displays an image superimposed on the real space; a server device that controls the AR device; Equipped with At least one of the server device and the AR device a tag information acquisition unit that acquires, from the camera, tag information that is printed on a member that is within the field of view of the camera and that is used in pre-processing a specimen that is the subject of pathological diagnosis; a specimen information acquiring unit that acquires specimen information related to the specimen that is associated with the acquired tag information; a specimen information display unit in the display unit that displays the specimen information superimposed on the member on which the tag information is printed; and The specimen information display unit When the member on which the tag information is printed and the member on which the tag information is not printed come close to or come into contact within the field of view of the camera, the specimen information of the member on which the tag information is printed is superimposed on the member on which the tag information is not printed and displayed. Pathology work support system.

3. The specimen information display unit When the member on which the tag information is printed and the member on which the tag information is not printed come close to or come into contact within the field of view of the camera, a notification is issued that the member on which the tag information is printed and the member on which the tag information is not printed have come close to or come into contact within the field of view of the camera, and then the specimen information of the member on which the tag information is printed is displayed superimposed on the member on which the tag information is not printed. The pathology work support system according to claim 2.

4. The AR device includes: further comprising an audio output unit for outputting audio; At least one of the server device and the AR device The method further includes a reading unit that outputs the sample information from a voice output unit when the sample information is displayed superimposed on the member on which the tag information is printed.

4. The pathology work support system according to claim 1.

5. The member is at least one of a pathology test request form for a pathological diagnosis of a specimen, a container for accommodating the specimen, a cassette for pre-processing the specimen, and tweezers for pre-processing the specimen.

5. The pathology work support system according to claim 1.

6. The tag information is a bar code printed on the member.

6. The pathology work support system according to claim 1.

7. The AR device is a head-mounted AR device.

7. The pathology work support system according to claim 1.

8. A camera that captures real space; a display unit that displays an image superimposed on the real space; a control unit that controls each unit; Equipped with The control unit a tag information acquisition unit that acquires, from the camera, tag information that is printed on a member that is within the field of view of the camera and that is used in pre-processing a specimen that is the subject of pathological diagnosis; a specimen information acquiring unit that acquires specimen information related to the specimen that is associated with the acquired tag information; a specimen information display unit in the display unit that displays the specimen information superimposed on the member on which the tag information is printed; a notification unit that notifies an operator that the specimen information does not match when the specimen information of the member on which the tag information is marked does not match the specimen information of another member on which the tag information is marked; AR device having the above.

9. A camera that captures an image of real space; a display unit that displays an image superimposed on the real space; a control unit that controls each unit; Equipped with The control unit a tag information acquisition unit that acquires, from the camera, tag information that is printed on a member that is within the field of view of the camera and that is used in pre-processing a specimen that is the subject of pathological diagnosis; a specimen information acquiring unit that acquires specimen information related to the specimen that is associated with the acquired tag information; a specimen information display unit in the display unit that displays the specimen information superimposed on the member on which the tag information is printed; and The specimen information display unit When the member on which the tag information is printed and the member on which the tag information is not printed come close to or come into contact within the field of view of the camera, the specimen information of the member on which the tag information is printed is superimposed on the member on which the tag information is not printed and displayed. AR equipment.

Images