Medical information processing apparatus, method, and program

Abstract

To improve convenience of medical treatment using a virtual space.SOLUTION: A medical information processing apparatus according to an embodiment comprises a generation unit, a determination unit, and a changing unit. The generation unit generates an avatar including a first structure part formed from a human body structure, and a second structure part formed from a non-human body structure. The determination unit determines a changed portion and a non-changed portion in the first structure part and the second structure part on the basis of analyte information on an analyte. The changing unit adds a change to the changed portion in the avatar according to a result of determination.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a medical information processing apparatus, method, and program.

Background Art

[0002] Conventionally, in virtual spaces such as chats and online games, an avatar has been used as an animated image representing a user. The use of such virtual spaces has also spread to the medical field. For example, in recent years, a metaverse clinic where a doctor examines a patient in a virtual space is known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the convenience of medical treatment using a virtual space. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. The problems corresponding to the respective effects of each configuration shown in the embodiments described later can also be positioned as other problems.

Means for Solving the Problems

[0005] The medical information processing apparatus according to the embodiment includes a generation unit, a determination unit, and a modification unit. The generation unit generates an avatar including a first structural part composed of a human body structure and a second structural part composed of a non-human body structure. The determination unit determines a modified part and an unmodified part in the first structural part and the second structural part based on subject information regarding the subject. The modification unit makes a modification to the modified part in the avatar according to the determination result.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 4

Figure 5A

Figure 5B

Modes for Carrying Out the Invention

[0007] Hereinafter, embodiments of a medical information processing apparatus, method, and program will be described in detail with reference to the drawings. Note that the medical information processing apparatus, method, and program according to the present application are not limited to the embodiments shown below.

[0008] (First Embodiment) As described above, virtual spaces have been used in the medical field. However, in a virtual space, since the subject appears as an avatar, which is different from the real space, even if a change occurs in the subject's body in the real space, the avatar will not change unless the change is reflected in the avatar. In order to make an accurate diagnosis in medical treatment in a virtual space, it is desirable for the avatar to be changed according to the state of the disease, but it is troublesome for the user to change it each time. Therefore, the medical information processing apparatus according to the present embodiment can improve the convenience of medical treatment using a virtual space by changing the avatar based on the information of the subject.

[0009] FIG. 1 is a block diagram showing an example of the configuration of a medical information processing apparatus according to the first embodiment. Note that FIG. 1 shows a medical information processing system 1 including a medical information processing apparatus 30. For example, as shown in FIG. 1, the medical information processing system 1 includes a data storage device 10, a terminal device 20a, a terminal device 20b, and a medical information processing apparatus 30, and each device is connected to be communicable with each other via a network 2. Here, the network 2 includes, for example, an in-hospital LAN (Local Area Network) or WAN (Wide Area Network) installed in a hospital.

[0010] Note that various other devices and systems such as a medical image diagnostic apparatus may be connected to the network 2 shown in FIG. 1. A medical image diagnostic apparatus is a device that images a subject to generate a medical image, and includes, for example, an X-ray CT (Computed Tomography) apparatus, an MRI (Magnetic Resonance Imaging) apparatus, an X-ray diagnostic apparatus, an ultrasonic diagnostic apparatus, a SPECT (Single Photon Emission Computed Tomography) apparatus, a PET (Positron Emission computed Tomography) apparatus, and the like.

[0011] The data storage device 10 stores the subject information of the subject. Specifically, the data storage device 10 receives medical information data (raw data, medical images, analysis results, reports, etc.) and physical information data (height, weight, etc.) from the terminal device 20a, the terminal device 20b, and a medical image diagnostic device (not shown) via the network 2, and stores the received subject information data in the storage circuit within its own device for storage. Here, the data of the subject information described above is managed with information related to the data of the subject information (such as the collected date and time and the storage location of the data) linked thereto. For example, the data storage device 10 is realized by a PACS (Picture Archiving and Communication System), a HIS (Hospital Information System), a RIS (Radiology Information System), etc.

[0012] The terminal device 20a is a device operated by a subject who receives medical treatment in a virtual space. For example, the terminal device 20a is realized by a personal computer (PC), a tablet PC, a PDA (Personal Digital Assistant), a mobile phone such as a smartphone, etc. The terminal device 20a displays various information received from the medical information processing device 30 on its own display 22a and accepts various operations via the input interface 21a of its own device. For example, the terminal device 20a is operated by a subject who is in a place away from a doctor, such as at home or in a hospital ward during hospitalization, and accepts and displays various input operations related to medical treatment in the virtual space provided by the medical information processing device 30. Taking an example, the subject operates his or her own avatar in the virtual space displayed on the display 22a of the terminal device 20a via the input interface 21a to communicate with the doctor, thereby receiving medical treatment using the virtual space. Although a single terminal device 20a is shown in FIG. 1, actually, a plurality of terminal devices 20a will be connected to the network 2.

[0013] The terminal device 20b is a device operated by a doctor or the like working in a hospital. For example, the terminal device 20b is realized by a mobile phone such as a personal computer, a tablet PC, a PDA, or a smartphone. The terminal device 20b displays various information received from the medical information processing device 30 on its own display 22b, and accepts various operations via its own input interface 21b. For example, the terminal device 20b is operated by a doctor who is at a location away from the subject, such as in a hospital or outside the hospital, and accepts and displays various input operations related to medical treatment in the virtual space provided by the medical information processing device 30. For example, the doctor operates his or her avatar in the virtual space displayed on the display 22b of the terminal device 20b via the input interface 21b to communicate with the subject, thereby performing medical treatment using the virtual space. Although a single terminal device 20b is shown in FIG. 1, actually, a plurality of terminal devices 20b will be connected to the network 2.

[0014] Note that the above-described input interface 21a and input interface 21b can be realized with the same configuration as the input interface 32 described later. Also, the above-described display 22a and display 22b can be realized with the same configuration as the display 33 described later.

[0015] As shown in FIG. 1, the medical information processing device 30 includes a communication interface 31, an input interface 32, a display 33, a memory circuit 34, and a processing circuit 35, and provides medical treatment using a virtual space. For example, the medical information processing device 30 is realized by a computer device such as a server or a workstation.

[0016] The communication interface 31 controls the transmission and communication of various data transmitted and received between the medical information processing device 30 and each device connected via the network 2. Specifically, the communication interface 31 is connected to the processing circuit 35, and transmits the data received from each device on the network 2 to the processing circuit 35, or transmits the data received from the processing circuit 35 to each device on the network 2. For example, the communication interface 31 is realized by a network card, a network adapter, a NIC (Network Interface Controller), etc.

[0017] The input interface 32 receives input operations of various instructions and various information from the operator. Specifically, the input interface 32 is connected to the processing circuit 35, converts the input operation received from the operator into an electrical signal, and transmits it to the processing circuit 35. For example, the input interface 32 is realized by a trackball, a switch button, a mouse, a keyboard, a touch pad that performs an input operation by touching the operation surface, a touch screen in which the display screen and the touch pad are integrated, a non-contact input interface using an optical sensor, and a voice input interface, etc. Note that in this specification, the input interface 32 is not limited to only those equipped with physical operation components such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and transmits this electrical signal to the control circuit is also included in the examples of the input interface 32.

[0018] The display 33 displays various information and various data. Specifically, the display 33 is connected to the processing circuit 35, and displays various information and various data received from the processing circuit 35. For example, the display 33 is realized by a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, etc.

[0019] The memory circuit 34 stores various data and various programs. Specifically, the memory circuit 34 is connected to the processing circuit 35, stores the data received from the processing circuit 35, or reads out the stored data and transmits it to the processing circuit 35. For example, the memory circuit 34 is realized by a semiconductor memory element such as a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, a hard disk, an optical disk, or the like. For example, the memory circuit 34 stores various data and various programs related to medical treatment using a virtual space. In addition, the memory circuit 34 stores materials for generating an avatar used in the virtual space and subject information of the subject acquired from the terminal device 20a, the terminal device 20b, and the data storage device 10. Note that the memory circuit 34 may be realized by a cloud computer connected to the medical information processing device 30 via the network 2.

[0020] The processing circuit 35 controls the entire medical information processing device 30. Specifically, the processing circuit 35 controls the transmission and reception of information performed between the terminal device 20a, the terminal device 20b, and the data storage device 10, and controls various processes for providing medical treatment using a virtual space. Note that the processing circuit 35 can also perform various processes according to an input operation via the input interface 32.

[0021] For example, as shown in FIG. 1, in the present embodiment, the processing circuit 35 of the medical information processing device 30 executes a control function 351, a generation function 352, a determination function 353, and a change function 354. Here, the control function 351 is an example of a notification unit. The generation function 352 is an example of a generation unit. The determination function 353 is an example of a determination unit. The change function 354 is an example of a change unit.

[0022] The control function 351 constructs a virtual space and provides the constructed virtual space to the terminal device 20a and the terminal device 20b, thereby providing medical treatment using the virtual space. Note that the control function 351 can provide the virtual space to each terminal device by using the application installed in each terminal device. Alternatively, the control function 351 can also provide the virtual space to each terminal device by displaying the virtual space in a web browser.

[0023] Here, the above-mentioned virtual space is a virtual two-dimensional or three-dimensional space, in which an avatar representing the subject and an avatar representing the doctor are arranged. In addition to the avatars, the virtual space can also be arranged with buildings representing hospitals, walls, windows, doors for constructing examination rooms and other facilities inside the building, and corridors, elevators, escalators for moving inside the building. Also, desks, chairs, etc. can be arranged in the examination room in the virtual space.

[0024] The generation function 352 generates an avatar to be operated in the virtual space. Specifically, the generation function 352 generates an avatar including a first structural part composed of the human body structure and a second structural part composed of non-human body structures. For example, the generation function 352 generates an avatar including a body part constructed based on the information of the anatomical structure of the human body and a dressing part for changing the avatar according to the subject's preference. The processing by the generation function 352 will be described in detail later.

[0025] The determination function 353 determines the changed parts and non-changed parts in the first structural part and the second structural part based on the subject information regarding the subject. Specifically, the determination function 353 determines, based on the subject information acquired by the control function 351, the changed parts to be changed and the non-changed parts not to be changed in the body part and the dressing part. The processing by the determination function 353 will be described in detail later.

[0026] The change function 354 makes changes to the changed locations in the avatar according to the determination result. Specifically, the change function 354 makes changes to the locations determined as the changed locations by the determination function 353 in the avatar of the subject. Note that the processing by the change function 354 will be described in detail later.

[0027] The above-described processing circuit 35 is realized by, for example, a processor. In that case, each of the above-described processing functions is stored in the storage circuit 34 in the form of a program executable by a computer. Then, the processing circuit 35 reads and executes each program stored in the storage circuit 34 to realize the functions corresponding to the respective programs. In other words, when the processing circuit 35 reads each program, it has each processing function shown in FIG. 1.

[0028] Note that the processing circuit 35 may be configured by combining a plurality of independent processors, and each processor may realize each processing function by executing a program. Also, each processing function of the processing circuit 35 may be appropriately distributed or integrated into a single or a plurality of processing circuits and realized. Further, each processing function of the processing circuit 35 may be realized by a combination of hardware such as a circuit and software. Here, an example in which a program corresponding to each processing function is stored in a single storage circuit 34 has been described, but the embodiment is not limited to this. For example, programs corresponding to each processing function may be stored in a distributed manner in a plurality of storage circuits, and the processing circuit 35 may be configured to read and execute each program from each storage circuit. Note that a part of each processing function of the processing circuit 35 may be realized by a cloud computer connected to the medical information processing apparatus 30 via the network 2.

[0029] Next, the processing procedure by the medical information processing apparatus 30 will be described with reference to FIG. 2, and then the details of each process will be described. FIG. 2 is a flowchart showing an example of the processing procedure of the medical information processing apparatus 30 according to the first embodiment. Note that FIG. 2 shows the processing after the control function 351 starts providing the virtual space.

[0030] For example, as shown in FIG. 2, in the present embodiment, the generation function 352 generates a body part of an avatar representing a subject (step S101), and generates a dressing part of the avatar (step S102). For example, the generation function 352 generates the body part and the dressing part of the avatar according to the operation by the subject operating the terminal device 20a. This process is realized, for example, by the processing circuit 35 calling and executing a program corresponding to the generation function 352 from the storage circuit 34.

[0031] Subsequently, the determination function 353 determines the changed part and the unchanged part of the avatar based on the subject information (step S103). Further, the determination function 353 determines whether or not it is a change timing to make a change to the changed part of the avatar based on the subject information (step S104). This process is realized, for example, by the processing circuit 35 calling and executing a program corresponding to the determination function 353 from the storage circuit 34.

[0032] Next, when it is determined in step S104 that it is the change timing (step S104, YES), the change function 354 executes a change to the changed part in the avatar of the subject (step S105). This process is realized, for example, by the processing circuit 35 calling and executing a program corresponding to the change function 354 from the storage circuit 34. Note that until it is determined in step S104 that it is the change timing, the change function 354 is in a standby state (step S104, NO).

[0033] Hereinafter, details of each process executed by the medical information processing apparatus 30 will be described.

[0034] (Generation process of the body part) As described in step S101 of FIG. 2, the generation function 352 generates the torso part (the first structural part) of the avatar of the subject. Here, as described above, the torso part includes information on the anatomical structure of the human body and is generated based on that information. FIG. 3A is a diagram for explaining the torso part of the avatar according to the first embodiment. For example, as shown in FIG. 3A, the generation function 352 generates a torso part showing the anatomical three-dimensional structure of the human body such as the skin, muscles, bones, nerves, blood vessels, and various organs.

[0035] For example, the generation function 352 generates the torso part using a general-purpose library (such as an anatomical atlas, etc.) showing information on the anatomical structure of the human body. That is, the generation function 352 generates a torso part having information on the anatomical structure of an actual human body as the basis of the avatar. The general-purpose library is stored in the storage circuit 34 for each gender and age, and the generation function 352 reads out the general-purpose library corresponding to the subject and generates the torso part.

[0036] Note that the generation function 352 can also generate the torso part using the anatomical structure of the individual subject. For example, when medical images of the subject, photos of the body surface, and body information (such as photos showing height, weight, and body type) are stored in the data storage device 10, the generation function 352 can also generate the torso part using the stored medical images, photos, and body information. In this case, for example, the generation function 352 can generate a torso part of an avatar similar to the subject by correcting the torso part generated using the general-purpose library based on the medical images, photos, and body information of the subject. That is, the generation function 352 corrects the overall shape of the torso part using the height and weight of the subject, photos showing the appearance of the subject, corrects the internal structure of the torso part using the medical images of the subject, and corrects the skin condition using the photos of the body surface.

[0037] Also, for example, when medical images of the entire body of a subject are stored, the generation function 352 can also generate a body part using the stored medical images without using a general-purpose library. That is, the generation function 352 generates the overall shape and internal structure of the body part using the medical images of the entire body of the subject. Note that the above-described method for generating the body part may be arbitrarily selectable by the subject. Further, the generation function 352 associates the generated body part with the subject information and stores it in the storage circuit 34.

[0038] (Generation process of dressing-up part) As described in step S102 of FIG. 2, the generation function 352 generates a dressing-up part (second structural part) of the avatar of the subject. Here, as described above, the dressing-up part is the appearance part of the avatar that can be changed according to the preference of the subject. That is, the generation function 352 generates a dressing-up part according to the operation of the subject who operates the terminal device 20a.

[0039] FIG. 3B is a diagram for explaining the dressing-up part of the avatar according to the first embodiment. For example, the generation function 352 generates a dressing-up part showing the appearance of the avatar such as hairstyle, clothing, shoes, as well as beard, glasses, accessories, makeup, bag, etc. as shown in FIG. 3B. Note that the dressing-up part may also include, in addition to the above, skin color, skin texture, cheek bulge, nose height, eye size, eyebrow thickness, etc.

[0040] For example, the control function 351 causes the display 22a of the terminal device 20a to display a display screen for generating a switching part of the avatar. The subject operates the input interface 21a while viewing the information displayed on the display 22a to perform selection, change, and adjustment of the appearances of the various avatars described above. The generation function 352 generates a dressing-up part according to the operation via the input interface 21a. Here, the generation function 352 associates the generated dressing-up part with the subject information and stores it in the storage circuit 34 so that the state of the dressing-up part set by the subject can be reused.

[0041] Note that in FIG. 2, a case is shown where the avatar's dressing part is generated after the avatar's body part is generated and before the changed parts and non-changed parts are determined. However, the embodiment is not limited to this, and the avatar's dressing part may be generated at any timing. That is, the avatar's dressing part may be changed at an arbitrary timing while moving in the virtual space.

[0042] As described above, when the generation function 352 generates the avatar's body part and dressing part, the subject moves in the virtual space displayed on the display 22a of the terminal device 20a as the avatar and receives medical treatment using the virtual space. FIG. 4 is a diagram showing a state of medical treatment using the virtual space according to the first embodiment. For example, the subject operates the avatar via the input interface 21a and goes to an examination room in the hospital installed in the virtual space to receive an examination by a doctor. Here, the state shown in FIG. 4 is displayed on the display 22a of the terminal device 20a operated by the subject under the control of the control function 351 and is also displayed on the display 22a of the terminal device 20b operated by the doctor.

[0043] The doctor examines the subject via the input interface 21b of the terminal device 20b while looking at the display 22a of the terminal device 20b. In the examination in the virtual space, the doctor and the subject communicate with each other, for example, by voice or input operations. Here, the avatar operated by the subject is one in which the body part showing the anatomical structure is shown by the appearance of the dressing part. That is, the avatar according to the present embodiment has the information of the anatomical structure of the human body with the appearance determined according to the preference of the subject. In the present embodiment, by making changes based on the subject information to such an avatar, the convenience in medical treatment using the virtual space is improved.

[0044] (Determination process of changed parts and non-changed parts) As described in step S103 of FIG. 2, the determination function 353 determines a change location where the avatar is changed based on the subject information and a non-change location where no change is made. Here, the determination function 353 can perform determination processing according to the situation of the subject. These will be described in order below.

[0045] For example, based on the medical information of the subject, the determination function 353 determines the location corresponding to the medical information in the body part (the first structural part) as the change location, and determines the body part other than the location determined as the change location in the body part and the replacement part (the second structural part) as the non-change location. Taking an example, after the avatar of the subject is generated, when a medical image or a photo of an affected part such as a skin disease is acquired, the determination function 353 determines the change location and the non-change location of the avatar based on the acquired medical image or photo. That is, the determination function 353 determines the location of the body part corresponding to the part where the medical image or photo is collected as the change location, and determines the other locations in the body part and the replacement part as the non-change locations. In other words, the determination function 353 determines to make a change only to the location of the body part corresponding to the part where the medical image is collected.

[0046] Also, for example, based on the physical information of the subject, the determination function 353 determines the location corresponding to the physical information in the body part (the first structural part) as the change location, and determines the body part other than the location determined as the change location in the body part and the replacement part (the second structural part) as the non-change location. Taking an example, after the avatar of the subject is generated, when information about the subject's weight or a photo of the whole body is acquired, the determination function 353 determines the change location and the non-change location of the avatar based on the acquired weight information or photo. For example, the determination function 353 determines the body part involved in the increase or decrease of weight or the body shape obtained from the photo as the change location, and determines the replacement part as the non-change location.

[0047] As described above, the determination function 353 can determine the body part to be the modification part and the non-modification parts other than the modification part based on the medical information and physical information of the subject. Here, the medical information of the subject may be the one collected at the actual hospital where the subject undergoes medical examinations and stored in the data storage device 10, or may be the one collected at another hospital and stored in the data storage device 10. Alternatively, the medical information of the subject may be the one obtained by the subject himself / herself and input via the terminal device 20a. Also, the physical information of the subject may be the one obtained by the subject himself / herself and input via the terminal device 20a, or may be the one obtained at the hospital and stored in the data storage device 10.

[0048] The control function 351 acquires the medical information and physical information of the subject stored by the data storage device 10, or the medical information and physical information input via the terminal device 20a. Then, the determination function 353 determines the modification part and non-modification parts of the avatar based on the medical information and physical information acquired by the control function 351.

[0049] Here, the determination function 353 can also perform the determination process based only on the information related to the disease. For example, after the avatar of the subject is generated, when a medical image of the subject and a photograph of the whole body of the subject are acquired, the determination function 353 determines only the position of the body part corresponding to the part where the medical image is collected as the modification part, and determines the other positions and the replacement part in the body part as non-modification parts. That is, the determination function 353 determines not to perform the modification of the body part based on the body shape of the subject obtained from the photograph of the whole body of the subject.

[0050] In the above-described embodiment, the case of determining the modification part and non-modification parts based on the medical information and physical information of the subject has been described. However, the embodiment is not limited thereto, and the determination function 353 can also determine the modification part and non-modification parts according to the medical department where the subject receives medical treatment.

[0051] For example, the determination function 353 can determine the modified parts and non-modified parts in the part included in the medical information according to the department of medicine where the subject receives medical treatment. For example, when a chest CT image of the subject is acquired and the department of medicine where the subject receives medical treatment is the department of respiratory medicine, the determination function 353 determines the lungs included in the chest CT image as the modified parts and determines other organs (such as the heart, etc.) included in the chest CT image as the non-modified parts. Also, for example, when an X-ray image of the foot and a photograph showing a skin disease are acquired and the department of medicine where the subject receives medical treatment is the department of orthopedics, the determination function 353 determines the foot as the modified part and determines the skin as the non-modified part.

[0052] Also, for example, the determination function 353 can determine the modified parts and non-modified parts in the dressing change part according to the department of medicine where the subject receives medical treatment. For example, when the department of medicine where the subject receives medical treatment is the department of dermatology, the determination function 353 determines the skin color, texture, makeup, etc. in the dressing change part as the non-modified parts. That is, the determination function 353 determines so as not to change the appearance of the skin. Also, for example, when the department of medicine where the subject receives medical treatment is the department of ophthalmology, the determination function 353 determines the glasses in the dressing change part as the non-modified parts.

[0053] Also, for example, when the department of medicine where the subject receives medical treatment is the department of psychiatry, the determination function 353 can also determine all items in the dressing change part as modifiable parts. That is, the determination function 353 determines all items in the dressing change part as modifiable parts so that the mental state of the subject can be easily reflected in the avatar.

[0054] As described above, the determination function 353 determines the modified parts and non-modified parts in the avatar, but the non-modified parts may also be set by the user (subject, doctor, etc.). For example, a doctor can also specify the non-modified parts in the avatar of the subject for efficient examination.

[0055] (Determination process of change timing) As described in step S104 of FIG. 2, the determination function 353 determines the change timing for making a change to the change location. Specifically, the determination function 353 determines the change timing for making a change to the change location based on the subject information. Here, the determination function 353 can determine various timings as the change timing. These will be described in order below.

[0056] For example, the determination function 353 can determine the timing when the medical image of the subject is collected as the timing for making a change to the change location. That is, the determination function 353 determines to change the avatar at the time when the medical image of the subject is acquired by the control function 351. Therefore, in this case, when the change location and non-change location in the base body are determined by the determination function 353, the change to the change location will be immediately reflected.

[0057] Also, for example, the determination function 353 can also determine the timing when the subject undergoes an examination using the avatar as the timing for making a change to the change location. Here, the timing when the subject undergoes an examination includes, for example, the time when the avatar representing the subject enters the examination room in the virtual space, the time desired by the doctor, etc. That is, the determination function 353 determines the time when the subject operates the avatar and enters the examination room in the virtual space, or the time specified by the doctor via the terminal device 20b, as the timing for making a change to the change location of the base body.

[0058] Also, for example, the determination function 353 can determine the timing of an input operation from the user (the subject or the doctor) as the change timing. That is, the determination function 353 determines the timing to add a change location in response to receiving an input operation from the user related to the diagnosis and treatment of the subject using the avatar. In such a case, for example, when the avatar becomes modifiable (e.g., when a medical image is acquired), the control function 351 notifies the user (the terminal device 20a or the terminal device 20b) of information indicating that the avatar is modifiable. The determination function 353 determines the time point when an approval operation for the notification is received as the timing to add a change to the change location.

[0059] As described above, the determination function 353 can determine various timings as the change timing for the change location of the elemental part related to the examination. However, since the change in the elemental part may be made during the examination, it may also return to the elemental part before the change when the examination is over. That is, the determination function 353 can also determine the time period between predetermined time points as the change timing so that the change location in the elemental part is changed when entering the examination room and returns to the elemental part before the change when leaving the examination room.

[0060] Also, since it is important that the change location in the elemental part is changed during the examination, it may be made so that the examination room cannot be entered until the change location in the elemental part is changed. In such a case, for example, when the avatar of the subject whose change location has not been changed tries to enter the examination room, the control function 351 sends a notification prompting the change to the change location to the terminal device 20a operated by the subject and controls so that the avatar cannot enter the examination room.

[0061] The determination function 353 can determine various change timings not only for the changed parts in the above-described main body part but also for the changes in the replacement part. As described above, the change in the replacement part can be made not only at the time of avatar generation but also at the time of movement, etc., and the determination function 353 can also determine the changeable timing and the unchangeable timing for this change respectively.

[0062] For example, the determination function 353 can make the replacement part changeable at any timing outside the hospital in the virtual space and make the replacement part unchangeable inside the hospital. Also, for example, the determination function 353 can make the replacement part changeable at any timing outside the examination room and make the replacement part unchangeable inside the examination room.

[0063] Note that the control function 351 can also prompt to return the change to the replacement part to the state before the change when the change in the replacement part is not allowed inside the hospital (or inside the examination room). For example, when the avatar of the subject receiving dermatological examination has the skin state changed by the replacement part, the control function 351 sends a prompting notification to the terminal device 20a operated by the subject to return the replacement part to the state before the change.

[0064] (Change process for the changed part) As described in step S105 of FIG. 2, the change function 354 executes the change for the changed part at the change timing. Specifically, the change function 354 makes a change to the changed part determined by the determination function 353 at the change timing determined by the determination function 353.

[0065] For example, the modification function 354 makes a modification that reflects medical information at the location to be modified in the main body part (the first structural part). Also, for example, the modification function 354 makes a modification that reflects physical information at the location to be modified in the main body part (the first structural part). Here, the modification function 354 makes a modification to the location to be modified of the avatar at the modification timing determined by the determination function 353. That is, the modification function 354 makes a modification to the location to be modified when the above-described various modification timings occur.

[0066] For example, when making a modification based on medical information, the modification function 354 changes the location to be modified in the main body part to the state included in the medical information. To give an example, when an X-ray image of the foot is acquired as medical information, the modification function 354 changes the foot in the main body part to the state depicted in the X-ray image. That is, the modification function 354 changes the state of bones, muscles, tendons, etc. included in the foot in the main body part to the state shown in the X-ray image. Also, for example, when a photograph of the face skin is acquired as medical information, the modification function 354 changes the state of the skin at the corresponding position on the face in the main body part to the state shown in the photograph.

[0067] Note that the modification function 354 can also associate and store medical information at the location to be modified in the main body part. For example, the modification function 354 can associate and store an X-ray image with the foot in the main body part, or associate and store a photograph with the face skin.

[0068] Also, for example, when making a modification based on physical information, the modification function 354 changes the location to be modified in the main body part to the body shape based on the physical information. To give an example, when the weight is acquired as physical information, the modification function 354 changes the abdominal circumference of the main body part along with the rate of increase or decrease in weight. Also, for example, when a photograph showing the whole body of the subject is acquired as physical information, the modification function 354 changes the body shape of the main body part to match the body shape of the subject shown in the photograph.

[0069] The subject undergoes medical treatment in the virtual space using the avatar after the changed part of the base body is changed by the change function 354 as described above. For example, when receiving a medical examination using the avatar after the changed part of the base body is changed, a doctor can observe the avatar as shown in FIGS. 5A and 5B. FIGS. 5A and 5B are diagrams showing an example of the avatar after the change according to the first embodiment. Note that FIGS. 5A and 5B show the state after the left ankle is changed as the changed part.

[0070] When the left ankle is changed as the changed part, for example, in the avatar whose appearance is changed by the dressing part, as shown in FIG. 5A, a region a1 for clearly indicating the changed part is displayed on the avatar. Thereby, the doctor can grasp the affected part of the subject at a glance. Then, as shown by the arrow a2 in FIG. 5B, the base body of the avatar is displayed in a state where the left ankle is swollen.

[0071] For example, the doctor observes the state of the changed base body shown in FIG. 5B by selecting the region a1 of the avatar that has entered the examination room in the state shown in FIG. 5A. Since the changed part (left ankle) of the base body shown in FIG. 5B is changed to reflect medical information, it can contribute to the diagnosis. Further, since the changed base body shown in FIG. 5B can also associate medical information (for example, an X-ray image) with the changed part (left ankle), the doctor can also confirm the medical information, which is the original information before deforming the base body.

[0072] In the above example, the case where a single change is made to the avatar has been described. However, through the follow-up observation of the affected part, multiple changes may be made to the avatar. In this case, the memory circuit 34 may store the change history of the avatar, and the control function 351 may control to display the information. That is, the affected part may be shown to change over time by the avatar. In such a case, for example, the change function 354 generates virtual body parts with the state of the affected part changed based on the change history. The control function 351 causes the terminal device 20b to continuously display the generated virtual body parts, thereby displaying an avatar showing the change over time of the affected part.

[0073] Also, in the above example, the case where an avatar is displayed with the changed part of the virtual body part hidden by the dressing part has been described. However, the embodiment is not limited to this, and an avatar may be displayed with the changed part of the virtual body part visible. For example, when the changed part is the skin of the back of the hand, an avatar may be displayed with the virtual body part of the changed part visible and the other virtual body part hidden by the dressing part. In this case, the doctor can observe the state of the affected part shown in the virtual body part by looking at the hand of the avatar who has entered the examination room.

[0074] Also, the change function 354 can also reflect the changes made to the dressing part of the avatar. As described above, the dressing part can also be controlled to be changed at any timing during the movement in the virtual space. That is, when the subject performs an operation to change the dressing part via the terminal device 20a, the change function 354 makes a change to the dressing part of the avatar on the condition that the timing when the operation is received is a changeable timing.

[0075] As described above, according to the first embodiment, the generation function 352 generates an avatar including a first structural part composed of a human body structure and a second structural part composed of a non-human body structure. The determination function 353 determines changed parts and unchanged parts in the first structural part and the second structural part based on the subject information regarding the subject. The change function 354 makes changes to the changed parts in the avatar according to the determination result. Therefore, the medical information processing apparatus 30 according to the first embodiment can display an avatar suitable for medical treatment using a virtual space by determining the changed parts and unchanged parts in the avatar based on the subject information, and can improve the convenience of medical treatment using a virtual space.

[0076] Also, according to the first embodiment, the determination function 353 determines, based on the medical information of the subject, the parts corresponding to the medical information in the first structural part as the changed parts, and determines the parts other than the parts determined as the changed parts in the first structural part and the second structural part as the unchanged parts. The change function 354 makes changes to the changed parts in the first structural part by reflecting the medical information. Therefore, the medical information processing apparatus 30 according to the first embodiment can change to an avatar reflecting the medical information, and can perform medical treatment more accurately.

[0077] Also, according to the first embodiment, the determination function 353 determines, based on the physical information of the subject, the parts corresponding to the physical information in the first structural part as the changed parts, and determines the parts other than the parts determined as the changed parts in the first structural part and the second structural part as the unchanged parts. The change function 354 makes changes to the changed parts in the first structural part by reflecting the physical information. Therefore, the medical information processing apparatus 30 according to the first embodiment can change to an avatar reflecting the physical information, and can perform medical treatment based on a more accurate state of the subject.

[0078] Further, according to the first embodiment, the determination function 353 determines the modified part and the non-modified part according to the department of medicine where the subject undergoes a medical examination. Therefore, the medical information processing apparatus 30 according to the first embodiment can use an appropriate avatar for each department of medicine, and can further improve the convenience of medical treatment using the virtual space.

[0079] Further, according to the first embodiment, the determination function 353 further determines the timing for making a change to the modified part based on the subject information. The change function 354 makes a change to the modified part of the avatar at the determined timing. Therefore, the medical information processing apparatus 30 according to the first embodiment can make a change to the avatar at an appropriate timing in medical treatment using the virtual space.

[0080] Further, according to the first embodiment, the determination function 353 determines the time point when the medical image of the subject is collected as the timing for making a change to the modified part. Therefore, the medical information processing apparatus 30 according to the first embodiment can display an avatar that always reflects the latest state of the subject.

[0081] Further, according to the first embodiment, the determination function 353 determines the time point when the subject undergoes an examination using the avatar as the timing for making a change to the modified part. Therefore, the medical information processing apparatus 30 according to the first embodiment can display an avatar in the state desired by the subject except during the examination.

[0082] Further, according to the first embodiment, the determination function 353 further determines the timing for making a change to the modified part in response to receiving an input operation from a user related to the medical treatment of the subject using the avatar. The change function 354 makes a change to the modified part of the avatar at the determined timing. Therefore, the medical information processing apparatus 30 according to the first embodiment can make a change to the avatar at an arbitrary timing desired by the user.

[0083] Also, according to the first embodiment, the control function 351 notifies the user of information indicating that the avatar is changeable. The determination function 353 determines the time point when an approval operation for the notification by the control function 351 is received as the timing to apply the change to the change point. Therefore, the medical information processing apparatus 30 according to the first embodiment enables the user to change the avatar at the timing desired by the user.

[0084] (Other embodiments) In the above-described embodiment, the case where the dressing change unit changes the skin color, skin texture, cheek swelling, nose height, eye size, eyebrow thickness, etc. has been described. However, the embodiment is not limited to this, and the dressing change unit may change the number of eyes, the number of arms, the number of heads, etc. to a configuration other than the human body. For example, the dressing change unit may generate an avatar with four arms. Also, when the part where such a change is made is not the affected part, it may remain as it is during the examination. For example, when the affected part is the foot, an avatar with four arms may be used for the examination.

[0085] Note that in the above-described embodiment, an example in which the notification unit, generation unit, determination unit, and change unit in this specification are realized by the control function, generation function, determination function, and change function of the processing circuit, respectively, has been described, but the embodiment is not limited to this. For example, the notification unit, generation unit, determination unit, and change unit in this specification may realize the same function not only by the control function, generation function, determination function, and change function described in the embodiment, but also by only hardware, only software, or a mixture of hardware and software.

[0086] Also, the term "processor" used in the description of the above-described embodiments means, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a circuit such as an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). Here, instead of storing a program in a storage circuit, the program may be directly incorporated into the circuit of the processor. In this case, the processor realizes its function by reading and executing the program incorporated in the circuit. Also, each processor of the present embodiment is not limited to being configured as a single circuit for each processor, and a plurality of independent circuits may be combined to be configured as one processor to realize its function.

[0087] Here, the medical information processing program executed by the processor is provided by being pre-installed in a ROM (Read Only Memory), a memory circuit, or the like. Note that this medical information processing program may be recorded and provided on a non-transitory computer-readable storage medium such as a CD (Compact Disk)-ROM, FD (Flexible Disk), CD-R (Recordable), DVD (Digital Versatile Disk) in a file in a form installable or executable on these devices. Further, this medical information processing program may be stored on a computer connected to a network such as the Internet and provided or distributed by being downloaded via the network. For example, this medical information processing program is composed of modules including the above-described respective processing functions. As actual hardware, the CPU reads out the medical image processing program from a storage medium such as a ROM and executes it, whereby each module is loaded onto the main storage device and generated on the main storage device.

[0088] Also, in the above-described embodiments and modified examples, each component of each illustrated device is conceptually functional and does not necessarily have to be physically configured as illustrated. That is, the specific form of the distribution or integration of each device is not limited to that illustrated, and all or a part of it can be functionally or physically distributed or integrated in any unit according to various loads, usage situations, and the like. Further, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic.

[0089] Also, among the respective processes described in the above-described embodiments and modified examples, all or part of the processes described as being automatically performed can be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, regarding the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings, they can be arbitrarily changed unless otherwise specified.

[0090] According to at least one of the embodiments described above, the convenience of medical treatment using a virtual space can be improved.

[0091] Although several embodiments have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, as well as in the invention described in the claims and its equivalent scope.

Description of Reference Numerals

[0092] 30 Medical information processing device 351 Control function 352 Generation function 353 Decision function 354 Change function

Claims

1. A generation unit that generates an avatar including a first structural part composed of a human body structure and a second structural part composed of a non-human body structure; A determination unit that determines changed parts and non-changed parts in the first structural part and the second structural part based on subject information regarding a subject; A modification unit that modifies the changed parts in the avatar according to the determination result; A medical information processing apparatus comprising the above.

2. The determination unit determines, based on medical information of the subject, a location corresponding to the medical information in the first structural part as the changed part, determines locations other than the determined changed part in the first structural part and the second structural part as the non-changed parts, The modification unit makes a modification reflecting the medical information to the changed part in the first structural part. The medical information processing apparatus according to claim 1.

3. The determination unit determines, based on physical information of the subject, a location corresponding to the physical information in the first structural part as the changed part, determines locations other than the determined changed part in the first structural part and the second structural part as the non-changed parts, The modification unit makes a modification reflecting the physical information to the changed part in the first structural part. The medical information processing apparatus according to claim 1.

4. The determination unit determines the changed parts and the non-changed parts according to the medical department where the subject receives medical treatment. The medical information processing apparatus according to claim 1.

5. The determination unit further determines a timing for making a modification to the changed part based on the subject information, The modification unit makes a modification to the changed part in the avatar at the determined timing. The medical information processing apparatus according to any one of claims 1 to 4.

6. The determination unit determines the time point when the medical image of the subject is collected as the timing for making a modification to the changed part. The medical information processing apparatus according to claim 5.

7. The determination unit determines the time point when the subject undergoes an examination using the avatar as the timing for making a modification to the changed part. The medical information processing apparatus according to claim 5.

8. The determination unit further determines the timing for adding the changed part in response to receiving an input operation from a user related to the medical treatment of the subject using the avatar. The medical information processing apparatus according to any one of claims 1 to 4, wherein the changing unit makes a change to the changed portion in the avatar at a determined timing.

9. further comprising a notification unit that notifies the user of information indicating that the avatar is changeable; The medical information processing apparatus according to claim 8, wherein the determination unit determines, as the timing for making a change to the changed portion, the time point when an approval operation for the notification by the notification unit is received.

10. generating an avatar including a first structural part composed of a human body structure and a second structural part composed of a non-human body structure; determining changed portions and non-changed portions in the first structural part and the second structural part based on subject information regarding the subject; making a change to the changed portion in the avatar according to the determination result; A medical information processing method comprising:

11. a generation function for generating an avatar including a first structural part composed of a human body structure and a second structural part composed of a non-human body structure; a determination function for determining changed portions and non-changed portions in the first structural part and the second structural part based on subject information regarding the subject; a change function for making a change to the changed portion in the avatar according to the determination result; A medical information processing program that causes a computer to execute.

Images