Program

Abstract

To provide a technique to support recording of measured biological information.SOLUTION: A program for causing a computer of a user terminal to implement a function to acquire, as measured values, analysis results of images of a measuring instrument that measures biological information picked up for every measurement time, a function to display the acquired measured values on the display part of the user terminal in association with the images of the acquisition source, and a function to receive correction of the measured values through operation to the user terminal.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] This disclosure relates to a program.

Background Art

[0002] In the Hypertension Treatment Guidelines 2019, it is recommended to measure home blood pressure twice each in the morning and at night. For this reason, the hypertension treatment assistance app (hereinafter also referred to as the "treatment app") prescribed to patients is provided with input fields twice each in the morning and at night.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The four recording operations, twice each in the morning and at night, are a burden for patients who have no habit of measuring home blood pressure, and there is a concern that the measurement and recording of home blood pressure will not continue. As one measure to reduce the work burden of users, there is also a health management app that uses the camera function of a smartphone. When using this type of health management app, the measured value can be automatically read and recorded simply by imaging the liquid crystal screen of the blood pressure monitor. However, in this type of health management app, the types of blood pressure monitors for which images can be read are limited. By the way, the reading accuracy depends on the imaging quality of the image. For this reason, when the imaging quality is low, the read value will be an incorrect numerical value.

[0005] One aspect of this disclosure aims to provide a technique for assisting the recording of measured biological information.

Means for Solving the Problems

[0006] As one aspect of the present disclosure, a program is provided for causing a computer of a user terminal to implement a function of acquiring, as a measurement value, an analysis result of an image obtained by imaging a measuring device that measures biological information for each measurement session, a function of displaying the acquired measurement value on a display unit of the user terminal in association with the original image, and a function of accepting correction of the measurement value through an operation on the user terminal.

Advantages of the Invention

[0007] According to one aspect of the present disclosure, it is possible to assist in recording measured biological information.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

MODE FOR CARRYING OUT THE INVENTION

[0009] <Terms> First, terms used in the embodiments described below will be explained. "Therapeutic app" refers to an app approved under the Pharmaceutical Affairs Law. There are two types of therapeutic apps: an app that runs on a terminal operated by a doctor and an app that runs on a terminal operated by a patient (i.e., a patient terminal). A patient terminal is an example of a user terminal. In the embodiments described below, the term "therapeutic app" is used in the sense of an app that runs on a terminal operated by a patient. In the following description, unless otherwise specified, "patient" is referred to as "user".

[0010] Therapeutic apps are used for recording biological information measured outside a medical institution, such as mood. Therapeutic apps are available from app stores. Examples of diseases for which therapeutic apps have been approved include hypertension, nicotine dependence, and insomnia. Examples of diseases for which therapeutic apps are under development include NASH (non-alcoholic steatohepatitis) and kidney disease.

[0011] "Health management app" refers to an app that is not approved under the Pharmaceutical Affairs Law and is used for recording biological information. Therefore, the users of health management apps are not necessarily so-called patients. Health management apps run on a terminal operated by a user (i.e., a user terminal). Health management apps are also available from app stores. Health management apps are also used for recording biological information, such as mood. Therapeutic apps and health management apps are examples of application programs that a user (including a patient) runs on a user terminal to record their own biological information.

[0012] "Biological information" refers to information measured from the body through a measuring device, such as blood pressure, pulse, respiration, body temperature, weight, blood glucose level, Na / K, oxygen saturation, and electroencephalogram. "Measuring device" refers to a device used for measuring biological information. Devices are classified, for example, as health management devices or medical devices. Measuring devices include, for example, automatic electronic blood pressure monitors, manual electronic blood pressure monitors, electronic thermometers, urine glucose meters for self-examination, blood glucose self-measuring devices, natriuretic potassium meters, pulse oximeters, weighing scales, body fat scales, body composition analyzers, activity monitors, pedometers, and heart rate monitors.

[0013] In the case of the embodiments described below, the "medical institution" is assumed to be an insurance medical institution. More specifically, the medical institution refers to an insurance medical institution to which a doctor who issues a prescription code necessary for activating the treatment application belongs. However, if pharmacists, health care providers, nurses, dietitians, hospital staff, and other medical personnel can issue prescription codes due to regulatory relaxation or the like, the medical institution also includes facilities and institutions where these medical personnel are present. Note that the issuance of the prescription code is not limited to issuance by medical treatment, and may also be issuance by out-of-pocket medical treatment (i.e., private medical treatment) or issuance by mixed medical treatment. Incidentally, medical treatment includes not only face-to-face medical treatment but also online medical treatment.

[0014] <Embodiment 1> Hereinafter, Embodiment 1 of the present disclosure will be described with reference to the drawings. <Device Configuration> FIG. 1 is a diagram for explaining a configuration example of a user terminal 10 assumed in Embodiment 1. The user terminal 10 is an information terminal operated by a user, and a treatment application is installed therein. Therefore, the user terminal 10 in the present embodiment is an information terminal operated by a patient for whom the treatment application has been prescribed.

[0015] The user terminal 10 in the present embodiment is a smartphone. The user terminal 10 shown in FIG. 1 includes a processor 11, a ROM (Read Only Memory) 12 in which a BIOS (= Basic Input Output System) or the like is stored, a RAM (Random Access Memory) 13 used as a work area of the processor 11, an auxiliary storage device 14, a communication interface 15, a display 16, an input reception device 17, and a camera 18. Each device is connected through signal lines (not shown).

[0016] The processor 11, the ROM 12, and the RAM 13 constitute a so-called computer. In the case of this embodiment, the auxiliary storage device 14 is a rewritable non-volatile semiconductor memory. The auxiliary storage device 14 may be an internal storage or an external storage. In the case of FIG. 1, the auxiliary storage device 14 stores a text recognition model 14A, a treatment app 14B, and an image folder 14C.

[0017] The text recognition model 14A is a machine learning trained model that outputs a character string from an image captured by the camera 18. The text recognition model 14A includes a preprocessing unit that extracts a partial image including a character string from the image to be processed, and a character conversion unit that takes the partial image extracted by the preprocessing unit as an input and outputs a corresponding character string. The text recognition model 14A in this embodiment is downloaded from the app store when the treatment app 14B is downloaded. However, the text recognition model 14A may also be a part of the treatment app 14B.

[0018] In the case of this embodiment, assume an app prescribed for a patient with hypertension as the treatment app 14B. Hereinafter, the treatment app 14B prescribed for a patient with hypertension is also referred to as a "hypertension app". The hypertension app is used for recording home blood pressure and the like. Note that the home blood pressure and the like recorded in the hypertension app are utilized for medical treatment at a medical institution. In this embodiment, assume an automatic electronic sphygmomanometer as the sphygmomanometer used for measuring home blood pressure. The automatic electronic sphygmomanometer is provided with a liquid crystal display or the like for displaying the measured blood pressure value.

[0019] The treatment app 14B is provided with a function to realize automatic input of the blood pressure value displayed on the liquid crystal display of the sphygmomanometer or the like in cooperation with the text recognition model 14A. Specifically, the treatment app 14B gives an image including the liquid crystal display of the sphygmomanometer as a subject to the text recognition model 14A, and records the blood pressure value and the like notified from the text recognition model 14A in association with the measurement time. Details of the processing operations executed by the treatment app 14B will be described later. The image folder 14C is a folder where the image files captured by the camera 18 are stored. The image files are composed of the data of the image body and the attribute information such as the date and time of imaging. The attribute information includes, for example, Exif (= Exchangeable Image File Format) data.

[0020] The communication interface 15 is a communication module compliant with, for example, Ethernet (registered trademark), wireless LAN (= Local Area Network), and mobile communication systems (4G, 5G). The display 16 is, for example, a liquid crystal display or an organic EL (= Electro - Luminescence) display. The display 16 is an example of a display unit. The input reception device 17 is, for example, a capacitive touch sensor or a power button. A device with a capacitive touch sensor attached to the surface of the display 16 is called a touch panel. The camera 18 is, for example, a CMOS (= Complementary Metal Oxide Semiconductor) sensor.

[0021] <Relationship between measurement timing and recording timing> FIG. 2 is a diagram for explaining the relationship between the measurement timing and the recording timing assumed in the first embodiment. As described above, in the hypertension treatment guideline 2019, it is recommended to measure home blood pressure four times in total, twice in the morning and twice in the evening. Therefore, also in FIG. 2, home blood pressure is measured twice in the morning and twice in the evening. In the case of this embodiment, each time the user measures blood pressure with a sphygmomanometer, the user uses the camera 18 (see FIG. 1) to capture the measured value displayed on the display screen of the sphygmomanometer.

[0022] FIG. 3 is a diagram for explaining the imaging of the display screen 21 of the sphygmomanometer 20 by the user terminal 10. In the case of this embodiment, when imaging the display screen 21, the user launches a camera app. The camera app is an app different from the treatment app 14B (see FIG. 1). On the display 16 of the user terminal 10 shown in FIG. 3, an operation screen of the activated camera app is displayed. In the imaging area 16A, an image of the display screen 21 during imaging by the camera 18 (see FIG. 1) is displayed.

[0023] When the imaging button 16B is tapped, the image being displayed in the imaging area 16A at the time of tapping is saved as an image file in the image folder 14C (see FIG. 1). In the case of FIG. 3, the systolic blood pressure (hereinafter referred to as "maximum blood pressure") is 140 mmHg, the diastolic blood pressure (hereinafter referred to as "minimum blood pressure") is 88 mmHg, and the pulse is 86 beats per minute.

[0024] Return to the description of FIG. 2. The image of the display screen 21 (see FIG. 3) is captured at least once for each measurement session. Therefore, there may be multiple images of the display screen 21 for each measurement session. In the case of this embodiment, the user activates only the camera app in the two morning measurements. That is, the user does not activate the treatment app 14B (see FIG. 1). Therefore, in the morning time zone, the user only captures and records the images of the display screen 21 corresponding to each measurement session.

[0025] Even in the two evening measurements, the user activates the camera app and captures the image of the display screen 21 for each measurement session. So far, it is the same as in the morning measurement, but in the evening measurement, after capturing the image of the display screen 21, the user activates the treatment app 14B (see FIG. 1) and instructs the reading of the blood pressure value from the image. That is, the user who operates the treatment app 14B described in this embodiment records the measurement values of the home blood pressure in one go for four times.

[0026] As will be described later, if there is no need to correct the numerical value of the blood pressure value read from the image of the display screen 21, the operations required of the user are only the tapping of the imaging button 16B (see FIG. 3), the activation operation of the treatment app 14B, and the operation of instructing the reading of the blood pressure value. Incidentally, when the treatment app 14B is launched twice a day, in the morning and at night, and the maximum blood pressure, minimum blood pressure, and pulse are manually entered for each measurement, it is necessary to manually enter the numerical values in the input fields prepared for each measurement, resulting in a larger number of operations compared to the treatment app 14B proposed in this embodiment.

[0027] In the case of a blood pressure monitor capable of Bluetooth connection, the workload for recording home blood pressure is less than that of manual input. However, if a problem occurs in the connection, recording of blood pressure values and the like becomes impossible until the problem is resolved. In particular, since many of the users who are required to record home blood pressure are elderly, there is a possibility that they may give up recording home blood pressure because the problem cannot be resolved. On the other hand, the treatment app 14B provided in this embodiment may be launched once a day, enabling the recording of home blood pressure for four measurements at once.

[0028] <Processing Operation> Hereinafter, based on the relationship shown in FIG. 2, the processing operation executed by the user terminal 10 will be described. FIG. 4 is a diagram for explaining an example of the processing operation executed by the user terminal 10 (see FIG. 1) used in the first embodiment. The processing operation shown in FIG. 4 is realized through the execution of various apps by the processor 11 (see FIG. 1). Note that the symbol S in the figure represents a step.

[0029] First, as a preparation operation, the processor 11 captures images of the display screen 21 (see FIG. 3) of the blood pressure monitor 20 (see FIG. 3) twice in the morning (step 101). In addition, the processor 11 captures images of the display screen of the blood pressure monitor twice at night (step 102). Note that a camera app is used to capture the images of the display screen 21. When the imaging of the measurement values for a total of four times in the morning and at night is completed, the processor 11 launches the treatment app 14B based on the user's operation (step 103).

[0030] When the treatment app 14B is launched, the processor 11 displays the home screen (step 104). FIG. 5 is a diagram for explaining an example of the home screen 100 of the treatment application 14B assumed in the first embodiment. On the home screen 100 shown in FIG. 5, there are arranged a display column 101 for the average value of home blood pressure and an operation button 102 for instructing the reading of blood pressure values from an image obtained by imaging the display screen 21 (see FIG. 3). In the case of FIG. 5, the operation button 102 is labeled "Photo capture".

[0031] In the case of FIG. 5, the display column 101 is composed of a display section 101A for the average morning blood pressure and a display section 101B for the average night blood pressure. In the display section 101A for the average morning blood pressure, the average value of the maximum blood pressure and the average value of the minimum blood pressure read from a plurality of images captured within 10 minutes are displayed. In the display section 101B for the average night blood pressure, the average value of the maximum blood pressure and the average value of the minimum blood pressure read from a plurality of images captured within 10 minutes are displayed.

[0032] Note that when there are a plurality of images of the display screen 21 captured at the same measurement time, an image read from any one of the images may be used for calculating the average value. Also, when there is only one image captured in the morning, the maximum blood pressure and the minimum blood pressure read from the one image may be displayed in the display section 101A for the average morning blood pressure. The same applies to the display section 101B for the average night blood pressure. In the case of FIG. 5, since there is no recorded home blood pressure, the display sections 101A and 101B are blank.

[0033] Return to the description of FIG. 4. When the processor 11 receives an operation of the "Photo capture" button (step 105), it extracts an image including the display screen 21 (see FIG. 3) of the sphygmomanometer 20 (see FIG. 3) as a subject from the image folder 14C (see FIG. 1) (step 106). In the case of the present embodiment, for the purpose of reducing the number of images to be read, among the images stored in the image folder, only the images of the sphygmomanometer that have not yet been the target of reading and that were captured within 24 hours from the activation of the treatment application 14B are the targets of extraction.

[0034] However, since the recording of blood pressure values by the activation of the treatment application 14B only needs to be done once a day, there is a risk that the user's interest in the treatment application 14B will decrease. Therefore, in the present embodiment, by restricting the image extraction target to images captured within 24 hours from the activation of the treatment application 14B, the user is required to activate the treatment application 14B at least once a day. Here, 24 hours is an example of a predetermined time.

[0035] Whether or not the subject includes the display screen 21 of the sphygmomanometer 20 can be determined by known image recognition techniques. Known image recognition techniques include a method of comparing a feature pattern prepared in advance based on the display screen 21 of the sphygmomanometer 20 with the image to be determined. In addition, known image recognition techniques also include a method of using a learned model obtained by machine learning of an image capturing the display screen 21 of the sphygmomanometer 20. In the case of the method using the learned model, the processor 11 provides the image stored in the image folder 14C (see FIG. 1) to the learned model and obtains an output as to whether the image to be processed includes the display screen 21 of the sphygmomanometer 20 as a subject. Also, whether or not it has been a reading target can be determined, for example, by the presence or absence of a flag added to the image that has been a reading target. Also, whether or not it was captured within 24 hours from the activation of the treatment application 14B can be determined by reading the imaging date and time recorded in the Exif data which is the attribute information of the image.

[0036] Next, the processor 11 reads the maximum blood pressure, minimum blood pressure, and pulse from the extracted image (step 107). Incidentally, the processor 11 provides each extracted image to the text recognition model 14A (see FIG. 1) and reads the maximum blood pressure, minimum blood pressure, and pulse. In addition, the processor 11 obtains the imaging date and time from the Exif data of each image. Subsequently, the processor 11 displays the read values and the image from which they were read side by side (step 108).

[0037] FIG. 6 is a diagram for explaining an example of a confirmation screen 110 for reading values displayed when the operation button 102 (see FIG. 5) is operated. In FIG. 6, the confirmation screen 110 is provided with a title of "Blood Pressure Photo Capture". On the confirmation screen 110 shown in FIG. 6, an explanatory text 111 describing the operation required of the user, a reading information column 112 corresponding to each measurement time, and a save button 113 are arranged.

[0038] In the case of FIG. 6, the explanatory text 111 displays "Please confirm that the value is correct and press Save. If it is different from the photo, you can change the value with the 'Edit' button." In the case of FIG. 6, four reading information columns 112 are displayed. Incidentally, the four correspond to each of the two measurements in the morning and the two measurements in the evening. However, the number of the reading information columns 112 corresponds to the number of images extracted in step 106 (see FIG. 4).

[0039] Therefore, if the number of images extracted in step 106 is, for example, two, the number of the reading information columns 112 displayed on the confirmation screen 110 is also two. On the other hand, if the number of images extracted in step 106 is five or more, the number of the reading information columns 112 displayed on the confirmation screen 110 is also five or more. However, the number of the reading information columns 112 displayed on the confirmation screen 110 at one time may be limited to four. In this case, the reading information columns 112 displayed on the confirmation screen 110 can be changed with a scroll bar.

[0040] The reading information column 112 shown in FIG. 6 is composed of a reduced image 112A of the image extracted as the reading target, a capture date and time 112B, a reading value 112C, an "Edit" button 112D, and a "Delete" button 112E. In the reduced image 112A, the image from which the reading was taken is reduced and displayed. The display size is preferably such that the numerical value can be read. However, it is also possible to enable the enlarged display of the image by pinching out, or to enable the display of a separate window with a large display size by double-tapping or the like.

[0041] For the imaging date and time 112B, the date and time when the image of the reading source was captured is displayed, for example, in month (MM) / day (DD) hour (hh):minute (mm). For the read value 112C, the maximum blood pressure (SSS mmHg), minimum blood pressure (DDD mmHg), and pulse rate (PP bpm) read from the image of the reading source are displayed. In the case of this embodiment, since the read value 112C is displayed side by side with the reduced image 112A, the user can check on the same screen whether the read numerical values are correct or not.

[0042] The "Edit" button 112D is a button for correcting numerical values when there is an error in the numerical values displayed in the read value 112C. When the "Edit" button 112D is operated, a separate window is displayed, and the maximum blood pressure, minimum blood pressure, and pulse rate can be corrected individually. The layout of the separate window and the like will be described later. The "Delete" button 112E is a button used to delete the reading information column 112. For example, it is used to delete unclear images, images outside the target, and extra images corresponding to the same measurement session. The save button 113 shown in FIG. 6 is labeled "Save". When the save button 113 is operated, the numerical values of all the read values 112C displayed on the confirmation screen 110 are finalized.

[0043] Return to the description of FIG. 4. In a state where the confirmation screen 110 (see FIG. 6) is displayed, the processor 11 determines whether an operation of the save button 113 has been detected (step 109). If an operation of the save button 113 is detected, an affirmative result is obtained in step 109. In this case, the processor 11 saves the read values in association with the measurement session (step 110).

[0044] After that, the processor 11 calculates the average value of the read values obtained from the images whose imaging times are within 10 minutes and displays it on the home screen 100 (see FIG. 7) (step 111). As described above, when there is only one measurement session in one or both of the morning and evening, the corresponding measurement value is displayed on the home screen 100. FIG. 7 is a diagram showing a display example of the home screen 100 displayed when the save button is operated on the confirmation screen 110 (see FIG. 6). In FIG. 7, the corresponding parts to those in FIG. 5 are denoted by the same reference numerals. In the case of FIG. 7, in the display section 101A of the morning average blood pressure, “145 mmHg / 100 mmHg” is displayed, and in the display section 101B of the night average blood pressure, “135 mmHg / 94 mmHg” is displayed.

[0045] Return to the description of FIG. 4. Next, the case where a negative result is obtained in step 109 will be described. When a negative result is obtained in step 109, it is the case where the “Edit” button 112D (see FIG. 6) or the “Delete” button 112E (see FIG. 6) is operated in any of the reading information fields 112 (see FIG. 6). In the case of FIG. 4, the processor 11 determines whether the “Edit” button 112D (see FIG. 6) has been operated (step 112). If the operation is of the “Edit” button 112D, an affirmative result is obtained in step 112.

[0046] For example, when the user determines that it is necessary to confirm the numerical value read because the image quality of the reduced image 112A (see FIG. 6) is poor, the “Edit” button 112D is operated. In this case, the processor 11 displays the edit screen 120 (see FIG. 8) (step 113). FIG. 8 is a diagram for explaining an example of the edit screen 120 displayed when the “Edit” button 112D in any of the reading information fields 112 displayed on the confirmation screen 110 (see FIG. 6) is operated.

[0047] On the edit screen 120 shown in FIG. 8, an imaging date and time field 121, an edit reception field 122, a reading source image field 123, a save button 124, and a “Cancel” button 125 are arranged. In the edit reception field 122, as an initial value, the numerical value of the reading value 112C on the confirmation screen 110 (see FIG. 6) is displayed. In the case of FIG. 8, in the edit reception field 122, “Systolic blood pressure 140 mmHg”, “Diastolic blood pressure 83 mmHg”, and “Pulse 86 bpm” are displayed.

[0048] Incidentally, the display size of the original image field 123 is larger than that of the reduced image 112A (see FIG. 6). Therefore, the user can more accurately confirm the content of the original image. In the case of FIG. 8, in the image of the display screen 21 (see FIG. 3) displayed in the original image field 123, the illumination is reflected in the position of the numerical value indicating the minimum blood pressure. For this reason, it is difficult to identify the numerical value, but it is possible for a person to read the minimum blood pressure as "88 mmHg". In addition, when the enlarged screen can be displayed by tapping the original image field 123, the numerical value can be read with a more enlarged image. In many cases, the numerical value can be read from the editing screen 120 or the enlarged screen.

[0049] In this case, since the reading value of the minimum blood pressure is incorrect, the user manually edits the numerical value of the minimum blood pressure in the edit reception field 122 from "83" to "88". In addition, for images in which the accuracy of character recognition is likely to decrease, in addition to the reflection of ambient light, there are also out-of-focus images and blurred images due to hand movement. Incidentally, when it is found that the reading value is correct by checking the image displayed in the original image field 123, the user operates the save button 124 or the "cancel" button 125 without editing the numerical value in the edit reception field 122. Incidentally, in the case of FIG. 8, the save button 124 is labeled "Save".

[0050] Return to the description of FIG. 4. When the save button 124 (see FIG. 8) is operated after editing the numerical value in the edit reception field 122 (see FIG. 8), or when the operation of the "cancel" button 125 (see FIG. 8) is received, the processor 11 saves the edited numerical value or cancels the edited numerical value (step 114). After that, the processor 11 returns to the determination in step 109. That is, when the processor 11 receives the operation of the save button 124 or the "cancel" button 125, it switches the display on the display 16 (see FIG. 1) from the edit screen 120 (see FIG. 8) to the confirmation screen 110 (see FIG. 6).

[0051] Next, the case where a negative result is obtained in step 112 will be described. When a negative result is obtained in step 112, it is the case where the "Delete" button 112E (see FIG. 6) corresponding to any one of the reading information columns 112 is operated. In this case, for example, an image unrelated to the sphygmomanometer 20 (see FIG. 3) may be confirmed from the reduced image 112A (see FIG. 6). In this case, the processor 11 deletes the corresponding reading information column 112 and returns to step 109 (step 115).

[0052] FIG. 9 is a diagram for explaining a display example of a confirmation screen 110 for explaining a state in which the corresponding reading information column 112 is deleted by the operation of the "Delete" button 112E. In FIG. 9, reference numerals corresponding to the corresponding parts in FIG. 6 are attached and shown. In the confirmation screen 110 shown in FIG. 6, four reading information columns 112 are displayed, but the number of reading information columns 112 in the confirmation screen 110 shown in FIG. 9 is three. However, when the number of reading information columns 112 before deletion is five or more, the apparent number of displayed items may not change.

[0053] <Parentheses> As described above, by using the treatment application 14B (see FIG. 1) described in the present embodiment, it is possible to collectively record the measurement results of the blood pressure values four times in the morning and at night, and the burden on the user when recording the home blood pressure can be reduced. In addition, since the images stored in the image folder are used as the reading source, it is possible to perform a time shift between the measurement timing and the recording timing of the home blood pressure. As a result, an effect of reducing the psychological burden on the user who records the home blood pressure can also be realized. For example, even a user who feels bothered to record the home blood pressure during a busy time before going out can record the home blood pressure in the treatment application slowly after returning home.

[0054] In addition, since the imaging of the image from which the numerical value is read is the same as that of taking a normal photograph, it is easy for elderly people to understand and there is no need to worry about problems such as Bluetooth connection failures. Therefore, the risk of interruption of home blood pressure records due to Bluetooth connection failures can be reduced. In addition, in the case of the treatment application 14B described in the present embodiment, the image from which the numerical value is read and the read value can be confirmed on the same screen. Therefore, even when the quality of the image from which the numerical value is read is poor, it is possible to notice an error in the read value. Further, in the case of the treatment application 14B in the present embodiment, when an error in the read value is noticed, it can be corrected directly on the screen, so the number of operations required of the user can be reduced compared to the case of retaking the image from which the value is read or measuring the home blood pressure again.

[0055] <Embodiment 2> Hereinafter, Embodiment 2 of the present disclosure will be described with reference to the drawings. <Device Configuration> Also in Embodiment 2, the same user terminal 10 (see FIG. 1) as in Embodiment 1 is assumed. However, a part of the functions executed by the treatment application 14B (see FIG. 1) is changed.

[0056] <Relationship between Measurement Timing and Recording Timing> FIG. 10 is a diagram for explaining the relationship between the measurement timing and the recording timing assumed in Embodiment 2. In the case of Embodiment 2, it is different from Embodiment 1 in that every time a measurement value of home blood pressure is imaged, the blood pressure value is read from the corresponding image.

[0057] This difference is due to the difference in the usage form of the treatment application 14B (see FIG. 1) by the user. That is, in Embodiment 2, it is assumed that the user always starts the treatment application 14B or the user who has little resistance to starting the treatment application 14B when measuring home blood pressure. Hereinafter, based on the timing relationship shown in FIG. 10, the processing operations executed by the user terminal 10 will be described.

[0058] <Processing operation> FIG. 11 is a diagram for explaining an example of a processing operation executed by the user terminal 10 (see FIG. 1) used in the second embodiment. In FIG. 11, reference numerals corresponding to the corresponding parts in FIG. 4 are shown. The processing operation shown in FIG. 11 is also realized through the execution of various applications by the processor 11 (see FIG. 1). The processing operation shown in FIG. 11 is started by the activation of the treatment application 14B (step 103). When the treatment application 14B is activated, the processor 11 displays the home screen (step 104).

[0059] FIG. 12 is a diagram for explaining an example of the home screen 130 of the treatment application 14B assumed in the second embodiment. In FIG. 12, reference numerals corresponding to the corresponding parts in FIG. 5 are shown. On the home screen 130 shown in FIG. 12, a display column 101 for the average value of home blood pressure and an operation button 131 for instructing the display of the measurement screen 140 (see FIG. 13) are arranged. In the case of FIG. 12, the operation button 131 is labeled "Measure". Also, since there is no recorded home blood pressure, the display parts 101A and 101B shown in FIG. 12 are blank.

[0060] Returning to the description of FIG. 11. In the case of this embodiment, when the processor 11 receives the operation of the operation button 131 (see FIG. 12), it displays the measurement screen 140 (see FIG. 13) (step 121). FIG. 13 is a diagram for explaining an example of the measurement screen 140. In the case of FIG. 13, the measurement screen 140 is labeled with the title "Blood Pressure Measurement". On the measurement screen 140 shown in FIG. 13, a selection column 141 for the measurement time, a display column 142 for the measurement date and time, a reading information column 143, a "Camera" button 144, and a save button 145 are arranged.

[0061] In the case of the selection column 141 for the measurement time in FIG. 13, "First time" is highlighted and "Second time" is grayed out. Therefore, the current display is the first measurement screen. In the reading information column 143, home blood pressure and the like read from an image obtained by imaging the display screen 21 (see FIG. 3) are displayed. For example, the maximum blood pressure value, the minimum blood pressure value, and the pulse value are displayed. Note that when the display screen 21 has not been imaged, no numerical value is displayed in the reading information column 143.

[0062] The "Camera" button 144 is a button for instructing the start of imaging of the display screen 21. When the "Camera" button 144 is operated, the display on the display 16 (see FIG. 1) switches to the camera output screen 150 (see FIG. 14). The save button 145 is a button for recording the blood pressure value and the like displayed in the reading information column 143 in association with the measurement times. In the case of FIG. 13, the save button 145 is labeled "Save".

[0063] Return to the description of FIG. 11. When the processor 11 receives an operation of the "Camera" button 144 (see FIG. 13) (step 122), the processor 11 displays the camera output screen 150 (see FIG. 14) (step 123). FIG. 14 is a diagram for explaining an example of the camera output screen 150 and the read value confirmation screen 160. The camera output screen 150 shown in FIG. 14 is provided with a title of "Blood pressure monitor screen reading".

[0064] On the camera output screen 150 shown in FIG. 14, an explanatory text 151 describing the operation required of the user, a live image display column 152, an imaging button 153, and a "Cancel" button 154 are arranged. In the case of FIG. 14, the explanatory text 151 displays "Please photograph the blood pressure monitor on which the measurement result is displayed and press the imaging button." The user adjusts the orientation, distance, etc. of the camera 18 while checking the display in the live image display column 152. Note that the image displayed in the live image display column 152 is temporarily stored in the RAM 13 (see FIG. 1) as cache data. The imaging button 153 shown in FIG. 14 is labeled "Take a picture". When the "Cancel" button 154 is operated, the display on the display 16 returns to the measurement screen 140 (see FIG. 13).

[0065] Return to the description of FIG. 11. When the processor 11 receives an operation of the imaging button 153 (see FIG. 14) (step 124), it reads the maximum blood pressure, minimum blood pressure, and pulse from the captured image (step 125). Incidentally, the processor 11 gives the captured image to the text recognition model 14A (see FIG. 1) and reads the maximum blood pressure, minimum blood pressure, and pulse. Subsequently, the processor 11 displays a reading value confirmation screen 160 (see FIG. 14) (step 126). On the reading value confirmation screen 160, the reading value and the image from which the value was read are displayed side by side.

[0066] On the reading value confirmation screen 160 shown in FIG. 14, there are arranged an explanatory text 161 describing the operation required of the user, a reading source image field 162, a reading value 163, a retake button 164, a determination button 165, and a "Cancel" button 166. In the case of FIG. 14, the explanatory text 161 shows "It will be reflected when you tap 'Determine'." This description indicates that if the reading value is correct, the determination button 165 should be tapped. In the case of FIG. 14, the determination button 165 is labeled "Determine".

[0067] In the reading source image field 162 shown in FIG. 14, the image captured when the imaging button 153 was tapped is displayed. In the case of FIG. 14, in the reading source image field 162, the influence of lighting is strongly shown at the display position of the minimum blood pressure. The image displayed in the reading source image field 162 is cache data. Therefore, the image displayed in the reading source image field 162 is read from the RAM 13. In the case of FIG. 14, the minimum blood pressure is displayed as "80 mmHg" in the reading value 163. However, the correct value of the minimum blood pressure confirmed from the reading source image field 162 is "88 mmHg".

[0068] The retake button 164 shown in FIG. 14 is literally a button used to instruct the retaking of the image serving as the reading source. When the retake button 164 is operated, the display on the display 16 switches to the camera output screen 150. In the case of FIG. 14, the retake button 164 is labeled "Retake". This retake button 164 is an example of a button used to instruct the correction of the measured value.

[0069] The determination button 165 is a button for temporarily storing the read value in association with the measurement session. Therefore, at the stage when the determination button 165 is operated, the blood pressure value for the measurement session has not been finalized. In other words, the determination button 165 is a button for switching (or updating) the display on the measurement screen 140 (see FIG. 13). Note that in the reading information field 143 (see FIG. 13), the value displayed in the read value 163 at the time when the determination button 165 is operated is displayed. The "Cancel" button 166 is a button for returning to the measurement screen 140 without saving the read value.

[0070] Return to the description of FIG. 11. The processor 11 determines whether the determination button 165 (see FIG. 14) has been operated while the read value confirmation screen 160 (see FIG. 14) is being displayed (step 127). If it is an operation of the retake button 164 (see FIG. 14) or an operation of the "Cancel" button 166 (see FIG. 14), a negative result is obtained in step 127. In this case, the processor 11 further determines whether the retake button 164 has been operated (step 128). If it is an operation of the retake button 164, an affirmative result is obtained in step 128. In this case, the processor 11 returns to step 125. That is, it returns to the display of the camera output screen 150.

[0071] On the other hand, if it is an operation of the "Cancel" button 166, a negative result is obtained in step 128. In this case, the processor 11 returns to step 122. That is, it returns to the display of the measurement screen 140 (see FIG. 13). Incidentally, when an affirmative result is obtained in step 127 (i.e., when the decision button 165 is operated), the processor 11 associates the read value with the measurement time and displays it in the read information field 143 (see FIG. 13) (step 129).

[0072] Next, the processor 11 determines whether the second measurement has ended (step 130). If the current read value corresponds to the first measurement time, a negative result is obtained in step 130. In this case, the processor 11 returns to step 122. On the other hand, if the current read value corresponds to the second measurement time, an affirmative result is obtained in step 130. In this case, the processor 11 accepts the operation of the save button 145 (see FIG. 13) (step 131).

[0073] Thereafter, the processor 11 calculates the average value of the read value corresponding to the first measurement and the read value corresponding to the second measurement and displays it on the home screen 130 (see FIG. 12) (step 132). Note that the calculation of the average value is limited to the case where the time when the read value corresponding to the first measurement is saved and the time when the read value corresponding to the second measurement is saved are within 10 minutes.

[0074] FIG. 15 is an example of the display on the home screen 130 when the recording of the morning home blood pressure is completed. In FIG. 15, the reference numerals corresponding to the corresponding parts in FIG. 12 are shown. In the case of FIG. 15, in the display section 101A for the average morning blood pressure, "145 mmHg / 100 mmHg" is displayed, and the display section 101B for the average night blood pressure remains blank.

[0075] <Parentheses> As described above, when using the treatment application 14B (see FIG. 1) described in this embodiment, the measured value can be read and recorded simply by imaging the display screen 21 of the sphygmomanometer 20 for each measurement. Note that before associating the blood pressure value read from the image with the measurement time and recording it, since the image used as the reading source and the read value can be compared on the same screen, it is easy to notice if the read value is incorrect. In addition, in this embodiment, since the correctness of the read value can be confirmed while the home blood pressure for each measurement session is being displayed, if the read value is incorrect, the display screen 21 can be immediately redone, and the correct blood pressure value can be recorded.

[0076] <Embodiment 3> Hereinafter, Embodiment 3 of the present disclosure will be described with reference to the drawings. <System Configuration> FIG. 16 is a diagram for explaining a configuration example of an information processing system assumed in Embodiment 3. In FIG. 16, the corresponding parts to those in FIG. 1 are shown with corresponding reference numerals. The information processing system shown in FIG. 16 is composed of a user terminal 10 and an analysis server 30. In this embodiment, it is different from Embodiment 1 in that the analysis server 30 executes the process of reading the blood pressure value from the image captured of the display screen 21 (see FIG. 3), and returns the read value, which is the analysis result, to the user terminal 10.

[0077] Note that the user terminal 10 and the analysis server 30 are communicably connected via a network N. The network N is, for example, a LAN (= Local area network), the Internet, or a mobile communication system (4G, 5G). In the case of FIG. 16, only one user terminal 10 is connected to the network N, but a plurality of user terminals 10 may be connected. Also, regarding the analysis server 30, a plurality of analysis servers 30 may cooperate to execute image analysis and notify the user terminal 10 of the read value as the analysis result.

[0078] Therefore, the text recognition model 14A is not stored in the user terminal 10 used in this embodiment. The analysis server 30 includes a processor 31, a ROM 32 in which the BIOS and the like are stored, a RAM 33 used as a work area of the processor 31, an auxiliary storage device 34, and a communication interface 35. Each device is connected through signal lines (not shown). The auxiliary storage device 34 shown in FIG. 16 stores a text recognition model 34A or a natural language analysis model 34B.

[0079] As the text recognition model 34A used in this embodiment, it is possible to adopt a model with higher analysis accuracy than the text recognition model 14A (see FIG. 1) described in the first embodiment. This is because the amount of calculation per unit time of the analysis server 30 is much larger than the amount of calculation per unit time of the user terminal 10. Therefore, as the text recognition model 34A, a model that requires a large amount of calculation but has high analysis accuracy can be used.

[0080] The natural language analysis model 34B is a model that learns not only character patterns but also the relationships between characters, and has higher analysis accuracy than the text recognition model 34A. However, it requires an even larger amount of calculation than the text recognition model 34A. The auxiliary storage device 34 in FIG. 16 depicts the text recognition model 34A and the natural language analysis model 34B, but actually, only one of them is stored.

[0081] <Processing sequence> FIG. 17 is a diagram for explaining an example of a processing sequence executed by the user terminal 10 used in the third embodiment. In FIG. 17, the corresponding parts to those in FIG. 4 are denoted by the same reference numerals. The processing sequence shown in FIG. 17 is one in which the analysis processing in the processing operation described in the first embodiment is shared by the analysis server 30. Therefore, the processing operations up to steps 101 to 106 are the same as those in the first embodiment. That is, when the user finishes measuring the home blood pressure for the second time at night and captures the display screen 21, the treatment application 14B (see FIG. 1) is launched to instruct the reading of the home blood pressure.

[0082] The processor 11 (see Fig. 1) that detected this operation extracts an image including the display screen 21 of the sphygmomanometer 20 from the image folder (step 106), and uploads the extracted image to the analysis server 30 (step 141). In the case of this embodiment, at least four images of the display screen 21 for morning and evening are uploaded. The analysis server 30 that has become the upload destination reads the maximum blood pressure, minimum blood pressure, and pulse from each of the uploaded images (step 142).

[0083] As described above, the analysis server 30 provides the uploaded image to the text recognition model 34A (see Fig. 16) or the natural language analysis model 34B (see Fig. 16), and reads blood pressure values and the like from within the image. When the reading of blood pressure values and the like is completed, the analysis server 30 notifies the user terminal 10 of the read numerical values (step 143). After this, the user terminal 10 executes steps 108 to 115.

[0084] <Parentheses> As described above, the treatment application 14B (see Fig. 1) described in this embodiment uploads the image to be analyzed to the analysis server 30 and acquires blood pressure values and the like as analysis results. Since the analysis server 30 can utilize more computing resources than the user terminal 10, the reading accuracy of blood pressure values and the like can be made higher than when analyzing the image within the user terminal 10. As a result, the number of times the user corrects the numerical values can also be reduced. Another advantage obtained by analyzing the image with the analysis server 30 is that the types of display screens 21 that can be read may increase.

[0085] For example, not only the layout of the maximum blood pressure, minimum blood pressure, and pulse on the display screen 21 of the sphygmomanometer 20, but also the types and sizes of fonts vary. Also, the measurement date and time may be displayed on the display screen 21. However, by analyzing the image with the analysis server 30 as in this embodiment, it becomes possible to more accurately extract blood pressure values and the like from within the display screen 21. In addition, in this embodiment, the processing sequence has been described on the premise of the processing operations of Embodiment 1, but it may also be based on the processing operations of Embodiment 2.

[0086] <Embodiment 4> In the aforementioned Embodiment 1, all the images extracted by the treatment application 14B based on a predetermined condition are set as analysis targets, but the user may be allowed to pre-select the images to be analyzed. FIG. 18 is a diagram for explaining an example of a selection screen 170 for analysis targets assumed in Embodiment 4.

[0087] The selection screen 170 shown in FIG. 18 is displayed before the confirmation screen 110 (see FIG. 6) is displayed when the operation button 102 is operated on the home screen 100 (see FIG. 5) (that is, when the "photo capture" button is operated). The selection screen 170 shown in FIG. 18 is labeled with the title "Selection of Images to be Analyzed". On the selection screen 170 shown in FIG. 18, an explanatory text 171 describing the operation required of the user, a candidate selection column 172 corresponding to the extracted images, a scroll bar 173, and a "decision" button 174 are arranged.

[0088] In the case of FIG. 18, the explanatory text 171 displays "Please select the image to be analyzed and press the decision button." The candidate selection column 172 shown in FIG. 18 is composed of a reduced image 172A of the extracted image and a corresponding check box 172B. When the scroll bar 173 is operated, the content of the reduced image 172A displayed within the screen changes.

[0089] As also described in the explanatory text 171, the "decision" button 174 is used to determine the analysis target. After the "decision" button 174 is operated, the processor 11 provides the images checked in the check box 172B to the text recognition model 14A (see FIG. 1) and reads out blood pressure values and the like included in the images. Note that after the analysis is completed, the processor 11 displays the confirmation screen 110 (see FIG. 6) on the display 16 (see FIG. 1).

[0090] <Small brackets> In the case of this embodiment, since the user can specify the image to be analyzed, the computing resources used for the analysis can be minimized. Also, even when additional costs are incurred for the use of the analysis server 30 (see FIG. 16) described in Embodiment 3, the cost required for the analysis can be minimized by narrowing down the number of images to be analyzed.

[0091] <Other embodiments> (1) As described above, the embodiments of the present disclosure have been described. However, the technical scope of the present disclosure is not limited to the scope described in the foregoing embodiments. It is obvious from the description of the claims that those obtained by making various changes or improvements to the foregoing embodiments are also included in the technical scope of the present disclosure.

[0092] (2) The processor in the foregoing embodiments refers to a processor in a broad sense, and includes, in addition to a general-purpose processor (for example, a CPU (= Central Processing Unit)), a dedicated processor (for example, a GPU (= Graphical Processing Unit), an ASIC (= Application Specific Integrated Circuit), an FPGA (= Field Programmable Gate Array), a programmable logic device, etc.). Also, the operations of the processor in each of the foregoing embodiments are not limited to a single processor, and a plurality of processors may cooperate to execute them. Also, the order of execution of each operation in the processor is not limited to the order described above, and may be individually changed.

[0093] (3) In the foregoing embodiments, a smartphone was exemplified as the user terminal 10 (see FIG. 1), but other terminals may be used as long as they are terminals on which treatment applications or health management applications are executed. Other terminals may include, for example, desktop computers, notebook computers, tablet computers, and wearable computers. Wearable computers may include, for example, smart glasses, headsets, and smartwatches.

[0094] (4) In the foregoing Embodiment 1, a 24-hour limit is set for the images to be read for blood pressure values, but the period for setting the limit may be extended or the limit itself may be eliminated. For example, images captured within 48 hours may be made targets for reading blood pressure values, or images captured within one week may be made targets for reading blood pressure values. In the case of the treatment application 14B of Embodiment 1, since the image reading may be performed only at night or only in the morning, there is a possibility that a measurement session in which 24 hours have already elapsed may appear when noticed.

[0095] For example, when reading the measurement values of the same day in the morning and at night at night, even if trying to read the previous day's data the next morning, if the time when the treatment application 14B is started has passed 24 hours since the time when the blood pressure value of the previous morning was imaged, the blood pressure value measured the previous morning cannot be automatically imported. Therefore, the limit for the imaging time of the images to be read for blood pressure values may be set to 24 hours or more. However, if the image of the display screen 21 captured the previous morning remains, it is possible to manually input the blood pressure value by referring to the image.

[0096] (5) In the aforementioned Embodiment 1, the images captured within 24 hours from the activation of the treatment application 14B are extracted as the objects for reading blood pressure values. However, images within a predetermined time (for example, 24 hours) from the imaging time to the current time may also be targeted. In this example, the predetermined time may also be 24 hours or more (for example, 48 hours, one week). In other words, images captured within a predetermined time from the operation time (current time) of the "Photo Capture" button may be used as the objects to be read.

[0097] (6) In the aforementioned Embodiment 1, in Step 106 (see FIG. 4), unread images of the sphygmomanometer are extracted. However, only unread images may be the extraction targets, only images of the sphygmomanometer may be the extraction targets, or these restrictions may not be applied.

[0098] (7) In the aforementioned Embodiments 1 and 3, the case where the home blood pressure is measured four times in total in the morning and evening and the operation button 102 (see FIG. 5) is operated after the images corresponding to each measurement are complete has been described. However, the acceptance of the operation on the operation button 102 is independent of the presence or number of images corresponding to each measurement. Therefore, as in the case of Embodiment 2, the operation button 102 may be operated each time a measurement is completed, and the blood pressure value may be read and recorded from the images corresponding to each measurement.

[0099] (8) In the aforementioned Embodiment 2, when there is an error in the read value, the processing operation of instructing to retake the display screen 21 (see FIG. 3) until the correct blood pressure value is read has been described. However, as in Embodiment 1, the read value may be corrected with reference to the displayed image.

[0100] (9) In the aforementioned embodiments, it is assumed that the sphygmomanometer 20 has a display screen 21 and the blood pressure value and other measurement results are digitally displayed on the display screen 21. However, the measurement results may also be analogically displayed. FIG. 19 is a diagram for explaining an example of a mercury column sphygmomanometer 22 that reads the blood pressure value from the height of mercury. In FIG. 19, the reference numerals corresponding to the corresponding parts in FIG. 3 are shown.

[0101] In the case of the mercury column type sphygmomanometer 22, the height of the mercury column becomes the blood pressure of the user. When the mercury column type sphygmomanometer 22 is imaged by the user terminal 10, the processor 11 (see FIG. 1) specifies the height of the tip of the mercury column from the scale 23 engraved along the glass tube and reads it as a blood pressure value. Note that, for a sphygmomanometer in which the blood pressure value is displayed in analog, it may be an aneroid type in which the needle moves along an arc. In the case of an image of an aneroid type sphygmomanometer, the processor 11 uses the numerical value of the tip position of the needle read from the scale as the blood pressure value.

[0102] (10) In the foregoing embodiment, the sphygmomanometer 20 is assumed as an example of a measuring instrument for measuring biological information. However, the technology of the foregoing embodiment is also applicable to reading a measured value from an image of a measuring instrument other than a sphygmomanometer. FIG. 20 is a diagram for explaining the case where the measuring instrument is an analog type weighing scale 24. In FIG. 20, the same reference numerals as those corresponding to the corresponding parts in FIG. 3 are shown. The user's weight will be read from an image of the dial 25 of the analog type weighing scale. Specifically, the user's weight is read at the position of the needle with respect to the scale printed in an arc shape.

[0103] Note that, in FIG. 20, an analog type weighing scale is illustrated, but a digital type weighing scale may also be used. In addition, the measuring instrument may be, for example, a thermometer, a urine glucose meter for self-examination, a blood glucose self-measuring device, a sodium potassium meter, a pulse oximeter, a body fat meter, a body composition meter, an activity meter, a pedometer, or a heart rate meter. Any of them may be not limited to a digital type, but may also be an analog type.

[0104] (11) In the foregoing embodiment, the case where a function of reading a measured value from an image of a measuring instrument is provided in a treatment application approved by the Pharmaceutical Affairs Law has been described. However, a similar function may be provided in a health management application.

[0105] (12) In the foregoing embodiment, the case where an image captured within a predetermined period (for example, within 24 hours from the start of the treatment application) is displayed on the selection screen 170 (see FIG. 18) has been described. In this case, the candidate selection column 172 (see FIG. 18) may be sorted and displayed by the imaging date or by time zone of the imaging date (e.g., "morning", "night"). By adopting this display function, the user can efficiently record blood pressure values for each specific time zone.

[0106] (13) In the above-described embodiment, there is no upper limit on the number of images that can be selected as the object of image analysis from the candidate selection column 172 (see FIG. 18) displayed on the selection screen 170 (see FIG. 18) among the images captured within a predetermined period (e.g., within 24 hours from the start of the treatment application). However, image analysis places a large processing load on the processor 101 (see FIG. 1) and also causes a long waiting time until the analysis result is displayed. Therefore, an upper limit may be set on the number of images that can be selected at one time. For example, the upper limit may be four.

[0107] (14) In the above-described embodiment, the case where an image captured within a predetermined period (e.g., within 24 hours from the start of the treatment application) is displayed on the selection screen 170 (see FIG. 18) has been described. In the case of the above-described embodiment, since the "photo capture" button can be operated any number of times, an image that was selected as the object of reading the measurement value in a previous operation is also displayed as a selection candidate. However, if an image for which the reading of the blood pressure value or the like has failed is always displayed as a reading candidate, the user will perform a useless selection operation many times. Also, repeatedly analyzing an image for which the reading has failed is a waste of computing resources such as the processor 11 (see FIG. 1). Further, there is no need to read an image for which the blood pressure value or the like has been successfully read in the past many times.

[0108] FIG. 21 is a diagram for explaining another display example of the selection screen 170 used for selecting an image to be analyzed. In FIG. 21, the reference numerals corresponding to the corresponding parts in FIG. 18 are shown. In the case of Fig. 21, the three reduced images 172A are labeled "Reduced Image A", "Reduced Image B", and "Reduced Image C" in the order from top to bottom in the figure. However, the reduced image 172A corresponding to "Reduced Image A" is shown in grayscale. Also, a label 172C reading "Acquired" is pop-up displayed on the front side of the reduced image 172A. Incidentally, "Acquired" is just an example, and other displays such as "Analyzed", "Read", and "Unselectable" are also possible. In addition, the checkbox 172B corresponding to "Reduced Image A" is also shown in grayscale. Here, "Reduced Image A" is an example of the first candidate. With these displays, the user can notice that "Reduced Image A" is an image captured within a predetermined period but has already been selected as an analysis target.

[0109] Fig. 22 is a diagram for explaining another display example of the selection screen 170 used for selecting an image to be analyzed. In Fig. 22, the corresponding parts to those in Fig. 18 are shown with corresponding reference numerals. Also in the case of Fig. 22, the three reduced images 172A are labeled "Reduced Image A", "Reduced Image B", and "Reduced Image C" in the order from top to bottom in the figure. In the case of Fig. 22, the reduced image 172A corresponding to "Reduced Image A" is shown in grayscale, and the corresponding checkbox 172B is not displayed.

[0110] By the reduced image 172A being shown in grayscale, the user can notice that "Reduced Image A" is an image captured within a predetermined period but has already been selected as an analysis target. Note that since the label 172C (see Fig. 21) is not displayed, the user can confirm the content of "Reduced Image A" although the visibility is reduced due to the grayscale display. Therefore, it can be a reference when the user selects "Reduced Image B" or "Reduced Image C", etc. Also, in the case of the selection screen 170 shown in Fig. 22, since the checkbox 172B is not displayed, it is possible to prevent "Reduced Image A" from being physically selected.

[0111] FIG. 23 is a diagram for explaining another display example of the selection screen 170 used for selecting an image to be analyzed. In FIG. 23, the corresponding parts to those in FIG. 18 are denoted by the same reference numerals. In the case of FIG. 23, two reduced images 172A are displayed on the selection screen 170. Specifically, in the order from top to bottom in the figure, "reduced image B" and "reduced image C" are displayed. Incidentally, also in the case of FIG. 23, a "reduced image A" (not shown) is included in the images captured within a predetermined period. However, the "reduced image A" (not shown) is an image that has been selected as an object for reading measurement values in a previous operation.

[0112] Therefore, on the selection screen 170 shown in FIG. 23, the "reduced image A" that cannot be a selection candidate is excluded from the display targets. That is, on the selection screen 170 shown in FIG. 23, only the candidates for selectable images are displayed. By using the selection screen 170 shown in FIG. 23, the user is physically unable to select the image that has been selected as an object for reading measurement values in a previous operation. Note that on the selection screen 170 shown in FIG. 23, in addition to the "determine" button 174 (see FIG. 18) used for determining the analysis target, a "capture now" button 175 is also displayed. By adopting the "capture now" button 175, it is possible to select the capture of the liquid crystal screen of the blood pressure monitor even on the selection screen 170 for the image to be analyzed. This "capture now" button 175 can also be adopted on the selection screens 170 shown in FIGS. 18, 21, and 22.

[0113] (15) In the above-described embodiment, it is assumed that even if the value is incorrect, some value is read and displayed from the image captured of the liquid crystal screen of the blood pressure monitor. However, in reality, there may be cases where the blood pressure value cannot be read. FIG. 24 is a diagram for explaining another display example of the camera output screen 150 and the read value confirmation screen 160. In FIG. 24, the corresponding parts to those in FIG. 14 are denoted by the same reference numerals. On the read value confirmation screen 160 in FIG. 14, the blood pressure value and the pulse value read by image analysis are displayed in the column of the read value 163.

[0114] However, on the reading value confirmation screen 160 shown in FIG. 24, "The value could not be read" is displayed in the column of the reading value 163. The display shown in FIG. 24 is displayed when even one of the numerical values to be displayed cannot be read. Here, the inability to read a numerical value means a state where the reliability of the numerical value read as a result of image analysis is lower than the threshold value. The reliability is calculated in the process of image analysis and output as part of the image analysis result. In addition, when even one of the numerical values to be displayed cannot be read, the read numerical value may be displayed at the location where the numerical value was read, the location where the numerical value could not be read may be left blank, and "There is a value that could not be read" or the like may be displayed separately. This display mode is an example of a display that associates the fact that the acquisition of the measurement value has failed with the image. This display can notify the user of the necessity of retaking the image used for the analysis of the measurement value.

[0115] (16) In the above-described embodiment, the case where the reading value confirmation screen 160 (see FIG. 14) is displayed when the imaging button 153 is operated on the camera output screen 150 (see FIG. 14) has been described. However, it takes about several seconds (for example, 2 to 3 seconds) from the start of the image analysis until the analysis result (for example, blood pressure value) is displayed. Note that this time is an example, and it depends on, for example, the processing ability of the user terminal 10 (see FIG. 1), the quality of the image to be analyzed, and the shooting environment. During the image analysis, the user has only to wait for the display of the analysis result. This waiting time (that is, idle time) is wasted time for the user.

[0116] Therefore, it is considered to use this idle time for explaining how to use the treatment application and the operation method. FIG. 25 is a diagram for explaining a transition screen 180 displayed in the process of transitioning from the camera output screen 150 to the reading value confirmation screen 160. In FIG. 25, the same reference numerals as the corresponding parts in FIG. 14 are shown. The transition screen 180 shown in FIG. 25 is given a title of "Blood pressure meter screen loading".

[0117] In addition, on the transition screen 180 shown in FIG. 25, a display column 181 for the processing state and a message column 182 are arranged. In the message column 182, for example, instructions on how to use the treatment application, precautions during measurement, measurement timing, and knowledge about diseases are displayed. In the case of FIG. 25, “Image analysis in progress” is displayed in the display column 181 for the processing state. With this display, the user can understand the current processing state. As a result, even when the time until the blood pressure value is displayed becomes long, the user can wait without anxiety for the display of the blood pressure value that is the analysis result. In the case of FIG. 25, the title “How to Use the App” is displayed in the message column 182, but depending on the content to be displayed, “Precautions during Measurement”, “Measurement Timing”, “Knowledge about Diseases”, etc. are displayed.

[0118] In the case of FIG. 25, “If you want to enter past blood pressure, you can manually enter it by pressing the icon in the upper right corner of the blood pressure data screen.” is displayed as “How to Use the App”. The user who sees this display can know that it is possible to enter past blood pressure manually. In the case of the above-described embodiment, it is possible to record past blood pressure values and the like from the images read from the image folder. Note that it is desirable that the content of the information displayed in the message column 182 be different information each time it is displayed. For example, information with a low frequency of user operations based on the user's operation history may be preferentially used as the display content. Alternatively, the display content may be changed randomly. With this display, it becomes possible to effectively utilize the idle time until the analysis result of the image is displayed.

[0119] <Summary> The disclosed examples described in the above-described embodiments are shown below. (((1))) A program for enabling a computer of a user terminal to have a function of acquiring, as measurement values, analysis results of images captured for each measurement session by a measuring device that measures biological information, a function of displaying the acquired measurement values on a display unit of the user terminal in association with the original images, and a function of accepting corrections to the measurement values through operations on the user terminal. According to this program, recording of the measured biological information can be supported. Also, correction of the acquired measurement values can be enabled.

[0120] (((2))) The function of accepting corrections is the program described in (((1))) that accepts correction inputs for the measurement values. According to this program, the numerical values displayed as analysis results can be easily corrected to correct values.

[0121] (((3))) The function of accepting corrections is the program described in (((1))) that updates the display on the display unit with the measurement values obtained from the re-captured image and the image after re-capture in response to an instruction for a re-capture operation. According to this program, even when re-capturing an image of the measurement value, the set of the image and the measurement value recorded in association with the measurement session can be made into one.

[0122] (((4))) The function of accepting corrections is the program described in (((2))) or (((3))) that records, in association with the measurement session, the numerical values displayed on the display unit at the time when an operation to confirm the recording of the measurement values is accepted. According to this program, the set of the image and the measurement value recorded in association with the measurement session can be made into one.

[0123] (((5))) The function of acquiring is the program described in any one of (((1))) to (((4))) that extracts, as analysis targets, images including the measuring device among the images stored in the image folder of the user terminal. According to this program, the number of images to be analyzed can be limited.

[0124] (((6))) The function to be obtained is the program described in any one of ((1)) to ((5)), which extracts a plurality of images with different measurement times as analysis targets. According to this program, the number of times of recording measurement values can be reduced.

[0125] (((7))) The function to be obtained is the program described in any one of ((1)) to ((4)), which uses the image selected by the user among the images captured by the user terminal as the analysis target. According to this program, the number of images to be analyzed can be limited.

[0126] (((8))) The function to be obtained is the program described in any one of ((1)) to ((6)), which extracts the images captured within a period retrogressing a predetermined time from the startup time of the self-program as processing targets. According to this program, the number of images to be analyzed can be limited.

[0127] (((9))) The function to be obtained is the program described in any one of ((1)) to ((6)), which extracts, as processing targets, the images that have not been acquired yet among the images whose elapsed time from the imaging time to the current time is within a predetermined time. According to this program, the number of images to be analyzed can be limited.

[0128] (((10))) The function to be obtained is the program described in any one of ((1)) to ((9)), which gives an image to the server and acquires the analysis result by the server as a measurement value. According to this program, the analysis accuracy of the measurement value can be improved.

[0129] (((11))) The function to be obtained is the program described in ((10)) that notifies the server of the measurement item corresponding to the measurement value to be recorded. According to this program, the analysis accuracy of the measurement value can be improved.

[0130] ((12)) The function to be obtained is the program described in any one of ((1)) to ((11)) that obtains the measurement value through the analysis of the image including the scale when the output of the measurement result by the measuring instrument is in analog format. According to this program, a measuring instrument with analog output can be included in the processing target.

[0131] ((13)) The program described in any one of ((1)) to ((12)) is a program that requires the input of a prescription code issued by a medical institution at the start of use. According to this program, the patient can be habituated to the daily recording of the measurement value.

[0132] ((14)) When obtaining the measurement value, it further has a function of displaying candidates of the image to be analyzed on the display unit, and the function of displaying candidates of the image on the display unit displays that the first candidate that has been used for obtaining the measurement value among the candidates has been used for obtaining the measurement value, or displays the first candidate in a non-selectable manner, or displays candidates other than the first candidate on the display unit. It is the program described in any one of ((1)) to ((13)). According to this program, it is possible to prevent the re-acquisition of the measurement value from the image that has been used for the analysis of the measurement value.

[0133] ((15)) It further has a function of displaying the average value of a plurality of measurement values obtained from the plurality of images corresponding to the time zone corresponding to the time when the plurality of images taken within a predetermined time are taken as the measurement value of the time zone. It is the program described in any one of ((1)) to ((14)). According to this program, the average value of the measurement values corresponding to each time zone can be automatically displayed. For example, even when the measurement values in the morning of a certain day are recorded at night on the same day or on another day, the average value of the measurement values can be displayed in association with the time zone corresponding to the time when each image was captured.

[0134] (((16))) When the measurement values cannot be obtained from the image through analysis, the program according to any one of ((1)) to ((14)) further has a function of displaying a message indicating that the acquisition of the measurement values has failed in association with the image. According to this program, it is possible to notify the user of the necessity of retaking the image used for the analysis of the measurement values.

[0135] (((17))) The program according to any one of ((1)) to ((14)) further has a function of displaying information regarding at least one of the usage method of the self-program, precautions during measurement, measurement timing, and knowledge about diseases from the start of the analysis of the image until the analysis result is obtained. According to this program, the idle time until the analysis result of the image is displayed can be effectively utilized.

Explanation of Signs

[0136] 10… User terminal, 14A, 34A… Text recognition model, 14B… Treatment application, 14C… Image folder, 20… Sphygmomanometer, 21… Display screen, 22… Mercury sphygmomanometer, 23… Scale, 24… Weighing scale, 25… Dial, 30… Analysis model, 34B… Natural language analysis model

Claims

1. A program for causing a computer of a user terminal to implement: a function of acquiring, as a measurement value, an analysis result of an image obtained by imaging a measuring device that measures biological information for each measurement; a function of displaying the acquired measurement value on a display unit of the user terminal in association with the image of the acquisition source; a function of accepting correction of the measurement value through an operation on the user terminal.

2. The program according to claim 1, wherein the function of accepting correction accepts a correction input for the measurement value.

3. The program according to claim 1, wherein the function of accepting correction updates the display on the display unit with the measurement value obtained from the re-imaged image and the image after re-imaging in response to an instruction for a re-imaging operation.

4. The program according to claim 2 or 3, wherein the function of accepting correction records, in association with the measurement time, the numerical value displayed on the display unit when an operation for determining the recording of the measurement value is accepted.

5. The program according to claim 1, wherein the function of acquiring extracts, as an analysis target, an image including the measuring device from among the images stored in an image folder of the user terminal.

6. The program according to claim 5, wherein the function of acquiring extracts a plurality of images with different measurement times as the analysis targets.

7. The program according to claim 1, wherein the function of acquiring uses, as an analysis target, an image selected by the user from among the images captured by the user terminal.

8. The program according to claim 1, wherein the function of acquiring extracts, as a processing target, an image captured within a period retrogressing a predetermined time from the startup time of the self-program.

9. The program according to claim 1, wherein the function of acquiring extracts, as a processing target, an image that has not been an acquisition target yet among the images in which the time elapsed from the imaging time to the current time is within a predetermined time.

10. The program according to claim 1, wherein the function of acquiring provides the image to a server and acquires, as the measurement value, an analysis result by the server.

11. The program according to claim 10, wherein the function of acquiring notifies the server of a measurement item corresponding to the measurement value to be recorded.

12. The program according to claim 1, wherein the function of acquiring acquires the measurement value through analysis of the image including a scale when the output of the measurement result by the measuring device is in an analog format. The program according to claim 1.

13. The program according to claim 1 is a program that requires the input of a prescription code issued by a medical institution at the start of use.

14. When acquiring measurement values, it further has a function of displaying candidates for images to be analyzed on the display unit, The function of displaying the candidates for the images on the display unit displays that the first candidate among the candidates that has been used for acquiring measurement values has been used for acquiring measurement values, or displays the first candidate in a non-selectable manner, or displays candidates other than the first candidate on the display unit. The program according to claim 1.

15. As measurement values for a time zone corresponding to the time when a plurality of images captured within a predetermined time were captured, a function of displaying an average value of a plurality of measurement values obtained from the plurality of images corresponding to the time zone, The program according to claim 1, which further has.

Images