System

Abstract

To provide a system for preventing wrong dosage by accurately grasping the category, administration, service period, and expiration date of a medicine.SOLUTION: The system includes an acquisition part, a determination part, a reading part, a specification part, and a provision part. The acquisition part acquires the image of the medicine captured by an imaging part. The determination part determines the category and administration of the medicine whose image is acquired by the acquisition part. The reading part reads the seal of a prescription. The specification part specifies the service period and expiration date of the medicine on the basis of the seal read by the reading part. The provision part provides a determination result by the determination part and a specification result by the specification part to a user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, it was difficult to accurately grasp the type of medicine, its usage, period of use, and expiration date, which posed the risk of misuse.

[0005] The system according to the embodiment aims to accurately grasp the type, usage, period of use, and expiration date of medicine, and to prevent misuse. [Means for solving the problem]

[0006] The system according to the embodiment includes an acquisition unit, a determination unit, a reading unit, an identification unit, and a providing unit. The acquisition unit acquires an image of the medicine captured by the imaging unit. The determination unit determines the type and usage of the medicine whose image was acquired by the acquisition unit. The reading unit reads the sticker on the prescription. The identification unit identifies the usage period and expiration date of the medicine based on the sticker read by the reading unit. The providing unit provides the user with the determination result by the determination unit and the identification result by the identification unit. Effect of the Invention

[0007] The system according to the embodiment can accurately grasp the type, usage, period of use, and expiration date of a medicine, and can prevent misuse. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The AI ​​system according to the embodiment of the present invention is a system that provides various information, mainly on how to use prescribed medicines and over-the-counter medicines. This AI system holds up a camera to determine the type and usage of the medicine, reads the prescription label to record the period of use and expiration date, records information such as chronic illnesses and current physical condition, and based on this information, presents related information such as foods and other items that require caution when taking the medicine. For example, when a user holds up a camera to take an image of a medicine, the image is acquired by the acquisition unit, and the determination unit determines the type and usage of the medicine. Next, there is a reading unit for reading the prescription label, and the reading unit reads the label. The read information is analyzed by the identification unit, and the period of use and expiration date of the medicine are identified. In addition, there is a management unit that manages information on the user's chronic illnesses and current physical condition, and a provision unit that provides information that requires caution based on this information. For example, if a specific medicine may have a negative effect on a chronic illness, the information is notified to the user. Finally, the provision unit provides the user with items that require caution when taking the medicine and related information based on the information obtained by the determination unit, identification unit, and management unit. For example, if there are foods or drinks that should not be taken with a particular medicine, that information is provided to the user. In this way, the AI ​​system of the present invention comprehensively manages information on the type of medicine, dosage, period of use, expiration date, chronic illnesses and physical condition, and provides appropriate information to the user, thereby supporting the safe use of medicine. As a result, the AI ​​system can comprehensively manage information on the type of medicine, dosage, period of use, expiration date, chronic illnesses and physical condition, and provide appropriate information to the user.

[0029] The AI ​​system according to the embodiment includes an acquisition unit, a determination unit, a reading unit, a specification unit, and a provision unit. The acquisition unit acquires an image of the medicine captured by the imaging unit. For example, the acquisition unit can capture an image of the medicine using a camera and acquire the image data. The determination unit determines the type and usage of the medicine whose image is acquired by the acquisition unit. For example, the determination unit can identify the type of medicine using image recognition technology and determine the method of taking it. The reading unit is a part for reading the sticker of the prescription. For example, the reading unit can read the information written on the sticker using OCR technology. The specification unit specifies the use period and expiration date of the medicine based on the sticker read by the reading unit. For example, the specification unit can analyze the information written on the sticker and specify the use period and expiration date of the medicine. The provision unit provides the determination result by the determination unit and the specification result by the specification unit. For example, the provision unit can provide the user with information on the type, usage, period, and expiration date of the medicine. As a result, the AI ​​system can comprehensively manage the type, usage, period, and expiration date of the medicine and provide the user with appropriate information.

[0030] The acquisition unit acquires the image of the medicine captured by the imaging unit. For example, the acquisition unit can capture an image of the medicine using a camera and acquire the image data. Specifically, the acquisition unit uses a high-resolution camera to capture detailed images of the medicine package and tablets. This allows the medicine's features such as its shape, color, and engraving to be accurately captured. Furthermore, the acquisition unit can grasp the overall image of the medicine by capturing multiple images from different angles. This improves the accuracy of identifying the medicine and reduces the risk of misrecognition. The acquisition unit processes the captured image data in real time and transmits it to a central database. The database stores images and related information of medicines captured in the past, and is used as basic data for identifying the type of medicine by comparing it with newly captured images. Furthermore, the acquisition unit has a function of automatically evaluating the quality of the image and re-capturing unclear or incomplete images. This allows high-quality image data to be always acquired and provided to subsequent processing departments.

[0031] The determination unit determines the type and usage of the medicine whose image is acquired by the acquisition unit. For example, the determination unit can use image recognition technology to identify the type of medicine and determine the method of taking it. Specifically, the determination unit employs an image recognition algorithm using deep learning to analyze features such as the shape, color, and engraving of the medicine. This allows the type of medicine to be identified with high accuracy. Furthermore, the determination unit estimates the method of taking the medicine from the information written on the medicine package and the shape of the medicine. For example, it determines whether the medicine is an oral or external medicine from the shape and size of the tablet, and identifies the method of use. In addition, the determination unit checks the accuracy of the type and usage of the medicine by comparing it with a past database. This reduces the risk of misrecognition and makes it possible to provide accurate information. Furthermore, the determination unit can learn new medicine information using AI and continuously improve the recognition accuracy. This allows the determination unit to always make accurate judgments based on the latest medicine information and provide users with reliable information.

[0032] The reading unit is a part for reading prescription stickers. For example, the reading unit can read information written on stickers using OCR technology. Specifically, the reading unit is equipped with a highly accurate OCR engine and converts the text information written on the sticker into digital data quickly and accurately. This allows the contents of the prescription to be read accurately, regardless of whether it is handwritten or printed. Furthermore, the reading unit has a function to automatically adjust the position and angle of the sticker, so that it can accurately read even if the sticker is affixed at an angle. The reading unit analyzes the read information in real time and sends it to a central database. The database stores past prescription information, and by comparing it with newly read information, the accuracy of the prescription contents can be confirmed. Furthermore, the reading unit has a function to automatically detect reading errors and re-read. This allows high-precision information to be obtained at all times and provided to subsequent processing departments.

[0033] The identification unit identifies the use period and expiration date of the drug based on the sticker read by the reading unit. For example, the identification unit can analyze the information written on the sticker and identify the use period and expiration date of the drug. Specifically, the identification unit uses OCR technology to analyze the read character information and extracts information such as the start date of use, the end date of use, and the expiration date. This makes it possible to accurately identify the use period and expiration date of the drug. Furthermore, the identification unit checks the accuracy of the use period and expiration date by comparing it with a past database. For example, by comparing it with past prescription information, it is possible to consistently manage the use period when the same drug is prescribed multiple times. In addition, the identification unit can learn new prescription information using AI and continuously improve the identification accuracy. As a result, the identification unit can always perform accurate identification based on the latest information and provide reliable information to the user. Furthermore, the identification unit can store the identified use period and expiration date as digital data and provide it to the subsequent provision department. As a result, the identification unit can comprehensively manage the use period and expiration date of the drug and provide appropriate information to the user.

[0034] The providing unit provides the determination result by the determining unit and the identification result by the identifying unit. For example, the providing unit can provide the user with information on the type, usage, period of use, and expiration date of the medicine. Specifically, the providing unit provides detailed information on the type, usage, period of use, and expiration date of the medicine through an application that displays information on the user's smartphone or tablet. This allows the user to check the medicine information anytime and anywhere. Furthermore, the providing unit has a voice assistant function and can provide medicine information by voice. This can accommodate users who have difficulty checking text information, such as visually impaired users and elderly users. In addition, the providing unit has a reminder function and can notify users when the time to take the medicine or the expiration date is approaching. This allows the user to use the medicine at the appropriate time without forgetting to take the medicine. Furthermore, the providing unit can collect user feedback and continuously improve the accuracy and effect of the information provided. For example, a more user-friendly system can be provided by reviewing the method of displaying information and the timing of notification based on user feedback. This allows the providing unit to provide users with quick and accurate information and support medicine management.

[0035] The management unit can manage information on the user's chronic illness and current physical condition. The management unit can, for example, record and manage information on the chronic illness and current physical condition input by the user. For example, the management unit can record information on the chronic illness, such as the disease name, symptoms, and treatment history input by the user. The management unit can also record information on the current physical condition, such as the body temperature, blood pressure, and heart rate input by the user. This allows the management unit to comprehensively manage information on the user's chronic illness and current physical condition. Some or all of the above-mentioned processing in the management unit may be performed using, for example, AI, or may be performed without using AI. For example, the management unit can input information on the chronic illness and current physical condition input by the user to AI, and the AI ​​can analyze and manage the information.

[0036] The providing unit can provide the user with information requiring attention based on the determination result by the determining unit, the identification result by the identifying unit, and the information managed by the managing unit. The providing unit can provide the user with information requiring attention based on, for example, the determination result of the type and usage of the medicine by the determining unit, the identification result of the use period and expiration date of the medicine by the identifying unit, and the information on the user's chronic illness and current physical condition managed by the managing unit. For example, the providing unit can notify the user of information that a specific medicine may adversely affect the user's chronic illness. In addition, the providing unit can also provide the user with information on foods and drinks that should not be taken with a specific medicine. In this way, the providing unit can provide the user with information requiring attention regarding the use of the medicine. Part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input information obtained by the determining unit, identifying unit, and managing unit to AI, and the AI ​​can analyze the information and provide the user with information requiring attention.

[0037] The providing unit can provide the user with information on drugs that require caution in combination with other drugs and related information. For example, if there is a food or drink that should not be taken with a specific drug, the providing unit can provide the user with that information. For example, the providing unit can notify the user of a food or drink that may cause side effects when taken with a specific drug. The providing unit can also notify the user of a food or drink that may reduce the effect of a specific drug when taken with the specific drug. This allows the providing unit to provide the user with information on drugs and food combinations that require caution. Some or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input information on drug and food combinations into AI, and the AI ​​can analyze the information and provide it to the user.

[0038] The acquisition unit can analyze the user's past drug use history and select the optimal image acquisition method. The acquisition unit can suggest the optimal image acquisition method based on, for example, the type of drug used by the user in the past. The acquisition unit can also determine the priority of image acquisition for a specific drug from the user's past drug use history. Furthermore, the acquisition unit can analyze the user's past drug use history and select the most efficient image acquisition method. This allows the acquisition unit to select the optimal image acquisition method based on the user's past drug use history. A part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI or may be performed without using AI. For example, the acquisition unit can input the user's past drug use history data to the generation AI, and the generation AI can analyze the data and select the optimal image acquisition method.

[0039] The acquisition unit can perform filtering based on the user's current living situation and field of interest when acquiring images of medicines. For example, if the user is interested in health, the acquisition unit can preferentially acquire images of medicines related to health. In addition, if the user is busy, the acquisition unit can also prioritize images of medicines that can be acquired quickly. Furthermore, if the user is interested in a specific disease, the acquisition unit can also preferentially acquire images of medicines related to the disease. This allows the acquisition unit to acquire optimal images of medicines according to the user's living situation and field of interest. Part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input data related to the user's living situation and field of interest into the generation AI, and the generation AI can analyze the data to filter optimal images of medicines.

[0040] When acquiring images of medicines, the acquisition unit can preferentially acquire images of medicines that are highly relevant in consideration of the geographical location information of the user. For example, when the user is in a specific area, the acquisition unit can preferentially acquire images of medicines that are commonly used in that area. In addition, when the user is traveling, the acquisition unit can also preferentially acquire images of medicines that are needed at the travel destination. Furthermore, when the user is at home, the acquisition unit can also preferentially acquire images of medicines that are used daily. This allows the acquisition unit to preferentially acquire images of optimal medicines based on the geographical location information of the user. Part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input the geographical location information of the user to the generation AI, and the generation AI can analyze the data and preferentially acquire images of medicines that are highly relevant.

[0041] The acquisition unit can analyze the user's social media activity when acquiring the image of the medicine, and can acquire the image of the related medicine. For example, when the user mentions a specific disease on social media, the acquisition unit can acquire the image of the medicine related to the disease. In addition, when the user posts about health on social media, the acquisition unit can also acquire the image of the medicine related to health. Furthermore, when the user mentions a specific medicine on social media, the acquisition unit can also acquire the image of the medicine. In this way, the acquisition unit can acquire the optimal image of the medicine based on the user's social media activity. A part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input the user's social media activity data to the generation AI, and the generation AI can analyze the data to acquire the image of the related medicine.

[0042] The judgment unit can adjust the level of detail of the judgment based on the importance of the medicine when judging the type and usage of the medicine. For example, in the case of an important medicine, the judgment unit can provide a detailed judgment result. In addition, in the case of a general medicine, the judgment unit can also provide a concise judgment result. Furthermore, in the case of a medicine to be used in an emergency, the judgment unit can quickly provide a judgment result. This allows the judgment unit to provide an optimal judgment result according to the importance of the medicine. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can input the importance data of the medicine to the generation AI, and the generation AI can analyze the data and adjust the level of detail of the judgment.

[0043] The judgment unit can apply different judgment algorithms depending on the category of the medicine when judging the type and usage of the medicine. For example, in the case of an antibiotic, the judgment unit can apply a specific judgment algorithm. In addition, in the case of a painkiller, the judgment unit can apply another judgment algorithm. Furthermore, in the case of a vitamin supplement, the judgment unit can apply a further different judgment algorithm. This allows the judgment unit to apply an optimal judgment algorithm depending on the category of the medicine. A part or all of the above-mentioned processing in the judgment unit may be performed using, for example, AI, or may be performed without using AI. For example, the judgment unit can input medicine category data to the generation AI, and the generation AI can analyze the data and apply different judgment algorithms.

[0044] The judgment unit can determine the priority of judgment based on the time of submission of the medicine when judging the type and usage of the medicine. For example, the judgment unit can give priority to judgment for a recently submitted medicine. In addition, the judgment unit can postpone judgment for a medicine that has been used for a long time. Furthermore, the judgment unit can give top priority to judgment for a medicine used in an emergency. This allows the judgment unit to determine the optimal judgment order based on the time of submission of the medicine. A part or all of the above-mentioned processing in the judgment unit may be performed using, for example, AI, or may be performed without using AI. For example, the judgment unit can input data on the time of submission of the medicine to the generation AI, and the generation AI can analyze the data to determine the priority of judgment.

[0045] The determination unit can adjust the order of determination based on the relevance of the medicines when determining the type and usage of the medicine. For example, in the case of medicines in the same category, the determination unit can perform determination in order of high relevance. In addition, in the case of medicines in different categories, the determination unit can also perform determination in order of low relevance. Furthermore, in the case of medicines related to a specific disease, the determination unit can also preferentially determine medicines related to the disease. This allows the determination unit to adjust the optimal determination order based on the relevance of the medicines. A part or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the relevance data of the medicines to the generation AI, and the generation AI can analyze the data and adjust the order of determination.

[0046] When reading a prescription sticker, the reading unit can adjust the level of detail of the reading based on the importance of the sticker. For example, the reading unit can read in detail a sticker on which important information is written. The reading unit can also read briefly a sticker on which general information is written. Furthermore, the reading unit can quickly read a sticker on a medicine to be used in an emergency. This allows the reading unit to provide an optimal level of detail of reading depending on the importance of the sticker. Some or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input the importance data of the sticker to the generating AI, and the generating AI can analyze the data to adjust the level of detail of the reading.

[0047] When reading the prescription sticker, the reading unit can determine the priority of reading based on the submission time of the sticker. For example, the reading unit can give priority to reading a sticker that has been submitted recently. In addition, the reading unit can also postpone reading a sticker that has been used for a long time. Furthermore, the reading unit can give top priority to reading a sticker for a medicine to be used in an emergency. This allows the reading unit to determine the optimal reading order based on the submission time of the sticker. A part or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input sticker submission time data to the generating AI, and the generating AI can analyze the data to determine the priority of reading.

[0048] The identification unit can adjust the level of detail of the identification based on the importance of the drug when identifying the use period and expiration date of the drug. For example, the identification unit can provide a detailed identification result in the case of an important drug. The identification unit can also provide a concise identification result in the case of a general drug. Furthermore, the identification unit can quickly provide an identification result in the case of a drug used in an emergency. This allows the identification unit to provide an optimal identification result according to the importance of the drug. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input the importance data of the drug to the generation AI, and the generation AI can analyze the data to adjust the level of detail of the identification.

[0049] The identification unit can apply different identification algorithms depending on the category of the drug when identifying the use period and expiration date of the drug. For example, in the case of an antibiotic, the identification unit can apply a specific identification algorithm. In addition, in the case of a painkiller, the identification unit can apply another identification algorithm. Furthermore, in the case of a vitamin supplement, the identification unit can apply a further different identification algorithm. This allows the identification unit to apply an optimal identification algorithm depending on the category of the drug. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input drug category data to the generation AI, and the generation AI can analyze the data and apply different identification algorithms.

[0050] The identification unit can determine the identification priority based on the submission time of the medicine when identifying the use period and expiration date of the medicine. For example, the identification unit can prioritize identification in the case of a recently submitted medicine. In addition, the identification unit can also postpone identification in the case of a medicine that has been used for a long time. Furthermore, the identification unit can give top priority to identification in the case of a medicine used in an emergency. This allows the identification unit to determine the optimal identification order based on the submission time of the medicine. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input data on the submission time of the medicine to the generation AI, and the generation AI can analyze the data to determine the identification priority.

[0051] The identification unit can adjust the identification order based on the relevance of the medicines when identifying the use period and expiration date of the medicine. For example, in the case of medicines in the same category, the identification unit can identify the medicines in order of high relevance. In addition, in the case of medicines in different categories, the identification unit can identify the medicines in order of low relevance. Furthermore, in the case of medicines related to a specific disease, the identification unit can preferentially identify the medicines related to the disease. This allows the identification unit to adjust the optimal identification order based on the relevance of the medicines. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input the relevance data of the medicines to the generation AI, and the generation AI can analyze the data and adjust the identification order.

[0052] The providing unit can adjust the level of detail of the information provided based on the importance of the information when providing the information. For example, the providing unit can provide a detailed explanation in the case of important information. In addition, the providing unit can provide a concise explanation in the case of general information. Furthermore, the providing unit can provide information quickly in the case of information required in an emergency. This allows the providing unit to provide an optimal level of detail of the information provided depending on the importance of the information. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the providing unit can input the importance data of the information to the generating AI, and the generating AI can analyze the data to adjust the level of detail of the information provided.

[0053] The providing unit can apply different providing algorithms depending on the category of information at the time of providing. For example, in the case of information on health, the providing unit can apply a specific providing algorithm. In addition, in the case of information on how to use medicine, the providing unit can apply another providing algorithm. Furthermore, in the case of information on food combinations, the providing unit can apply a further different providing algorithm. This allows the providing unit to apply an optimal providing algorithm depending on the category of information. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input category data of information to the generating AI, and the generating AI can analyze the data and apply different providing algorithms.

[0054] The providing unit can determine the priority of the provision based on the time of submission of the information at the time of provision. For example, the providing unit can provide information that has been submitted recently with priority. In addition, the providing unit can postpone the provision of information that has been used for a long time. Furthermore, the providing unit can provide information that is necessary in an emergency with the highest priority. This allows the providing unit to determine the optimal order of provision based on the time of submission of the information. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the submission time data of the information to the generating AI, and the generating AI can analyze the data to determine the priority of the provision.

[0055] The providing unit can adjust the order of provision based on the relevance of information at the time of provision. For example, in the case of information of the same category, the providing unit can provide the information in order of high relevance. In addition, in the case of information of different categories, the providing unit can also provide the information in order of low relevance. Furthermore, in the case of information related to a specific disease, the providing unit can also provide information related to the disease preferentially. Thereby, the providing unit can adjust the optimal order of provision based on the relevance of information. A part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the relevance data of information to the generating AI, and the generating AI can analyze the data and adjust the order of provision.

[0056] The management unit can adjust the level of detail of management based on the importance of information when managing chronic disease and physical condition information. For example, the management unit can manage important information in detail. The management unit can also manage general information simply. Furthermore, the management unit can quickly manage information required in an emergency. This allows the management unit to provide an optimal level of detail of management depending on the importance of the information. Some or all of the above-mentioned processing in the management unit may be performed using, for example, AI, or may be performed without using AI. For example, the management unit can input information importance data to the generation AI, and the generation AI can analyze the data and adjust the level of detail of management.

[0057] The management unit can determine the priority of management based on the time of information submission when managing chronic disease and physical condition information. For example, the management unit can prioritize management of recently submitted information. In addition, the management unit can postpone information that has been used for a long time. Furthermore, the management unit can manage information required in an emergency with the highest priority. This allows the management unit to determine the optimal management order based on the time of information submission. A part or all of the above-mentioned processing in the management unit may be performed using, for example, AI, or may be performed without using AI. For example, the management unit can input information submission time data to the generation AI, and the generation AI can analyze the data to determine the priority of management.

[0058] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0059] The acquisition unit can analyze the user's past drug use history and select the optimal image acquisition method. For example, the acquisition unit can suggest the optimal image acquisition method based on the type of drug used by the user in the past. The acquisition unit can also determine the priority of image acquisition for a specific drug from the user's past drug use history. Furthermore, the acquisition unit can analyze the user's past drug use history and select the most efficient image acquisition method. This allows the acquisition unit to select the optimal image acquisition method based on the user's past drug use history. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, AI, or may be performed without using AI. For example, the acquisition unit can input the user's past drug use history data to the generation AI, and the generation AI can analyze the data and select the optimal image acquisition method.

[0060] The judgment unit can adjust the level of detail of the judgment based on the importance of the medicine when judging the type and usage of the medicine. For example, in the case of an important medicine, a detailed judgment result can be provided. In addition, the judgment unit can also provide a concise judgment result in the case of a general medicine. Furthermore, in the case of a medicine to be used in an emergency, the judgment unit can quickly provide a judgment result. This allows the judgment unit to provide an optimal judgment result according to the importance of the medicine. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can input the importance data of the medicine to the generation AI, and the generation AI can analyze the data and adjust the level of detail of the judgment.

[0061] When reading a prescription sticker, the reading unit can adjust the level of detail of the reading based on the importance of the sticker. For example, in the case of a sticker containing important information, the reading unit can read in detail. In addition, in the case of a sticker containing general information, the reading unit can also read it briefly. Furthermore, in the case of a sticker for medicine to be used in an emergency, the reading unit can also read it quickly. This allows the reading unit to provide an optimal level of detail of reading depending on the importance of the sticker. Some or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input the importance data of the sticker to the generating AI, and the generating AI can analyze the data and adjust the level of detail of the reading.

[0062] The identification unit can apply different identification algorithms depending on the category of the drug when identifying the period of use and the expiration date of the drug. For example, in the case of an antibiotic, a specific identification algorithm can be applied. In addition, the identification unit can apply another identification algorithm in the case of a painkiller. Furthermore, the identification unit can apply a further different identification algorithm in the case of a vitamin supplement. This allows the identification unit to apply an optimal identification algorithm depending on the category of the drug. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input drug category data to the generation AI, and the generation AI can analyze the data and apply different identification algorithms.

[0063] The providing unit can adjust the level of detail of the information provided based on the importance of the information when providing the information. For example, in the case of important information, a detailed explanation can be provided. In addition, the providing unit can provide a concise explanation in the case of general information. Furthermore, the providing unit can provide information quickly in the case of information required in an emergency. This allows the providing unit to provide an optimal level of detail of the information provided depending on the importance of the information. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the providing unit can input the importance data of the information to the generating AI, and the generating AI can analyze the data to adjust the level of detail of the information provided.

[0064] The process flow of the first embodiment will be briefly described below.

[0065] Step 1: The acquisition unit acquires an image of the medicine captured by the imaging unit. For example, the acquisition unit can capture an image of the medicine using a camera and acquire the image data. Step 2: The determination unit determines the type and method of use of the medicine whose image has been acquired by the acquisition unit. For example, the determination unit can identify the type of medicine using image recognition technology and determine the method of taking the medicine. Step 3: The reading unit is a unit for reading the prescription sticker. For example, the reading unit can read the information written on the sticker using OCR technology. Step 4: The identification unit identifies the use period and expiration date of the medicine based on the label read by the reading unit. For example, the identification unit can analyze the information written on the label and identify the use period and expiration date of the medicine. Step 5: The providing unit provides the determination result by the determining unit and the identification result by the identifying unit. For example, the providing unit can provide the user with information regarding the type, usage, period of use, and expiration date of the medicine.

[0066] (Example 2) The AI ​​system according to the embodiment of the present invention is a system that provides various information, mainly on how to use prescribed medicines and over-the-counter medicines. This AI system holds up a camera to determine the type and usage of the medicine, reads the prescription label to record the period of use and expiration date, records information such as chronic illnesses and current physical condition, and based on this information, presents related information such as foods and other items that require caution when taking the medicine. For example, when a user holds up a camera to take an image of a medicine, the image is acquired by the acquisition unit, and the determination unit determines the type and usage of the medicine. Next, there is a reading unit for reading the prescription label, and the reading unit reads the label. The read information is analyzed by the identification unit, and the period of use and expiration date of the medicine are identified. In addition, there is a management unit that manages information on the user's chronic illnesses and current physical condition, and a provision unit that provides information that requires caution based on this information. For example, if a specific medicine may have a negative effect on a chronic illness, the information is notified to the user. Finally, the provision unit provides the user with items that require caution when taking the medicine and related information based on the information obtained by the determination unit, identification unit, and management unit. For example, if there are foods or drinks that should not be taken with a particular medicine, that information is provided to the user. In this way, the AI ​​system of the present invention comprehensively manages information on the type of medicine, dosage, period of use, expiration date, chronic illnesses and physical condition, and provides appropriate information to the user, thereby supporting the safe use of medicine. As a result, the AI ​​system can comprehensively manage information on the type of medicine, dosage, period of use, expiration date, chronic illnesses and physical condition, and provide appropriate information to the user.

[0067] The AI ​​system according to the embodiment includes an acquisition unit, a determination unit, a reading unit, a specification unit, and a provision unit. The acquisition unit acquires an image of the medicine captured by the imaging unit. For example, the acquisition unit can capture an image of the medicine using a camera and acquire the image data. The determination unit determines the type and usage of the medicine whose image is acquired by the acquisition unit. For example, the determination unit can identify the type of medicine using image recognition technology and determine the method of taking it. The reading unit is a part for reading the sticker of the prescription. For example, the reading unit can read the information written on the sticker using OCR technology. The specification unit specifies the use period and expiration date of the medicine based on the sticker read by the reading unit. For example, the specification unit can analyze the information written on the sticker and specify the use period and expiration date of the medicine. The provision unit provides the determination result by the determination unit and the specification result by the specification unit. For example, the provision unit can provide the user with information on the type, usage, period, and expiration date of the medicine. As a result, the AI ​​system can comprehensively manage the type, usage, period, and expiration date of the medicine and provide the user with appropriate information.

[0068] The acquisition unit acquires the image of the medicine captured by the imaging unit. For example, the acquisition unit can capture an image of the medicine using a camera and acquire the image data. Specifically, the acquisition unit uses a high-resolution camera to capture detailed images of the medicine package and tablets. This allows the medicine's features such as its shape, color, and engraving to be accurately captured. Furthermore, the acquisition unit can grasp the overall image of the medicine by capturing multiple images from different angles. This improves the accuracy of identifying the medicine and reduces the risk of misrecognition. The acquisition unit processes the captured image data in real time and transmits it to a central database. The database stores images and related information of medicines captured in the past, and is used as basic data for identifying the type of medicine by comparing it with newly captured images. Furthermore, the acquisition unit has a function of automatically evaluating the quality of the image and re-capturing unclear or incomplete images. This allows high-quality image data to be always acquired and provided to subsequent processing departments.

[0069] The determination unit determines the type and usage of the medicine whose image is acquired by the acquisition unit. For example, the determination unit can use image recognition technology to identify the type of medicine and determine the method of taking it. Specifically, the determination unit employs an image recognition algorithm using deep learning to analyze features such as the shape, color, and engraving of the medicine. This allows the type of medicine to be identified with high accuracy. Furthermore, the determination unit estimates the method of taking the medicine from the information written on the medicine package and the shape of the medicine. For example, it determines whether the medicine is an oral or external medicine from the shape and size of the tablet, and identifies the method of use. In addition, the determination unit checks the accuracy of the type and usage of the medicine by comparing it with a past database. This reduces the risk of misrecognition and makes it possible to provide accurate information. Furthermore, the determination unit can learn new medicine information using AI and continuously improve the recognition accuracy. This allows the determination unit to always make accurate judgments based on the latest medicine information and provide users with reliable information.

[0070] The reading unit is a part for reading prescription stickers. For example, the reading unit can read information written on stickers using OCR technology. Specifically, the reading unit is equipped with a highly accurate OCR engine and converts the text information written on the sticker into digital data quickly and accurately. This allows the contents of the prescription to be read accurately, regardless of whether it is handwritten or printed. Furthermore, the reading unit has a function to automatically adjust the position and angle of the sticker, so that it can accurately read even if the sticker is affixed at an angle. The reading unit analyzes the read information in real time and sends it to a central database. The database stores past prescription information, and by comparing it with newly read information, the accuracy of the prescription contents can be confirmed. Furthermore, the reading unit has a function to automatically detect reading errors and re-read. This allows high-precision information to be obtained at all times and provided to subsequent processing departments.

[0071] The identification unit identifies the use period and expiration date of the drug based on the sticker read by the reading unit. For example, the identification unit can analyze the information written on the sticker and identify the use period and expiration date of the drug. Specifically, the identification unit uses OCR technology to analyze the read character information and extracts information such as the start date of use, the end date of use, and the expiration date. This makes it possible to accurately identify the use period and expiration date of the drug. Furthermore, the identification unit checks the accuracy of the use period and expiration date by comparing it with a past database. For example, by comparing it with past prescription information, it is possible to consistently manage the use period when the same drug is prescribed multiple times. In addition, the identification unit can learn new prescription information using AI and continuously improve the identification accuracy. As a result, the identification unit can always perform accurate identification based on the latest information and provide reliable information to the user. Furthermore, the identification unit can store the identified use period and expiration date as digital data and provide it to the subsequent provision department. As a result, the identification unit can comprehensively manage the use period and expiration date of the drug and provide appropriate information to the user.

[0072] The providing unit provides the determination result by the determining unit and the identification result by the identifying unit. For example, the providing unit can provide the user with information on the type, usage, period of use, and expiration date of the medicine. Specifically, the providing unit provides detailed information on the type, usage, period of use, and expiration date of the medicine through an application that displays information on the user's smartphone or tablet. This allows the user to check the medicine information anytime and anywhere. Furthermore, the providing unit has a voice assistant function and can provide medicine information by voice. This can accommodate users who have difficulty checking text information, such as visually impaired users and elderly users. In addition, the providing unit has a reminder function and can notify users when the time to take the medicine or the expiration date is approaching. This allows the user to use the medicine at the appropriate time without forgetting to take the medicine. Furthermore, the providing unit can collect user feedback and continuously improve the accuracy and effect of the information provided. For example, a more user-friendly system can be provided by reviewing the method of displaying information and the timing of notification based on user feedback. This allows the providing unit to provide users with quick and accurate information and support medicine management.

[0073] The management unit can manage information on the user's chronic illness and current physical condition. The management unit can, for example, record and manage information on the chronic illness and current physical condition input by the user. For example, the management unit can record information on the chronic illness, such as the disease name, symptoms, and treatment history input by the user. The management unit can also record information on the current physical condition, such as the body temperature, blood pressure, and heart rate input by the user. This allows the management unit to comprehensively manage information on the user's chronic illness and current physical condition. Some or all of the above-mentioned processing in the management unit may be performed using, for example, AI, or may be performed without using AI. For example, the management unit can input information on the chronic illness and current physical condition input by the user to AI, and the AI ​​can analyze and manage the information.

[0074] The providing unit can provide the user with information requiring attention based on the determination result by the determining unit, the identification result by the identifying unit, and the information managed by the managing unit. The providing unit can provide the user with information requiring attention based on, for example, the determination result of the type and usage of the medicine by the determining unit, the identification result of the use period and expiration date of the medicine by the identifying unit, and the information on the user's chronic illness and current physical condition managed by the managing unit. For example, the providing unit can notify the user of information that a specific medicine may adversely affect the user's chronic illness. In addition, the providing unit can also provide the user with information on foods and drinks that should not be taken with a specific medicine. In this way, the providing unit can provide the user with information requiring attention regarding the use of the medicine. Part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input information obtained by the determining unit, identifying unit, and managing unit to AI, and the AI ​​can analyze the information and provide the user with information requiring attention.

[0075] The providing unit can provide the user with information on drugs that require caution in combination with other drugs and related information. For example, if there is a food or drink that should not be taken with a specific drug, the providing unit can provide the user with that information. For example, the providing unit can notify the user of a food or drink that may cause side effects when taken with a specific drug. The providing unit can also notify the user of a food or drink that may reduce the effect of a specific drug when taken with the specific drug. This allows the providing unit to provide the user with information on drugs and food combinations that require caution. Some or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input information on drug and food combinations into AI, and the AI ​​can analyze the information and provide it to the user.

[0076] The acquisition unit can estimate the user's emotion and adjust the timing of acquiring the medicine image based on the estimated user's emotion. For example, when the user is feeling stressed, the acquisition unit can adjust to acquire the medicine image at a timing when the user can relax. In addition, the acquisition unit can also adjust to acquire the medicine image quickly when the user is in a hurry. Furthermore, the acquisition unit can also adjust to take time to acquire a detailed image when the user is relaxed. In this way, the acquisition unit can acquire the medicine image at an optimal timing according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and adjust the timing of acquiring the medicine image.

[0077] The acquisition unit can analyze the user's past drug use history and select the optimal image acquisition method. The acquisition unit can suggest the optimal image acquisition method based on, for example, the type of drug used by the user in the past. The acquisition unit can also determine the priority of image acquisition for a specific drug from the user's past drug use history. Furthermore, the acquisition unit can analyze the user's past drug use history and select the most efficient image acquisition method. This allows the acquisition unit to select the optimal image acquisition method based on the user's past drug use history. A part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI or may be performed without using AI. For example, the acquisition unit can input the user's past drug use history data to the generation AI, and the generation AI can analyze the data and select the optimal image acquisition method.

[0078] The acquisition unit can perform filtering based on the user's current living situation and field of interest when acquiring images of medicines. For example, if the user is interested in health, the acquisition unit can preferentially acquire images of medicines related to health. In addition, if the user is busy, the acquisition unit can also prioritize images of medicines that can be acquired quickly. Furthermore, if the user is interested in a specific disease, the acquisition unit can also preferentially acquire images of medicines related to the disease. This allows the acquisition unit to acquire optimal images of medicines according to the user's living situation and field of interest. Part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input data related to the user's living situation and field of interest into the generation AI, and the generation AI can analyze the data to filter optimal images of medicines.

[0079] The acquisition unit can estimate the user's emotion and determine the priority of the image of the medicine to be acquired based on the estimated emotion of the user. For example, when the user feels anxious, the acquisition unit can preferentially acquire an image of a medicine that gives a sense of security. In addition, when the user is excited, the acquisition unit can also preferentially acquire an image of a medicine to regain composure. Furthermore, when the user is relaxed, the acquisition unit can also preferentially acquire an image of a medicine to provide detailed information. In this way, the acquisition unit can preferentially acquire an image of an optimal medicine according to the emotion of the user. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the emotion of the user and determine the priority of the image of the medicine.

[0080] When acquiring images of medicines, the acquisition unit can preferentially acquire images of medicines that are highly relevant in consideration of the geographical location information of the user. For example, when the user is in a specific area, the acquisition unit can preferentially acquire images of medicines that are commonly used in that area. In addition, when the user is traveling, the acquisition unit can also preferentially acquire images of medicines that are needed at the travel destination. Furthermore, when the user is at home, the acquisition unit can also preferentially acquire images of medicines that are used daily. This allows the acquisition unit to preferentially acquire images of optimal medicines based on the geographical location information of the user. Part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input the geographical location information of the user to the generation AI, and the generation AI can analyze the data and preferentially acquire images of medicines that are highly relevant.

[0081] The acquisition unit can analyze the user's social media activity when acquiring the image of the medicine, and can acquire the image of the related medicine. For example, when the user mentions a specific disease on social media, the acquisition unit can acquire the image of the medicine related to the disease. In addition, when the user posts about health on social media, the acquisition unit can also acquire the image of the medicine related to health. Furthermore, when the user mentions a specific medicine on social media, the acquisition unit can also acquire the image of the medicine. In this way, the acquisition unit can acquire the optimal image of the medicine based on the user's social media activity. A part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input the user's social media activity data to the generation AI, and the generation AI can analyze the data to acquire the image of the related medicine.

[0082] The determination unit can estimate the user's emotion and adjust the determination method of the type and usage of the medicine based on the estimated user's emotion. For example, when the user feels anxious, the determination unit can apply a determination method that provides a detailed explanation. Furthermore, when the user is relaxed, the determination unit can also apply a determination method that provides a concise explanation. Furthermore, when the user is in a hurry, the determination unit can apply a determination method that quickly provides a determination result. This allows the determination unit to apply an optimal determination method according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the determination unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the determination unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and adjust the determination method of the type and usage of the medicine.

[0083] The judgment unit can adjust the level of detail of the judgment based on the importance of the medicine when judging the type and usage of the medicine. For example, in the case of an important medicine, the judgment unit can provide a detailed judgment result. In addition, in the case of a general medicine, the judgment unit can also provide a concise judgment result. Furthermore, in the case of a medicine to be used in an emergency, the judgment unit can quickly provide a judgment result. This allows the judgment unit to provide an optimal judgment result according to the importance of the medicine. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can input the importance data of the medicine to the generation AI, and the generation AI can analyze the data and adjust the level of detail of the judgment.

[0084] The judgment unit can apply different judgment algorithms depending on the category of the medicine when judging the type and usage of the medicine. For example, in the case of an antibiotic, the judgment unit can apply a specific judgment algorithm. In addition, in the case of a painkiller, the judgment unit can apply another judgment algorithm. Furthermore, in the case of a vitamin supplement, the judgment unit can apply a further different judgment algorithm. This allows the judgment unit to apply an optimal judgment algorithm depending on the category of the medicine. A part or all of the above-mentioned processing in the judgment unit may be performed using, for example, AI, or may be performed without using AI. For example, the judgment unit can input medicine category data to the generation AI, and the generation AI can analyze the data and apply different judgment algorithms.

[0085] The determination unit can estimate the user's emotion and adjust the display method of the determination result based on the estimated user's emotion. For example, when the user is nervous, the determination unit can provide a simple and highly visible display method. Furthermore, when the user is relaxed, the determination unit can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the determination unit can also provide a display method that focuses on the main points. This allows the determination unit to provide an optimal display method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the determination unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the determination unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and adjust the display method of the determination result.

[0086] The judgment unit can determine the priority of judgment based on the time of submission of the medicine when judging the type and usage of the medicine. For example, the judgment unit can give priority to judgment for a recently submitted medicine. In addition, the judgment unit can postpone judgment for a medicine that has been used for a long time. Furthermore, the judgment unit can give top priority to judgment for a medicine used in an emergency. This allows the judgment unit to determine the optimal judgment order based on the time of submission of the medicine. A part or all of the above-mentioned processing in the judgment unit may be performed using, for example, AI, or may be performed without using AI. For example, the judgment unit can input data on the time of submission of the medicine to the generation AI, and the generation AI can analyze the data to determine the priority of judgment.

[0087] The determination unit can adjust the order of determination based on the relevance of the medicines when determining the type and usage of the medicine. For example, in the case of medicines in the same category, the determination unit can perform determination in order of high relevance. In addition, in the case of medicines in different categories, the determination unit can also perform determination in order of low relevance. Furthermore, in the case of medicines related to a specific disease, the determination unit can also preferentially determine medicines related to the disease. This allows the determination unit to adjust the optimal determination order based on the relevance of the medicines. A part or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the relevance data of the medicines to the generation AI, and the generation AI can analyze the data and adjust the order of determination.

[0088] The reading unit can estimate the user's emotion and adjust the timing of reading the prescription sticker based on the estimated emotion of the user. For example, when the user is stressed, the reading unit can adjust to read the prescription sticker at a timing when the user can relax. Also, when the user is in a hurry, the reading unit can adjust to read the prescription sticker quickly. Furthermore, when the user is relaxed, the reading unit can adjust to take time to read detailed information. This allows the reading unit to read the prescription sticker at an optimal timing according to the emotion of the user. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reading unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reading unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and adjust the timing of reading the prescription sticker.

[0089] When reading a prescription sticker, the reading unit can adjust the level of detail of the reading based on the importance of the sticker. For example, the reading unit can read in detail a sticker on which important information is written. The reading unit can also read briefly a sticker on which general information is written. Furthermore, the reading unit can quickly read a sticker on a medicine to be used in an emergency. This allows the reading unit to provide an optimal level of detail of reading depending on the importance of the sticker. Some or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input the importance data of the sticker to the generating AI, and the generating AI can analyze the data to adjust the level of detail of the reading.

[0090] The reading unit can estimate the user's emotion and determine the priority of stickers to be read based on the estimated user's emotion. For example, when the user feels anxious, the reading unit can preferentially read stickers on which information that gives a sense of security is described. In addition, when the user is excited, the reading unit can also preferentially read stickers on which information for regaining composure is described. Furthermore, when the user is relaxed, the reading unit can also preferentially read stickers on which detailed information is described. This allows the reading unit to determine the optimal priority of stickers according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reading unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reading unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and determine the priority of stickers to be read.

[0091] When reading the prescription sticker, the reading unit can determine the priority of reading based on the submission time of the sticker. For example, the reading unit can give priority to reading a sticker that has been submitted recently. In addition, the reading unit can also postpone reading a sticker that has been used for a long time. Furthermore, the reading unit can give top priority to reading a sticker for a medicine to be used in an emergency. This allows the reading unit to determine the optimal reading order based on the submission time of the sticker. A part or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input sticker submission time data to the generating AI, and the generating AI can analyze the data to determine the priority of reading.

[0092] The identification unit can estimate the user's emotion and adjust the method of identifying the period of use and the expiration date of the medicine based on the estimated emotion of the user. For example, when the user feels anxious, the identification unit can apply a method of providing a detailed explanation. Furthermore, when the user is relaxed, the identification unit can also apply a method of providing a concise explanation. Furthermore, when the user is in a hurry, the identification unit can apply a method of providing a quick identification result. This allows the identification unit to apply an optimal identification method according to the emotion of the user. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the identification unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and adjust the method of identifying the period of use and the expiration date of the medicine.

[0093] The identification unit can adjust the level of detail of the identification based on the importance of the drug when identifying the use period and expiration date of the drug. For example, the identification unit can provide a detailed identification result in the case of an important drug. The identification unit can also provide a concise identification result in the case of a general drug. Furthermore, the identification unit can quickly provide an identification result in the case of a drug used in an emergency. This allows the identification unit to provide an optimal identification result according to the importance of the drug. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input the importance data of the drug to the generation AI, and the generation AI can analyze the data to adjust the level of detail of the identification.

[0094] The identification unit can apply different identification algorithms depending on the category of the drug when identifying the use period and expiration date of the drug. For example, in the case of an antibiotic, the identification unit can apply a specific identification algorithm. In addition, in the case of a painkiller, the identification unit can apply another identification algorithm. Furthermore, in the case of a vitamin supplement, the identification unit can apply a further different identification algorithm. This allows the identification unit to apply an optimal identification algorithm depending on the category of the drug. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input drug category data to the generation AI, and the generation AI can analyze the data and apply different identification algorithms.

[0095] The identification unit can estimate the user's emotion and adjust the display method of the identification result based on the estimated user's emotion. For example, when the user is nervous, the identification unit can provide a simple and highly visible display method. Furthermore, when the user is relaxed, the identification unit can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the identification unit can also provide a display method that focuses on the main points. This allows the identification unit to provide an optimal display method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the identification unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and adjust the display method of the identification result.

[0096] The identification unit can determine the identification priority based on the submission time of the medicine when identifying the use period and expiration date of the medicine. For example, the identification unit can prioritize identification in the case of a recently submitted medicine. In addition, the identification unit can also postpone identification in the case of a medicine that has been used for a long time. Furthermore, the identification unit can give top priority to identification in the case of a medicine used in an emergency. This allows the identification unit to determine the optimal identification order based on the submission time of the medicine. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input data on the submission time of the medicine to the generation AI, and the generation AI can analyze the data to determine the identification priority.

[0097] The identification unit can adjust the identification order based on the relevance of the medicines when identifying the use period and expiration date of the medicine. For example, in the case of medicines in the same category, the identification unit can identify the medicines in order of high relevance. In addition, in the case of medicines in different categories, the identification unit can identify the medicines in order of low relevance. Furthermore, in the case of medicines related to a specific disease, the identification unit can preferentially identify the medicines related to the disease. This allows the identification unit to adjust the optimal identification order based on the relevance of the medicines. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input the relevance data of the medicines to the generation AI, and the generation AI can analyze the data and adjust the identification order.

[0098] The providing unit can estimate the user's emotion and adjust the expression method of the information to be provided based on the estimated user's emotion. For example, when the user feels anxious, the providing unit can provide an expression method that gives a sense of security. In addition, when the user is relaxed, the providing unit can also provide an expression method including detailed information. Furthermore, when the user is in a hurry, the providing unit can also provide a concise and quick expression method. This allows the providing unit to provide an optimal expression method of information according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the providing unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the providing unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and adjust the expression method of the information to be provided.

[0099] The providing unit can adjust the level of detail of the information provided based on the importance of the information when providing the information. For example, the providing unit can provide a detailed explanation in the case of important information. In addition, the providing unit can provide a concise explanation in the case of general information. Furthermore, the providing unit can provide information quickly in the case of information required in an emergency. This allows the providing unit to provide an optimal level of detail of the information provided depending on the importance of the information. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the providing unit can input the importance data of the information to the generating AI, and the generating AI can analyze the data to adjust the level of detail of the information provided.

[0100] The providing unit can apply different providing algorithms depending on the category of information at the time of providing. For example, in the case of information on health, the providing unit can apply a specific providing algorithm. In addition, in the case of information on how to use medicine, the providing unit can apply another providing algorithm. Furthermore, in the case of information on food combinations, the providing unit can apply a further different providing algorithm. This allows the providing unit to apply an optimal providing algorithm depending on the category of information. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input category data of information to the generating AI, and the generating AI can analyze the data and apply different providing algorithms.

[0101] The providing unit can estimate the user's emotion and determine the priority of information to be provided based on the estimated user's emotion. For example, when the user feels anxious, the providing unit can provide information that gives a sense of security preferentially. In addition, when the user is excited, the providing unit can also provide information to help the user regain his / her composure preferentially. Furthermore, when the user is relaxed, the providing unit can also provide detailed information preferentially. This allows the providing unit to determine the priority of optimal information according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the providing unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and determine the priority of the information to be provided.

[0102] The providing unit can determine the priority of the provision based on the time of submission of the information at the time of provision. For example, the providing unit can provide information that has been submitted recently with priority. In addition, the providing unit can postpone the provision of information that has been used for a long time. Furthermore, the providing unit can provide information that is necessary in an emergency with the highest priority. This allows the providing unit to determine the optimal order of provision based on the time of submission of the information. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the submission time data of the information to the generating AI, and the generating AI can analyze the data to determine the priority of the provision.

[0103] The providing unit can adjust the order of provision based on the relevance of information at the time of provision. For example, in the case of information of the same category, the providing unit can provide the information in order of high relevance. In addition, in the case of information of different categories, the providing unit can also provide the information in order of low relevance. Furthermore, in the case of information related to a specific disease, the providing unit can also provide information related to the disease preferentially. Thereby, the providing unit can adjust the optimal order of provision based on the relevance of information. A part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the relevance data of information to the generating AI, and the generating AI can analyze the data and adjust the order of provision.

[0104] The management unit can estimate the user's emotions and adjust the management method of chronic disease and physical condition information based on the estimated user's emotions. For example, when the user feels anxious, the management unit can provide a management method that gives a sense of security. Furthermore, when the user is relaxed, the management unit can also provide a management method that includes detailed information. Furthermore, when the user is in a hurry, the management unit can also provide a simple and quick management method. This allows the management unit to provide an optimal management method according to the user's emotions. The estimation of emotions is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the management unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the management unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotions and adjust the management method of chronic disease and physical condition information.

[0105] The management unit can adjust the level of detail of management based on the importance of information when managing chronic disease and physical condition information. For example, the management unit can manage important information in detail. The management unit can also manage general information simply. Furthermore, the management unit can quickly manage information required in an emergency. This allows the management unit to provide an optimal level of detail of management depending on the importance of the information. Some or all of the above-mentioned processing in the management unit may be performed using, for example, AI, or may be performed without using AI. For example, the management unit can input information importance data to the generation AI, and the generation AI can analyze the data and adjust the level of detail of management.

[0106] The management unit can estimate the user's emotion and determine the priority of information to be managed based on the estimated user's emotion. For example, when the user feels anxious, the management unit can preferentially manage information that gives a sense of security. In addition, when the user is excited, the management unit can also preferentially manage information to help the user regain his / her composure. Furthermore, when the user is relaxed, the management unit can also preferentially manage detailed information. This allows the management unit to determine the priority of optimal information according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the management unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the management unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and determine the priority of the information to be managed.

[0107] The management unit can determine the priority of management based on the time of information submission when managing chronic disease and physical condition information. For example, the management unit can prioritize management of recently submitted information. In addition, the management unit can postpone information that has been used for a long time. Furthermore, the management unit can manage information required in an emergency with the highest priority. This allows the management unit to determine the optimal management order based on the time of information submission. A part or all of the above-mentioned processing in the management unit may be performed using, for example, AI, or may be performed without using AI. For example, the management unit can input information submission time data to the generation AI, and the generation AI can analyze the data to determine the priority of management.

[0108] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0109] The acquisition unit can analyze the user's past drug use history and select the optimal image acquisition method. For example, the acquisition unit can suggest the optimal image acquisition method based on the type of drug used by the user in the past. The acquisition unit can also determine the priority of image acquisition for a specific drug from the user's past drug use history. Furthermore, the acquisition unit can analyze the user's past drug use history and select the most efficient image acquisition method. This allows the acquisition unit to select the optimal image acquisition method based on the user's past drug use history. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, AI, or may be performed without using AI. For example, the acquisition unit can input the user's past drug use history data to the generation AI, and the generation AI can analyze the data and select the optimal image acquisition method.

[0110] The acquisition unit can estimate the user's emotion and determine the priority of the image of the medicine to be acquired based on the estimated emotion of the user. For example, when the user feels anxious, the acquisition unit can preferentially acquire the image of the medicine that gives a sense of security. In addition, when the user is excited, the acquisition unit can also preferentially acquire the image of the medicine to regain composure. Furthermore, when the user is relaxed, the acquisition unit can also preferentially acquire the image of the medicine to provide detailed information. In this way, the acquisition unit can preferentially acquire the image of the medicine that is most suitable according to the emotion of the user. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the emotion of the user and determine the priority of the image of the medicine.

[0111] The judgment unit can adjust the level of detail of the judgment based on the importance of the medicine when judging the type and usage of the medicine. For example, in the case of an important medicine, a detailed judgment result can be provided. In addition, the judgment unit can also provide a concise judgment result in the case of a general medicine. Furthermore, in the case of a medicine to be used in an emergency, the judgment unit can quickly provide a judgment result. This allows the judgment unit to provide an optimal judgment result according to the importance of the medicine. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can input the importance data of the medicine to the generation AI, and the generation AI can analyze the data and adjust the level of detail of the judgment.

[0112] The determination unit can estimate the user's emotion and adjust the display method of the determination result based on the estimated user's emotion. For example, if the user is nervous, a simple and highly visible display method can be provided. In addition, if the user is relaxed, the determination unit can also provide a display method including detailed information. Furthermore, if the user is in a hurry, the determination unit can also provide a display method that focuses on the main points. This allows the determination unit to provide an optimal display method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the determination unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the determination unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and adjust the display method of the determination result.

[0113] When reading a prescription sticker, the reading unit can adjust the level of detail of the reading based on the importance of the sticker. For example, in the case of a sticker containing important information, the reading unit can read in detail. In addition, in the case of a sticker containing general information, the reading unit can also read it briefly. Furthermore, in the case of a sticker for medicine to be used in an emergency, the reading unit can also read it quickly. This allows the reading unit to provide an optimal level of detail of reading depending on the importance of the sticker. Some or all of the above-mentioned processing in the reading unit may be performed using, for example, AI, or may be performed without using AI. For example, the reading unit can input the importance data of the sticker to the generating AI, and the generating AI can analyze the data and adjust the level of detail of the reading.

[0114] The reading unit can estimate the user's emotion and determine the priority of stickers to be read based on the estimated user's emotion. For example, if the user feels anxious, the reading unit can preferentially read stickers with information that gives a sense of security. In addition, if the user is excited, the reading unit can also preferentially read stickers with information to regain composure. Furthermore, if the user is relaxed, the reading unit can also preferentially read stickers with detailed information. This allows the reading unit to determine the optimal priority of stickers according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reading unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the reading unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and determine the priority of stickers to be read.

[0115] The identification unit can apply different identification algorithms depending on the category of the drug when identifying the period of use and the expiration date of the drug. For example, in the case of an antibiotic, a specific identification algorithm can be applied. In addition, the identification unit can apply another identification algorithm in the case of a painkiller. Furthermore, the identification unit can apply a further different identification algorithm in the case of a vitamin supplement. This allows the identification unit to apply an optimal identification algorithm depending on the category of the drug. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input drug category data to the generation AI, and the generation AI can analyze the data and apply different identification algorithms.

[0116] The identification unit can estimate the user's emotion and adjust the display method of the identification result based on the estimated user's emotion. For example, when the user is nervous, a simple and highly visible display method can be provided. Furthermore, when the user is relaxed, the identification unit can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the identification unit can also provide a display method that focuses on the main points. In this way, the identification unit can provide an optimal display method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the identification unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and adjust the display method of the identification result.

[0117] The providing unit can adjust the level of detail of the information provided based on the importance of the information when providing the information. For example, in the case of important information, a detailed explanation can be provided. In addition, the providing unit can provide a concise explanation in the case of general information. Furthermore, the providing unit can provide information quickly in the case of information required in an emergency. This allows the providing unit to provide an optimal level of detail of the information provided depending on the importance of the information. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the providing unit can input the importance data of the information to the generating AI, and the generating AI can analyze the data to adjust the level of detail of the information provided.

[0118] The providing unit can estimate the user's emotion and determine the priority of information to be provided based on the estimated user's emotion. For example, when the user feels anxious, information that gives a sense of security can be provided preferentially. In addition, when the user is excited, the providing unit can also provide information to help the user regain his / her composure preferentially. Furthermore, when the user is relaxed, the providing unit can also provide detailed information preferentially. This allows the providing unit to determine the priority of optimal information according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the providing unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the providing unit can input the user's facial expression data to the generation AI, and the generation AI can analyze the data to estimate the user's emotion and determine the priority of the information to be provided.

[0119] The process flow of the second embodiment will be briefly described below.

[0120] Step 1: The acquisition unit acquires an image of the medicine captured by the imaging unit. For example, the acquisition unit can capture an image of the medicine using a camera and acquire the image data. Step 2: The determination unit determines the type and method of use of the medicine whose image has been acquired by the acquisition unit. For example, the determination unit can identify the type of medicine using image recognition technology and determine the method of taking the medicine. Step 3: The reading unit is a unit for reading the prescription sticker. For example, the reading unit can read the information written on the sticker using OCR technology. Step 4: The identification unit identifies the use period and expiration date of the medicine based on the label read by the reading unit. For example, the identification unit can analyze the information written on the label and identify the use period and expiration date of the medicine. Step 5: The providing unit provides the determination result by the determining unit and the identification result by the identifying unit. For example, the providing unit can provide the user with information regarding the type, usage, period of use, and expiration date of the medicine.

[0121] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0122] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0123] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0124] For example, the acquisition unit can take an image of the medicine using the camera 42 of the smart device 14 and acquire the image data. The determination unit can be realized, for example, by the specific processing unit 290 of the data processing device 12, and can identify the type of medicine and determine the method of taking it using image recognition technology. The reading unit can read the prescription sticker using the camera 42 of the smart device 14. The identification unit can be realized, for example, by the specific processing unit 290 of the data processing device 12, and can analyze the information written on the sticker and identify the use period and expiration date of the medicine. The provision unit can be realized, for example, by the control unit 46A of the smart device 14, and can provide the user with information on the type, usage, use period, and expiration date of the medicine. The correspondence between each unit and the device and control unit is not limited to the above example, and various changes are possible.

[0125] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0126] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0127] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0128] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0129] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0130] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0131] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0132] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0133] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0134] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0135] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0136] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0137] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0138] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0139] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0140] For example, the acquisition unit can take an image of the medicine using the camera 42 of the smart glasses 214 and acquire the image data. The determination unit can be realized, for example, by the specific processing unit 290 of the data processing device 12, and can identify the type of medicine and determine the method of taking it using image recognition technology. The reading unit can read the prescription sticker using the camera 42 of the smart glasses 214. The identification unit can be realized, for example, by the specific processing unit 290 of the data processing device 12, and can analyze the information written on the sticker and identify the use period and expiration date of the medicine. The provision unit can be realized, for example, by the control unit 46A of the smart glasses 214, and can provide the user with information on the type, usage, use period, and expiration date of the medicine. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0141] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0142] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0143] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0144] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0145] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0146] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0147] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0148] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0149] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0150] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0151] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0152] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0153] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0154] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0155] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0156] For example, the acquisition unit can take an image of the medicine using the camera 42 of the headset terminal 314 and acquire the image data. The determination unit can be realized, for example, by the specific processing unit 290 of the data processing device 12, and can identify the type of medicine and determine the method of taking it using image recognition technology. The reading unit can read the prescription sticker using the camera 42 of the headset terminal 314. The identification unit can be realized, for example, by the specific processing unit 290 of the data processing device 12, and can analyze the information written on the sticker and identify the use period and expiration date of the medicine. The provision unit can be realized, for example, by the control unit 46A of the headset terminal 314, and can provide the user with information on the type, usage, use period, and expiration date of the medicine. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0157] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0158] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0159] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0160] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0161] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0162] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0163] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0164] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0165] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0166] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0167] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0168] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0169] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0170] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0171] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0172] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0173] For example, the acquisition unit can take an image of the medicine using the camera 42 of the robot 414 and acquire the image data. The determination unit can be realized, for example, by the specific processing unit 290 of the data processing device 12, and can identify the type of medicine and determine the method of taking it using image recognition technology. The reading unit can read the prescription sticker using the camera 42 of the robot 414. The identification unit can be realized, for example, by the specific processing unit 290 of the data processing device 12, and can analyze the information written on the sticker and identify the use period and expiration date of the medicine. The provision unit can be realized, for example, by the control unit 46A of the robot 414, and can provide the user with information on the type, usage, use period, and expiration date of the medicine. The correspondence between each unit and the device and control unit is not limited to the above example, and various changes are possible.

[0174] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0175] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0176] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0177] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0178] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0179] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0180] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0181] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0182] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0183] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0184] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0185] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0186] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0187] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0188] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0189] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0190] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0191] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0192] (Appendix 1) an acquisition unit that acquires an image of the medicine captured by the imaging unit; a determination unit for determining a type and a method of use of the medicine about which the image is acquired by the acquisition unit; A reading unit for reading a prescription sticker; An identification unit that identifies the period of use and the expiration date of the medicine based on the label read by the reading unit; a providing unit that provides a user with a determination result by the determining unit and a determination result by the identifying unit. A system characterized by: (Appendix 2) A management unit for managing information on a chronic illness of a user and information on the current physical condition of the user is provided. 2. The system of claim 1, (Appendix 3) The providing unit is Provide the user with information requiring attention based on the determination result by the determination unit, the identification result by the identification unit, and the information managed by the management unit. 2. The system of claim 1, (Appendix 4) The providing unit is Provide the user with information about medication interactions that require caution 2. The system of claim 1, (Appendix 5) The acquisition unit is The system estimates the user's emotions and adjusts the timing of drug image acquisition based on the estimated user's emotions. 2. The system of claim 1, (Appendix 6) The acquisition unit is Analyze the user's past drug use history and select the image acquisition method 2. The system of claim 1, (Appendix 7) The acquisition unit is When acquiring drug images, they are filtered based on the user's current life situation and areas of interest. 2. The system of claim 1, (Appendix 8) The acquisition unit is Estimate the user's emotion and prioritize the images of medicines to be acquired based on the estimated emotion. 2. The system of claim 1, (Appendix 9) The acquisition unit is When acquiring images of medicines, the system prioritizes acquiring images of medicines that are highly relevant to the user based on the user's geographic location information. 2. The system of claim 1, (Appendix 10) The acquisition unit is When acquiring a drug image, the user's social media activity is analyzed to acquire related drug images. 2. The system of claim 1, (Appendix 11) The determination unit is The system estimates the user's emotions and adjusts the method of determining the type and use of medication based on the estimated user emotions. 2. The system of claim 1, (Appendix 12) The determination unit is When determining the type and use of a drug, the level of detail of the determination is adjusted based on the importance of the drug. 2. The system of claim 1, (Appendix 13) The determination unit is When determining the type and use of a drug, different algorithms are applied depending on the drug category. 2. The system of claim 1, (Appendix 14) The determination unit is To estimate a user's emotion and adjust a method of displaying a judgment result based on the estimated user's emotion. 2. The system of claim 1, (Appendix 15) The determination unit is When determining the type and use of a drug, the priority of the determination is determined based on the time of submission of the drug. 2. The system of claim 1, (Appendix 16) The determination unit is When determining the type and use of a drug, the order of determination is adjusted based on the relevance of the drug. 2. The system of claim 1, (Appendix 17) The reading unit is The system estimates the user's emotions and adjusts the timing of reading prescription stickers based on the estimated user emotions. 2. The system of claim 1, (Appendix 18) The reading unit is When reading prescription stickers, adjust the reading resolution based on the importance of the sticker 2. The system of claim 1, (Appendix 19) The reading unit is The user's emotions are estimated, and the priority of stickers to be read is determined based on the estimated user's emotions. 2. The system of claim 1, (Appendix 20) The reading unit is When reading prescription stickers, prioritize reading based on when the sticker was submitted 2. The system of claim 1, (Appendix 21) The identification unit is To estimate a user's emotion and adjust a method for identifying a period of use and an expiration date of a medicine based on the estimated user's emotion. 2. The system of claim 1, (Appendix 22) The identification unit is When identifying the duration and expiry date of a drug, adjust the level of specific detail based on the importance of the drug. 2. The system of claim 1, (Appendix 23) The identification unit is Applying different identification algorithms depending on the drug category when identifying the duration and expiry date of the drug 2. The system of claim 1, (Appendix 24) The identification unit is Inferring a user's emotion and adjusting the way certain results are displayed based on the inferred user emotion 2. The system of claim 1, (Appendix 25) The identification unit is When identifying the duration and expiration date of a drug, a specific priority is determined based on the time of submission of the drug. 2. The system of claim 1, (Appendix 26) The identification unit is Adjust the specific order based on the relevance of the drugs when identifying the duration and expiry date of the drugs 2. The system of claim 1, (Appendix 27) The providing unit is Estimate the user's emotions and adjust the way information is presented based on the estimated user's emotions 2. The system of claim 1, (Appendix 28) The providing unit is When providing information, adjust the level of detail provided based on the importance of the information 2. The system of claim 1, (Appendix 29) The providing unit is When serving, different serving algorithms are applied depending on the category of information 2. The system of claim 1, (Appendix 30) The providing unit is Estimate the user's emotions and prioritize the information to be provided based on the estimated user's emotions. 2. The system of claim 1, (Appendix 31) The providing unit is At the time of submission, the information will be prioritized based on when it was submitted. 2. The system of claim 1, (Appendix 32) The providing unit is When serving, adjust the order of serving based on the relevance of the information 2. The system of claim 1, (Appendix 33) The management unit To estimate a user's emotion and adjust a method of managing chronic disease and physical condition information based on the estimated user's emotion. 3. The system of claim 2, (Appendix 34) The management unit When managing chronic illness and health information, adjust the level of detail based on the importance of the information 3. The system of claim 2, (Appendix 35) The management unit To estimate a user's emotion and determine the priority of information to be managed based on the estimated user's emotion. 3. The system of claim 2, (Appendix 36) The management unit When managing chronic illness and health information, prioritize management based on when the information is submitted 3. The system of claim 2, [Explanation of symbols]

[0193] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. an acquisition unit that acquires an image of the medicine captured by the imaging unit; a determination unit for determining a type and a method of use of the medicine about which the image is acquired by the acquisition unit; A reading unit for reading a prescription sticker; An identification unit that identifies a period of use and an expiration date of a medicine based on the label read by the reading unit; a providing unit that provides a user with a determination result by the determining unit and a determination result by the identifying unit. A system characterized by:

2. A management unit for managing information on the chronic illness of the user and information on the current physical condition of the user 2. The system of claim 1 .

3. The providing unit is Provide the user with information requiring attention based on the determination result by the determination unit, the identification result by the identification unit, and the information managed by the management unit.

3. The system of claim 2.

4. The providing unit is Provide the user with information on medication interactions that require caution and related information 2. The system of claim 1 .

5. The acquisition unit is The emotion of the user is estimated, and a timing for acquiring an image of the medicine is adjusted based on the estimated emotion of the user.

2. The system of claim 1 .

6. The acquisition unit is Analyzing the user's past drug use history and selecting an image acquisition method.

2. The system of claim 1 .

7. The acquisition unit is When acquiring images of medicines, filtering is performed based on the user's current living situation and areas of interest.

2. The system of claim 1 .

8. The acquisition unit is Estimating the user's emotion and determining a priority order of images of medicines to be acquired based on the estimated emotion of the user.

2. The system of claim 1 .

9. The acquisition unit is When acquiring images of medicines, images of medicines having high relevance are acquired preferentially in consideration of geographical location information of the user.

2. The system of claim 1 .

10. The acquisition unit is Upon acquiring a drug image, the user's social media activity is analyzed to acquire related drug images.

2. The system of claim 1 .

Images