Operation method of system for acquiring diagnosis information on medical image existing within field of view of user wearing wearable device

Abstract

The present disclosure relates to a method of operation of a system. The method of operation comprises the steps of: a wearable device capturing at least one image through a camera included in the wearable device; the wearable device inputting the image into at least one first artificial intelligence model for recognizing a medical image in which at least a part of the body is captured, and identifying whether a medical image is recognized within the image based on the output data; if it is identified that a medical image is recognized within the image, the wearable device transmitting the identified medical image to a server; and the server inputting the identified medical image into at least one second artificial intelligence model for generating diagnostic information, and obtaining diagnostic information of a subject corresponding to the identified medical image based on the output data.

Description

Technology Field

[0001] The present disclosure relates to a method of operating a system, and more specifically, to a system capable of recognizing a medical image existing in the user's field of vision in real time based on an image captured through a wearable device, transmitting it to a server, and obtaining diagnostic information from the server. Background Technology

[0002] Recently, due to technological advancements, the application of devices such as smart glasses in medical environments is gradually increasing. Smart glasses can be used to provide information in real time within the medical staff's field of vision or to record surgical and procedural processes through video recording functions.

[0003] However, generally, video recording or observation through smart glasses is often limited to passive recording, and the ability to analyze or judge medical information contained in the video in real time has been limited. Typically, medical staff have been able to diagnose by checking the recorded video in real time through a monitor or by saving the data for post-analysis.

[0004] Therefore, there has recently been active development of technologies that assist in diagnosis by automatically interpreting medical images through AI-based image analysis models. However, since these analysis functions are primarily performed on in-hospital servers or fixed devices, there is a need to propose a real-time analysis system integrated with devices worn by medical staff. Prior art literature

[0005] Published Patent Application No. 10-2024-0052597 The problem to be solved

[0006] Through the present disclosure, we aim to provide a system capable of recognizing medical images present in the user's field of vision in real time based on images captured by a wearable device, transmitting them to a server, and obtaining diagnostic information from the server.

[0007] The purposes of the present disclosure are not limited to those mentioned above, and other purposes and advantages of the present disclosure not mentioned may be understood from the following description and will be more clearly understood from the embodiments of the present disclosure. Furthermore, it will be readily apparent that the purposes and advantages of the present disclosure can be realized by the means and combinations thereof set forth in the claims.

[0008] Support Project: Mighty Hallym 4.0 Vision

[0009] Sponsoring Organization: Ilsong Academy

[0010] Implementing Organization: Medical Center Industry-Academic Cooperation Foundation

[0011] Major Project: Research on Chatbots and Benchmarks for Digestive Disease Medical History Questionnaires and Patient Counseling means of solving the problem

[0012] A method of operation of a system according to one embodiment of the present disclosure comprises: a step in which a wearable device captures at least one image through a camera included in the wearable device; a step in which the wearable device inputs the image into at least one first artificial intelligence model for recognizing a medical image in which at least a part of the body is captured, and identifies whether a medical image is recognized within the image based on the output data; a step in which, if it is identified that a medical image is recognized within the image, the wearable device transmits the identified medical image to a server; and a step in which the server inputs the identified medical image into at least one second artificial intelligence model for generating diagnostic information, and obtains diagnostic information of a subject corresponding to the identified medical image based on the output data.

[0013] At this time, the step of capturing at least one image through a camera included in the wearable device may involve the wearable device acquiring at least one image captured through a camera formed to capture at least one image within the field of view of a user wearing the wearable device.

[0014] At this time, the step of identifying whether a medical image is recognized within the image can be performed by the wearable device inputting a plurality of images sequentially captured by the camera into the first artificial intelligence model and identifying whether a medical image is recognized from at least one of the plurality of images based on the output data.

[0015] At this time, the step of transmitting the identified medical image to the server may be such that the wearable device identifies communication status information based on the packet transmission and reception speed between the server and the wearable device, the wearable device sets the number of medical images that can be transmitted per unit time based on the identified communication status information, the wearable device selects at least one of the identified medical images based on the set number of medical images that can be transmitted, and transmits the selected medical image to the server.

[0016] At this time, the step of transmitting the identified medical images to the server is such that when the number of medical images identified per unit time exceeds the set number of transmittable images, the wearable device sets the last medical image transmitted to the server as a reference medical image, the wearable device identifies the similarity between the reference medical image and the identified medical images, the wearable device selects medical images among the identified medical images in which the identified similarity is less than a threshold within the set number of transmittable images, and transmits the selected medical images to the server.

[0017] Meanwhile, the step of obtaining diagnostic information of a subject corresponding to the identified medical image can be performed by the server inputting the identified medical image into the second artificial intelligence model to obtain the diagnostic information including at least one of an area where a lesion is identified on the identified medical image and a type of lesion.

[0018] At this time, the method of operation of the above system may include the step of the server transmitting the diagnostic information to the wearable device and the step of the wearable device providing the diagnostic information to the user.

[0019] At this time, the step of providing the diagnostic information to the user may involve the wearable device identifying a bounding box containing a lesion area that includes at least one lesion on a medical image identified from a first image taken at a first time point based on the diagnostic information, and outputting the bounding box through an optical display included in the wearable device.

[0020] At this time, the step of providing the diagnostic information to the user is such that, when an area corresponding to the lesion area is identified on a medical image identified from a second image captured by the camera at a second time point after the first time point, the wearable device identifies a location where the bounding box is displayed on the optical display based on the location of the area corresponding to the lesion area on the second image, and outputs the bounding box according to the identified location.

[0021] A method of operation of a wearable device according to one embodiment of the present disclosure comprises: a step of capturing at least one image through a camera formed to capture at least one image within a field of view angle range set based on the direction of an optical display formed to be located within the field of view of a user wearing the wearable device; a step of identifying whether a medical image is recognized within the image based on data output by inputting the image into at least one first artificial intelligence model for recognizing a medical image in which at least a part of the body is captured; and a step of transmitting the identified medical image to a server and obtaining diagnostic information of a subject corresponding to the identified medical image from the server when it is identified that a medical image is recognized within the image.

[0022] At this time, the step of obtaining diagnostic information of a subject corresponding to the identified medical image from the server may be performed by inputting the identified medical image into at least one second artificial intelligence model stored in the server, which is trained to generate diagnostic information, and obtaining the diagnostic information based on the output data.

[0023] A non-transient computer-readable medium according to one embodiment of the present disclosure stores at least one instruction that is executed by at least one electronic device and causes said electronic device to perform the method of operation of claim 10. Effects of the invention

[0024] According to the present invention, a user can capture an image corresponding to the user's field of vision in real time without any separate operation using only a wearable device, and automatically determine whether the medical image is recognized through artificial intelligence-based analysis, thereby collecting data suitable for generating diagnostic information.

[0025] In addition, as diagnostic information generated by analyzing medical images recognized by the wearable device through an artificial intelligence model stored on a server is transmitted to the wearable device, the diagnostic information is provided visually (e.g., bounding box) in real time through the wearable device, thereby allowing medical staff to quickly recognize the location of the lesion and improve diagnostic efficiency. Brief explanation of the drawing

[0026] FIG. 1 is a drawing illustrating the configuration of a system according to one embodiment of the present disclosure. FIG. 2 is a block diagram illustrating the configuration of a wearable device according to one embodiment of the present disclosure. FIG. 3 is a block diagram illustrating the configuration of a server according to one embodiment of the present disclosure. FIG. 4 is a diagram illustrating the operation of a wearable device providing diagnostic information to a user according to one embodiment of the present disclosure. FIG. 5 is a diagram illustrating the configuration of an optical display of a wearable device according to one embodiment of the present disclosure. FIG. 6 is a flowchart illustrating the operation of a wearable device according to one embodiment of the present disclosure. Specific details for implementing the invention

[0027] Before specifically describing the present disclosure, the method of description in the specification and drawings is described.

[0028] First, the terms used in this specification and claims have been selected based on general terms considering their functions in the various embodiments of this disclosure. However, these terms may vary depending on the intent of those skilled in the art, legal or technical interpretations, and the emergence of new technologies. Additionally, some terms have been arbitrarily selected by the applicant. Such terms may be interpreted according to the meanings defined in this specification; in the absence of specific definitions, they may be interpreted based on the overall content of this specification and common technical knowledge in the relevant field.

[0029] In addition, the same reference numbers or symbols described in each drawing attached to this specification represent parts or components that perform substantially the same function. For convenience of explanation and understanding, the same reference numbers or symbols are used to describe different embodiments. That is, even if components having the same reference number are all depicted in multiple drawings, the multiple drawings do not imply a single embodiment.

[0030] Additionally, in this specification and claims, terms including ordinal numbers, such as "first," "second," etc., may be used to distinguish between components. These ordinal numbers are used to distinguish identical or similar components from one another, and the meaning of the terms should not be limited by the use of such ordinal numbers. For example, the order of use or arrangement of components combined with such ordinal numbers should not be restricted by the number. If necessary, each ordinal number may be used interchangeably.

[0031] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "consisting of" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0032] In the embodiments of the present disclosure, terms such as "module," "unit," "part," etc. are used to refer to a component that performs at least one function or operation, and such component may be implemented in hardware or software, or in a combination of hardware and software. Additionally, a plurality of "modules," "units," "parts," etc. may be integrated into at least one module or chip and implemented as at least one processor, except where each needs to be implemented in specific individual hardware.

[0033] Furthermore, in the embodiments of the present disclosure, when a part is described as being connected to another part, this includes not only a direct connection but also an indirect connection through another medium. Additionally, the meaning that a part includes a certain component implies that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0034] In the present disclosure, the expression “at least one of a, b, or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “a, b, and c all”, or variations thereof.

[0035] The terms used in this disclosure have been selected to be as widely used and general as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant explanatory sections. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0036] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art as described in this specification. Additionally, terms including ordinal numbers, such as "first" or "second," used in this specification may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another.

[0037] When a part of a specification is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "part" or "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0038] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present disclosure in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0039] FIG. 1 is a drawing illustrating the configuration of a system according to one embodiment of the present disclosure.

[0040] Referring to FIG. 1, the system (1000) may include a wearable device (100) and a server (200).

[0041] In one embodiment, when a medical image is identified in the user's field of vision through a wearable device (100), the system (1000) can generate diagnostic information about a body part included in the medical image through a server (200).

[0042] Specifically, the wearable device (100) can capture at least one image through a camera included in the wearable device (100), and input the captured image into at least one first artificial intelligence model (10) for recognizing a medical image in which at least a part of the body is captured, and identify whether a medical image is recognized within the image based on the output data.

[0043] The first artificial intelligence model (10) may be a model for recognizing medical images in which a specific body part is captured according to a specific shooting method.

[0044] For example, the first artificial intelligence model (10) may be a model trained to recognize a medical image of a target body part consisting of at least a part of the gastrointestinal tract (e.g., esophagus, stomach, duodenum, etc.) captured by a target imaging method (e.g., endoscopy) that invades the body and captures at least a part of the body.

[0045] To this end, the first artificial intelligence model (10) may be trained based on training images in which each of the various body parts, including the target body part, is captured using various shooting methods. For example, the first artificial intelligence model (10) may be trained based on training data in which label data including the body part included in the training image and the shooting method of the training image is matched for each training image, so that it can identify whether the target body part includes a medical image captured using the target shooting method from an image input to the first artificial intelligence model (10).

[0046] That is, the first artificial intelligence model (10) can be trained not to determine whether the input image corresponds to a medical image in which the target body part is captured by a target-capturing method, but to determine whether a medical image in which the target body part is captured by a target-capturing method is recognized on the input image (whether the image contains a medical image in which the target body part is captured by a target-capturing method).

[0047] Additionally, the first artificial intelligence model (10) may be trained to extract and output the medical image included in the input image when a medical image is recognized on the input image.

[0048] In this case, the wearable device (100) can input at least one image into the first artificial intelligence model (10) and, based on the output data, determine whether the image input into the first artificial intelligence model (10) is a medical image and obtain at least one of the medical images included in the image input into the first artificial intelligence model (10).

[0049] Specifically, if the image input to the first artificial intelligence model (10) includes a medical image, the wearable device (100) can acquire the medical image (included in the image input to the first artificial intelligence model (10)) based on the output data of the first artificial intelligence model (10), and if the image input to the first artificial intelligence model (10) does not include a medical image, the wearable device (100) can identify that the medical image was not recognized (from the image input to the first artificial intelligence model (10)) based on the output data of the first artificial intelligence model (10).

[0050] Meanwhile, the first artificial intelligence model (10) may be a model based on a CNN (Convolutional Neural Network), or a model based on at least one of various learning algorithms such as an RNN (Recurrent Neural Network), a Random Forest, a Support Vector Machine, a Decision Tree, a DNN (Deep Neural Network), an LSTM (Long Short-Term Memory), or a Transformer, but is not limited thereto.

[0051] At this time, the wearable device (100) may have obtained and stored a first artificial intelligence model (10) that has been pre-trained from an external source, or it may obtain the first artificial intelligence model (10) by training it.

[0052] If it is identified that a medical image is recognized within an image, the wearable device (100) can transmit the medical image recognized within the image to a server (200).

[0053] Here, the wearable device (100) can set the number of medical images that can be transmitted per unit time to the server (200) according to the communication status between the wearable device (100) and the server (200).

[0054] To this end, the wearable device (100) can obtain communication status information by estimating the communication status based on the packet (or data) transmission and reception speed between the wearable device (100) and the server (200).

[0055] Here, the communication status may correspond to any one of the status types such as good, normal, or congested, and the communication status information may include, but is not limited to, round-trip time, transmission speed, communication status, rate of change of transmission speed, (described later) integrated transmission speed, rate of change of integrated transmission speed, etc.

[0056] For example, a wearable device (100) can identify the round-trip time based on the time when the wearable device (100) sends a ping message to a server (200) to measure the round-trip time (RTT) and the time when a response signal for the ping message is received from the server (200), and can estimate the communication state based on this.

[0057] For example, the wearable device (100) may estimate that the communication status is good as the round-trip time is short, and may estimate that the communication status is good if the round-trip time is less than or equal to a preset reference time (e.g., 2ms, 10ms), and may estimate that the communication status is congested if it exceeds the reference time.

[0058] Additionally, the wearable device (100) may estimate the transmission speed based on the round-trip time and a preset window size. Here, the window size is a value that is received in advance from the server (200) and set, and represents the maximum data size that the server (200) can receive from the wearable device (100) at once.

[0059] Specifically, the wearable device (100) can estimate the transmission speed by dividing the window size by the round-trip time.

[0060] Additionally, the wearable device (100) may estimate the transmission speed between the wearable device (100) and the server (200) during the process of transmitting a medical image (recognized through the first artificial intelligence model (100)) to the server (200).

[0061] Specifically, when the reception of a medical image transmitted from a wearable device (100) is completed, the server (200) transmits a reception completion signal to the wearable device (100). Accordingly, the wearable device (100) can identify the time taken for transmission based on the time when the transmission of the medical image to the server (200) began and the time when the reception completion signal was received from the server (200), and can identify the transmission speed based on the size of the data transmitted to the server (200) (: size of the medical image (byte)) and the identified time taken for transmission.

[0062] Here, when multiple medical images are transmitted to a server (200) at once, the wearable device (100) can identify the sum of the sizes of each of the multiple medical images transmitted to the server (200) at once as the size of the data transmitted to the server (200).

[0063] At this time, the wearable device (100) may identify an integrated transmission speed based on the transmission speed estimated based on the round-trip time and the transmission speed estimated based on the time taken for transmission according to a certain period (e.g., 1 second, 5 seconds, 10 seconds, etc.), and may also identify the communication status between the wearable device (100) and the server (200) based on the integrated transmission speed.

[0064] In this case, the wearable device (100) can identify the transmission speed estimated based on the round-trip time as the first transmission speed and the transmission speed estimated based on the time taken for transmission as the second transmission speed.

[0065] Specifically, the wearable device (100) can identify the average value of the first transmission speed and the second transmission speed as the integrated transmission speed if the ratio of the difference value between the first transmission speed and the second transmission speed to the first transmission speed is less than a preset value (e.g., 20%).

[0066] That is, the wearable device (100) can determine the relative error regardless of the magnitude of the transmission speed itself by using the ratio of the difference between the transmission speeds to the first transmission speed, rather than the absolute difference between the transmission speeds, and thereby can apply a consistent standard to high-speed communication environments and low-speed communication environments.

[0067] If the ratio of the difference value between the first transmission speed and the second transmission speed to the first transmission speed is greater than or equal to a preset value, the wearable device (100) can apply a weight to each of the first transmission speed and the second transmission speed, and then sum the first transmission speed and the second transmission speed to which the weights have been applied to calculate an integrated transmission speed.

[0068] Here, the wearable device (100) may set weights applied to each of the first transmission speed and the second transmission speed based on the size of the data transmitted to the server (200), and may set weights such that the sum of the weights maintains a constant value.

[0069] Specifically, the wearable device (100) can set the weight applied to the second transmission speed to be larger (than the weight applied to the first transmission speed) as the size of the data transmitted to the server (200) increases (judging that as the time required for data transmission increases, the influence of instantaneous delay on the transmission speed decreases and the influence of the actual network condition increases).

[0070] Through this, the wearable device (100) can calculate the integrated transmission speed by taking into account the round-trip time and the size of the data (transmitted to the server), thereby identifying the integrated transmission speed that comprehensively reflects the actual network processing capability and environmental changes without relying on simple delay time or a single standard.

[0071] Meanwhile, the wearable device (100) can estimate the communication status between the wearable device (100) and the server (200) based on the integrated transmission speed.

[0072] For example, if the rate of change of the integrated transmission speed identified according to a certain period exceeds a preset first reference value (e.g., 30%), the wearable device (100) identifies the communication status between the wearable device (100) and the server (200) as congested; if the rate of change of the integrated transmission speed is less than or equal to a preset second reference value (e.g., 10%), the communication status is identified as good; and if the rate of change of the integrated transmission speed exceeds the second reference value and is less than or equal to the first reference value, the communication status is identified as normal.

[0073] Here, the rate of change of the integrated transmission speed is a value regarding the degree of change in the integrated transmission speed, and refers to a value representing the ratio of the difference between the subsequently identified integrated transmission speed and the first identified integrated transmission speed among the continuously identified integrated transmission speeds.

[0074] At this time, if the communication status between the wearable device (100) and the server (200) is identified as congested, the wearable device (100) can set the number of medical images that can be transmitted per unit time to the server (200), and can select and transmit medical images according to the set number of medical images.

[0075] In one embodiment, the wearable device (100) can set the number of possible transmissions based on a value calculated by applying a correction value according to the rate of change of the integrated transmission speed to the maximum number of transmissions.

[0076] Here, the maximum number of transmissions can be set according to the window size received in advance from the server (200) and the average size of the medical image identified from the image captured by the camera (130).

[0077] Specifically, the wearable device (100) can identify the value calculated by dividing the window size by the average size of the medical image as the maximum number of transmissions.

[0078] At this time, the wearable device (100) may set a reduction value according to the rate of change of the transmission speed, and the degree to which the reduction value is set is greater as the rate of change of the transmission speed increases, and accordingly, the number of possible transmissions may be set to be smaller as the rate of change of the transmission speed increases.

[0079] Through this, the wearable device (100) can minimize the possibility of data transmission failure or loss by adjusting the number of medical images that can be transmitted according to network conditions.

[0080] Meanwhile, if the number of medical images identified per unit time (through the first artificial intelligence model (10)) exceeds the number of transmissionable images set above, the wearable device (100) can select at least one of the newly identified medical images based on the similarity between the medical image last transmitted to the server (200) and the newly identified medical image, and transmit the selected medical image to the server (200).

[0081] To this end, the wearable device (100) can set the last medical image transmitted to the server (200) as a reference medical image and identify the similarity between the reference medical image and the medical image identified (through the first artificial intelligence model (10)).

[0082] For example, a wearable device (100) can extract at least one feature vector from an image based on various algorithms such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded-Up Robust Features). In this case, the wearable device (100) can calculate the distance between feature vectors extracted from different images and identify a higher similarity as the calculated distance is shorter, and can identify the similarity between different images based on cosine similarity using feature vectors extracted from images.

[0083] At this time, the wearable device (100) can select medical images within the set number of transmittable images among the medical images identified (through the first artificial intelligence model (10)) whose similarity to a reference medical image is less than a threshold, and transmit the selected medical images to the server (200). At this time, the reference medical image is changed to a medical image identified from the image taken most recently among the selected medical images.

[0084] Specifically, the wearable device (100) selects a medical image among the medical images identified from each of the plurality of images sequentially captured by the camera (130) that has a similarity to a reference medical image that is less than a threshold, and if the number of medical images that have a similarity to a reference medical image that is less than the threshold exceeds the number of transmissionable images set above, it may select a medical image identified from the first captured image.

[0085] Meanwhile, if it is identified that the communication status between the wearable device (100) and the server (200) is not congested according to the communication status information, the wearable device (100) may not set the number of transmissions possible per unit time.

[0086] When a medical image is received from a wearable device (100), the server (200) can input the medical image received from the wearable device (100) into at least one second artificial intelligence model (20) for generating diagnostic information, and based on the output data, obtain diagnostic information of a subject corresponding to the medical image received from the wearable device (100).

[0087] The second artificial intelligence model (20) may be a model for generating diagnostic information about a specific body part.

[0088] For example, the second artificial intelligence model (20) may be a model trained to generate diagnostic information for a target body part including at least a part of the gastrointestinal tract, including at least one of a region where a lesion (e.g., cell tissue deformation, tumor, etc.) is identified in an image and at least one of the types of lesions identified in an image (e.g., normal, atrophy, intestinal metaplasia, dysplasia, gastric cancer, etc.).

[0089] To this end, the second artificial intelligence model (20) can be trained based on training data consisting of a plurality of training images in which a target body part is photographed and labeling data including at least one of whether a lesion is included, the location of the lesion, and the type of the lesion for each of the plurality of training images.

[0090] Accordingly, the wearable device (100) can obtain diagnostic information including at least one of the area where a lesion is identified in the medical image and the type of lesion identified in the medical image, based on the output data obtained by inputting a medical image of a target body part to the second artificial intelligence model (20).

[0091] Meanwhile, the second artificial intelligence model (20) may be a model based on a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Random Forest, a Support Vector Machine, a Decision Tree, a Deep Neural Network (DNN), a Long Short-Term Memory (LSTM), a Transformer, or at least one of various learning algorithms, but is not limited thereto.

[0092] At this time, the server (200) may have obtained and stored a pre-trained second artificial intelligence model (20) from an external source, or it may obtain the second artificial intelligence model (20) by training it.

[0093] Meanwhile, the server (200) can transmit diagnostic information obtained through the second artificial intelligence model (20) to the wearable device (100).

[0094] Accordingly, the wearable device (100) can provide diagnostic information received from the server (200) to the user.

[0095] As an additional embodiment, the wearable device (100) may activate a mode related to a method of obtaining diagnostic information based on the data processing speed of the wearable device (100).

[0096] For example, a mode related to a method for obtaining diagnostic information may include a first mode of transmitting a plurality of medical images identified through a first artificial intelligence model (10) to a server (200) and obtaining diagnostic information for each of the plurality of medical images from the server (200), a second mode of transmitting at least some of the plurality of medical images identified through the first artificial intelligence model (10) to a server (200) and obtaining diagnostic information from the server (200), a third mode of transmitting at least one medical image to a server (200) and obtaining diagnostic information from the server (200), etc.

[0097] In this case, the server (200) may additionally include an artificial intelligence model for identifying at least one medical image from an image, and the wearable device (100) may transmit at least one image captured through the camera (130) to the server (200) and receive information about a lesion included in the image from the server (200). At this time, the artificial intelligence model stored in the server (200) and the first artificial intelligence model (10) may be models having the same structure, but are not limited thereto.

[0098] In one embodiment, the wearable device (100) basically acquires diagnostic information according to the first mode, and can identify whether the second mode or the third mode is activated based on the data processing speed of the wearable device (100) and the transmission speed of the wearable device (100).

[0099] To this end, the wearable device (100) can identify the data processing speed based on the number of images processed per unit time. Here, the images processed by the wearable device (100) include images used to identify the presence or absence of medical images.

[0100] That is, the data processing speed of the wearable device (100) refers to the speed related to the operation of the wearable device (100) identifying a medical image from an image captured by the camera (130) through the first artificial intelligence model (10).

[0101] At this time, if the data processing speed of the wearable device (100) is greater than or equal to a threshold speed (e.g., 20 items per unit time), the wearable device (100) can identify whether the second mode is activated according to the communication status between the wearable device (100) and the server (200).

[0102] Specifically, the wearable device (100) may not activate the second mode when the communication status is good, and may activate the second mode when the communication status is normal or congested.

[0103] When the second mode is activated, the wearable device (100) can set the medical image in which the diagnostic information containing the lesion area was last identified among the diagnostic information received from the server (200) as the reference image.

[0104] At this time, the wearable device (100) selects a medical image among a plurality of medical images identified after a reference image is set that has a similarity to the reference image below a threshold, transmits the selected medical image to a server (200), and obtains diagnostic information from the server (200). In this case, the reference image is changed to the selected medical image.

[0105] That is, the wearable device (100) can selectively transmit some of the multiple medical images identified through the first artificial intelligence model (10) to the server (200) to reduce the amount of communication, and obtain diagnostic information from the server (200).

[0106] Meanwhile, the wearable device (100) can identify whether the third mode is activated if the data processing speed of the wearable device (100) is less than a threshold speed (e.g., 20 items per unit time).

[0107] For example, the wearable device (100) may not activate the third mode if the communication status between the wearable device (100) and the server (200) is normal or congested, and may activate the third mode if the communication status between the device (100) and the server (200) is good.

[0108] That is, when the wearable device (100) transmits the image itself captured through the camera (130) to the server (200) to reduce the computational load of the wearable device (100), the server (200) identifies a medical image from the image captured through the camera (130) and identifies at least one lesion area from the identified medical image to generate diagnostic information, so that the wearable device (100) can obtain diagnostic information from the server (200).

[0109] Through this, the wearable device (100) can minimize the delay in diagnostic processing and ensure the continuity of diagnostic information acquisition by implementing an adaptive structure that allows the wearable device (100) to select an appropriate diagnostic acquisition path depending on the situation by switching the mode of the method of acquiring diagnostic information considering the computational processing capability of the wearable device (100) and the communication status with the server (200).

[0110] FIG. 2 is a block diagram illustrating the configuration of a wearable device according to one embodiment of the present disclosure.

[0111] Referring to FIG. 2, the wearable device (100) may include a memory (110), an optical display (120), a camera (130), a processor (140), and a communication interface (150).

[0112] The memory (110) stores various programs or data temporarily or non-temporarily and transmits the stored information to the processor upon the call of the processor (120). Additionally, the memory (110) can store various information required for the operation, processing, or control operation of the processor (120) in an electronic format.

[0113] The memory (110) may include, for example, at least one of a main memory and an auxiliary memory. The main memory may be implemented using a semiconductor storage medium such as ROM and / or RAM.

[0114] In one embodiment, the memory (110) may include at least one first artificial intelligence model (10) for recognizing a medical image in which at least a part of the body is captured.

[0115] The optical display (120) is configured to visually output various images or information.

[0116] In one embodiment, the optical display (120) may be formed to be located within the field of view of a user wearing the wearable device (100).

[0117] Accordingly, the optical display (120) can output various images or information under the control of the processor (140) and provide them to a user wearing the wearable device (100).

[0118] For example, the optical display (120) can visually output information based on at least one of various optical structures, such as an optical waveguide type, a pin mirror type, or a holographic type.

[0119] The camera (130) is configured to take at least one image (or image frame).

[0120] In one embodiment, the camera (130) may be formed to capture at least one image (or image frame) within a viewing angle range set based on the direction of the optical display (120) (e.g., horizontal viewing angle range -100° to 100°, vertical viewing angle range -70° to 70°, etc.).

[0121] For example, the camera (130) may include a lens that refracts and collects or disperses one or more lights (e.g., a convex lens, a concave lens, a spherical lens, a flat lens, a wide-angle lens, etc.), an image sensor that converts light into an electric charge to acquire an image (e.g., a CCD (Charge-Coupled Device), a CMOS (Complementary Metal-Oxide Semiconductor)), an image signal processor, or a flash. In addition, the camera (130) may include an aperture, a viewfinder, a zebra device that detects whether the image is overexposed through a CCD inside the camera, etc. The wearable device (100) can acquire an RGB image by sensing light in the visible light region through the camera (130). The camera (130) can acquire an infrared image by sensing light in the infrared region. However, it is not limited thereto, and the wearable device (100) can acquire an image by sensing light of various wavelengths through the camera (130).

[0122] Additionally, the camera (130) may be implemented as a stereo camera including a plurality of lenses positioned at regular intervals.

[0123] In this case, the camera (130) can perform image matching by comparing images simultaneously captured through each of the multiple lenses to identify the same object or feature included in each image, and by calculating the disparity, which is the positional difference value of the same object included in each image, to calculate distance information for each point of the image, thereby generating a depth map in which distance information is matched for each pixel of the image.

[0124] Accordingly, the processor (140) can identify whether a medical image is recognized for each of the multiple images sequentially captured by the camera (130).

[0125] The processor (140) controls the overall operation of the wearable device (100). Specifically, the processor (140) is connected to the configuration of the wearable device (100) including the memory as described above, and can control the overall operation of the wearable device (100) by executing at least one instruction stored in the memory as described above.

[0126] In one embodiment, the processor (140) inputs a plurality of images sequentially captured by the camera (130) into the first artificial intelligence model (10), and when at least one medical image is identified based on the output data, the identified medical image can be transmitted to the server (200) through the communication interface (150).

[0127] In particular, the processor (140) can be implemented as a single processor, as well as as multiple processors.

[0128] The processor (140) may be implemented in various ways. For example, one or more processors (140) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. One or more processors (140) may control one or any combination of other components of an electronic device and may perform operations or data processing related to communication. One or more processors (140) may execute one or more programs or instructions stored in memory. For example, one or more processors (140) may perform a method according to one embodiment of the present disclosure by executing one or more instructions stored in memory.

[0129] The communication interface (150) may include a wireless communication interface, a wired communication interface, or an input interface. The wireless communication interface may communicate with various external devices using wireless communication technology or mobile communication technology. Such wireless communication technologies may include, for example, Bluetooth, Bluetooth Low Energy, CAN communication, Wi-Fi, Wi-Fi Direct, ultrawide band (UWB), Zigbee, infrared data association (IrDA), or near field communication (NFC), and mobile communication technologies may include 3GPP, Wi-Max, LTE (Long Term Evolution), 5G, etc.

[0130] A wireless communication interface can be implemented using an antenna, a communication chip, a substrate, etc., capable of transmitting electromagnetic waves to the outside or receiving electromagnetic waves transmitted from the outside.

[0131] A wired communication interface can communicate with various devices based on a wired communication network. Here, the wired communication network can be implemented using physical cables, such as, for example, pair cables, coaxial cables, fiber optic cables, or Ethernet cables.

[0132] Depending on the embodiment, either the wireless communication interface or the wired communication interface may be omitted. Accordingly, the wearable device (100) may include only a wireless communication interface or only a wired communication interface. In addition, the electronic device (100) may be equipped with an integrated communication interface (150) that supports both wireless connection via the wireless communication interface and wired connection via the wired communication interface.

[0133] The wearable device (100) is not limited to having one communication interface (150) that performs a communication connection in one way, but may include a plurality of communication interfaces (150) that perform a communication connection in a plurality of ways.

[0134] Meanwhile, the wearable device (100) may be a device worn on the head by a user, and may correspond to, for example, smart glasses, but is not limited thereto.

[0135] FIG. 3 is a block diagram illustrating the configuration of a server according to one embodiment of the present disclosure.

[0136] Referring to FIG. 3, the server (200) may include memory (210), a processor (220), and a communication interface (230).

[0137] The memory (210) stores various programs or data temporarily or non-temporarily and transmits the stored information to the processor upon the call of the processor (120). Additionally, the memory (210) can store various information required for the operation, processing, or control operation of the processor (120) in an electronic format.

[0138] The memory (210) may include, for example, at least one of a main memory and an auxiliary memory. The main memory may be implemented using a semiconductor storage medium such as ROM and / or RAM.

[0139] In one embodiment, the memory (210) may include at least one second artificial intelligence model (20) for generating diagnostic information.

[0140] The processor (220) controls the overall operation of the server (200). Specifically, the processor (220) is connected to the configuration of the server (200) including memory as described above, and can control the overall operation of the server (200) by executing at least one instruction stored in the memory as described above.

[0141] In one embodiment, the processor (220) can input a medical image received from a wearable device (100) into a second artificial intelligence model (20) and obtain diagnostic information of a subject corresponding to a medical image received from a wearable device (100) based on the output data.

[0142] In particular, the processor (220) can be implemented as a single processor, as well as as multiple processors.

[0143] The processor (220) may be implemented in various ways. For example, one or more processors (220) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. One or more processors (220) may control one or any combination of other components of an electronic device and may perform operations or data processing related to communication. One or more processors (220) may execute one or more programs or instructions stored in memory. For example, one or more processors (220) may perform a method according to one embodiment of the present disclosure by executing one or more instructions stored in memory.

[0144] The communication interface (230) may include a wireless communication interface, a wired communication interface, or an input interface. The wireless communication interface may communicate with various external devices using wireless communication technology or mobile communication technology. Such wireless communication technologies may include, for example, Bluetooth, Bluetooth Low Energy, CAN communication, Wi-Fi, Wi-Fi Direct, ultrawide band (UWB), Zigbee, infrared data association (IrDA), or near field communication (NFC), and mobile communication technologies may include 3GPP, Wi-Max, LTE (Long Term Evolution), 5G, etc.

[0145] A wireless communication interface can be implemented using an antenna, a communication chip, a substrate, etc., capable of transmitting electromagnetic waves to the outside or receiving electromagnetic waves transmitted from the outside.

[0146] A wired communication interface can communicate with various devices based on a wired communication network. Here, the wired communication network can be implemented using physical cables, such as, for example, pair cables, coaxial cables, fiber optic cables, or Ethernet cables.

[0147] Depending on the embodiment, either the wireless communication interface or the wired communication interface may be omitted. Accordingly, the server (200) may include only the wireless communication interface or only the wired communication interface. In addition, the electronic device (100) may be equipped with an integrated communication interface (230) that supports both wireless connection via the wireless communication interface and wired connection via the wired communication interface.

[0148] The server (200) is not limited to having one communication interface (230) that performs a communication connection in one manner, but may include multiple communication interfaces (230) that perform communication connections in multiple manners.

[0149] Meanwhile, the server (200) may be an FTP server (File Transfer Protocol Server), a web server, a database server, or a cloud server, but is not limited thereto. The server may perform one or more functions and operations, and may be implemented as a single device or distributed across multiple devices so that each function and operation is implemented.

[0150] FIG. 4 is a diagram illustrating the operation of a wearable device providing diagnostic information to a user according to one embodiment of the present disclosure.

[0151] Referring to FIG. 4, when diagnostic information is received from a server (200), the wearable device (100) can provide diagnostic information to a user through an optical display (120) that includes at least one of an area where a lesion is identified on a medical image and a type of lesion.

[0152] In one embodiment, the wearable device (100) can identify a bounding box containing a lesion area containing at least one lesion on a first image in which a medical image is identified, when the diagnostic information includes an area in which a lesion is identified, and control an optical display (120) to display the bounding box of the lesion area.

[0153] For example, a wearable device (100) can control the optical display (120) to output a medical image identified from a first image to at least a portion of the optical display (120), and can also control the optical display (120) to display a bounding box of a lesion area together with the medical image identified from the first image.

[0154] In addition, the wearable device (100) may control the optical display (120) to output the type of lesion included in the diagnostic information to at least a portion of the optical display (120).

[0155] As an additional example, the wearable device (100) can identify a location where a bounding box of a lesion area is displayed on an optical display (120) based on the location of the lesion area on the first image, and control the optical display (120) to display a bounding box of a lesion area according to the identified location.

[0156] In an additional embodiment, the wearable device (100) can identify whether there is a region corresponding to the lesion region on the second medical image included in the second image when the medical image is identified based on the output data of the second image at a second time point after the first time point after the first time point in which the first image was taken by the camera (130).

[0157] For example, the wearable device (100) can identify whether there is a region corresponding to a lesion region from the second image based on template matching.

[0158] Specifically, the wearable device (100) acquires a lesion image including pixels constituting a lesion region on a first medical image included in a first image, moves the lesion image in pixel units on a second medical image included in a second image, compares a part of the second medical image with the lesion image to identify the similarity between the part of the second medical image and the lesion image, and can identify an area on the second medical image where the similarity with the lesion image exceeds a threshold as an area corresponding to the lesion region.

[0159] For example, a wearable device (100) can identify similarity by comparing the pixel values ​​of each of the second medical image and the lesion image.

[0160] For example, a wearable device (100) can identify the similarity between a part of a second medical image and a lesion image based on the mean squared error, correlation coefficient, normalized cross-correlation, etc.

[0161] For example, a wearable device (100) can identify similarity based on the difference between the pixel value of each pixel constituting a part of a second medical image and the pixel value of a pixel of a lesion image (having coordinates) corresponding to each pixel constituting a part of the second medical image. In this case, the wearable device (100) can identify a higher degree of similarity between the part of the second image and the lesion image as the value obtained by squaring and summing each difference value between the pixel values ​​(: sum of squares) becomes smaller.

[0162] As an additional example, the wearable device (100) may identify similarity based on the feature vectors of each of the second medical image and the lesion image.

[0163] For example, a wearable device (100) can extract at least one feature vector from an image based on various algorithms such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded-Up Robust Features). In this case, the wearable device (100) can calculate the distance between feature vectors extracted from different images and identify a higher similarity as the calculated distance is shorter, or it can identify the similarity between different images based on cosine similarity using feature vectors extracted from images.

[0164] Accordingly, when an area corresponding to the lesion area is identified, the wearable device (100) can control the optical display (120) so that a bounding box is displayed on the area corresponding to the lesion area in the second medical image.

[0165] That is, by capturing an image within a field of view angle set based on the direction of an optical display (120) formed to be located within the user's field of view through a camera (130), the wearable device (100) displays an area identified as having a lesion on the image captured by the camera (130) through the optical display (120), and by superimposing diagnostic information within the user's field of view, the user can recognize the diagnostic information along with the real scene.

[0166] If a region corresponding to a lesion region is not identified in the second medical image, the wearable device (100) may divide the lesion image (which is an image corresponding to a lesion region) into a plurality of unit images, divide the second medical image into a plurality of comparison regions, and then set at least one comparison region to identify similarity for each of the plurality of unit images.

[0167] That is, if the wearable device (100) cannot find an area corresponding to the entire lesion area because only a part of the lesion area is included in the second medical image, it can divide the lesion image into a plurality of unit images and identify whether there is an area corresponding to each unit image in the second medical image.

[0168] For example, a wearable device (100) can divide a lesion image into four regions by dividing it along the x-axis and y-axis, and identify the region corresponding to the first quadrant as the first unit image, the region corresponding to the second quadrant as the second unit image, the region corresponding to the third quadrant as the third unit image, and the region corresponding to the fourth quadrant as the fourth unit image.

[0169] In addition, the wearable device (100) can divide the second medical image into four regions by dividing it along the x-axis and y-axis, and identify the region corresponding to the first quadrant as the first comparison region, the region corresponding to the second quadrant as the second comparison region, the region corresponding to the third quadrant as the third comparison region, and the region corresponding to the fourth quadrant as the fourth comparison region.

[0170] At this time, the wearable device (100) may set at least one comparison area for identifying similarity on a second medical image for each of a plurality of unit images.

[0171] For example, when a lesion image (including the entire lesion area) is included in a second medical image, considering that there is a high probability that the first unit image is located on the first comparison area, the wearable device (100) may be configured to compare similarity by moving the first unit image in pixel units in an area excluding the first comparison area of ​​the second medical image.

[0172] Based on this logic, the wearable device (100) can be configured to compare similarity by moving pixel by pixel in an area excluding the second comparison area of ​​the second medical image, the third unit image can be configured to compare similarity by moving pixel by pixel in an area excluding the third comparison area of ​​the second medical image, and the fourth unit image can be configured to compare similarity by moving pixel by pixel in an area excluding the fourth comparison area of ​​the second medical image.

[0173] That is, when a lesion image is included in a second medical image, the wearable device (100) can compare similarity by moving the unit image on a comparison area excluding a comparison area where each unit image is likely to be located.

[0174] Through this, the wearable device (100) can reduce meaningless similarity comparisons by narrowing the range for identifying similarity for unit images and more quickly estimate the area corresponding to a part of the lesion area on the second medical image.

[0175] Accordingly, the wearable device (100) can identify an area corresponding to a part of a lesion area on a second medical image based on the similarity identified for each of the plurality of unit images.

[0176] Specifically, the wearable device (100) can identify, for each of the plurality of unit images, an area in which a similarity exceeding a threshold is identified as an area corresponding to the unit image (part of the lesion area).

[0177] Accordingly, when an area corresponding to any one of the multiple unit images is identified on the second medical image, the wearable device (100) can control the optical display (120) so that a bounding box is displayed on the area corresponding to the unit image on the second medical image, and can also control the optical display (120) to display a message indicating that the area where the bounding box is displayed corresponds to a part of the lesion area.

[0178] Additionally, if a region corresponding to at least one unit image is identified on the second medical image, the wearable device (100) may provide an overlay of a lesion image on the second image (which includes the second medical image).

[0179] In one embodiment, the wearable device (100) can set a position where a lesion image is displayed on a second image based on a unit image included in a second medical image, and control an optical display (120) to output a lesion image according to the set position.

[0180] Specifically, the wearable device (100) can arrange a plurality of unit images based on a unit image included in a second medical image to set a position where the lesion image is displayed on the second image so that the lesion image and the second medical image partially overlap.

[0181] Through this, the location of the lesion can be intuitively guided even if only a part of the lesion is identified within the medical image, and by providing the full image of the lesion, it helps to quickly and intuitively recognize the shape of the lesion.

[0182] Meanwhile, if an area corresponding to a lesion area is not identified in the second image, the wearable device (100) can control the optical display (120) to output a medical image identified from the first image to at least a portion of the optical display (120), and can also control the optical display (120) to display a bounding box of the lesion area together with the medical image identified from the first image.

[0183] In addition, if an area corresponding to the lesion area is not identified in the second image, the wearable device (100) may transmit diagnostic information to the user's terminal.

[0184] For example, the user's terminal may be a smartphone, tablet PC, laptop PC, desktop PC, etc., but is not limited thereto.

[0185] FIG. 5 is a diagram illustrating the configuration of an optical display of a wearable device according to one embodiment of the present disclosure.

[0186] Referring to FIG. 5, the optical display (120) may include a light source unit (121), an optical processing unit (122), and an output unit (123).

[0187] The light source unit (121) is configured to generate visual information by outputting light.

[0188] In one embodiment, the light source unit (121) can generate visual information (e.g., text, image, graphic, video, etc.) to visually provide diagnostic information to the user under the control of the processor (140).

[0189] For example, the light source (121) can be implemented as an LED (Light Emitting Diode), Micro LED, OLED (Organic Light Emitting Diode), etc., but is not limited thereto.

[0190] The optical processing unit (122) is configured to control the direction of travel of light output from the light source unit (121).

[0191] In one embodiment, the optical processing unit (122) can control the direction in which the output light is reflected / refracted / diffracted according to the visual information generated from the light source unit (121).

[0192] For example, the optical processing unit (122) may include, but is not limited to, an in-coupler, a waveguide, a pin mirror, a prism, a diffraction grating, etc.

[0193] The output unit (123) is configured to convey visual information to the user.

[0194] In one embodiment, the output unit (123) can output light whose direction of travel is controlled by the optical processing unit (122) in the direction where the user's eye is located.

[0195] For example, the output unit (123) may include, but is not limited to, an out-coupler, a pin mirror, a lens, a prism, a diffraction grating, etc.

[0196] Meanwhile, at least some of the light source unit (121), optical processing unit (122), and output unit (123) may be implemented as a single configuration, and each of the light source unit (121), optical processing unit (122), and output unit (123) may be implemented separately, but is not limited thereto.

[0197] FIG. 6 is a flowchart illustrating the operation of a wearable device according to one embodiment of the present disclosure.

[0198] Referring to FIG. 6, a wearable device (100) can input a plurality of images captured by a camera into a first artificial intelligence model (10) (S110).

[0199] In one embodiment, the wearable device (100) can input an image in real time into the first artificial intelligence model (10) when the image is captured by the camera (130).

[0200] The wearable device (100) can identify whether a medical image is recognized for at least one of a plurality of images according to the output of the first artificial intelligence model (10) (S120).

[0201] In one embodiment, the wearable device (100) can input a plurality of images into the first artificial intelligence model (10) and, based on the output data, identify whether the medical image captured by the target shooting method is recognized as the target shooting area.

[0202] If, based on the data output by inputting an image to the first artificial intelligence model (10), the medical image in which the target shooting area is captured in the target shooting method is not recognized, the wearable device (100) may perform the step of inputting at least one image newly captured by the camera (130) into the first artificial intelligence model (10).

[0203] When a medical image is identified based on the output of the first artificial intelligence model (10), the wearable device (100) can transmit the medical image identified through the first artificial intelligence model (10) to the server (200) (S130).

[0204] In one embodiment, the wearable device (100) sets the number of medical images that can be transmitted per unit time according to the communication status with the server (200), and if the number of medical images identified per unit time exceeds the set number of transmissions, at least one of the identified medical images is selected based on the set number of transmissions, and the selected medical image can be transmitted to the server (200).

[0205] Specifically, the wearable device (100) can set the last medical image transmitted to the server (200) as a reference medical image, identify the similarity between the reference medical image and the identified medical image, select medical images among the identified medical images in which the identified similarity is less than a threshold within the set number of transmittable images, and transmit the selected medical images to the server (200).

[0206] The wearable device (100) can receive diagnostic information of a subject corresponding to a medical image from the server (200) (S140).

[0207] Specifically, the wearable device (100) can obtain diagnostic information of a subject based on the output data by inputting the identified medical image into at least one second artificial intelligence model (20) that is trained to generate diagnostic information and stored in a server (200).

[0208] Accordingly, the wearable device (100) can provide diagnostic information to the user. Meanwhile, the various embodiments described above may be implemented by combining two or more embodiments as long as they do not conflict or contradict each other.

[0209] Specifically, each of the multiple operations, steps, and configurations for implementing one embodiment may be embodied in another embodiment, or the operations and steps of another embodiment may be followed by the last operation or step of one embodiment, but are not limited thereto.

[0210] Meanwhile, computer instructions or computer programs for performing processing operations in the various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When such computer instructions or computer programs stored in the non-transitory computer-readable medium are executed by a processor of a specific device, the specific device described above performs processing operations according to the various embodiments described above.

[0211] A non-transient computer-readable medium refers to a medium that stores data semi-permanently and can be read by a device, unlike media that store data for a short period of time such as registers, caches, and memory. Specific examples of non-transient computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0212] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0213] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure. Explanation of the symbols

[0214] 1000: System 100: Wearable device 110: Memory 120: Optical display 121: Light source 122: Optical processing unit 123: Output section 130: Camera 140: Processor 150: Communication interface 200: Server 210: Memory 220: Processor 230: Communication Interface 10: The first artificial intelligence model 20: The Second Artificial Intelligence Model

Claims

Claim 1 A method of operation of a system comprises: a step in which a wearable device captures at least one image within the field of view of a user wearing the wearable device through a camera included in the wearable device; a step in which the wearable device inputs the image into at least one first artificial intelligence model for recognizing a medical image in which at least a part of the body is captured, and identifies whether a medical image is recognized within the image based on the output data; and a step in which, if it is identified that a medical image is recognized within the image, the wearable device transmits the identified medical image to a server. and the server inputs the identified medical image into at least one second artificial intelligence model for generating diagnostic information and obtains diagnostic information of a subject corresponding to the identified medical image based on the output data; the step of transmitting the identified medical image to the server comprises: the wearable device identifying communication status information according to the packet transmission and reception speed between the server and the wearable device; the wearable device setting a number of medical images that can be transmitted per unit time for transmission to the server based on the identified communication status information; the wearable device selecting at least one of the identified medical images based on the set number of transmissions and transmitting the selected medical image to the server, wherein if the number of medical images identified per unit time exceeds the set number of transmissions, the wearable device setting the last medical image transmitted to the server as a reference medical image; the wearable device identifying the similarity between the reference medical image and the identified medical image; and the wearable device [transmitting] a medical image among the identified medical images in which the identified similarity is less than a threshold A method of operation of a system for selecting within a set number of transmittable images and transmitting the selected medical images to the server. Claim 2 delete Claim 3 In claim 1, the step of identifying whether a medical image is recognized within the image comprises a method of operation of a system in which the wearable device identifies whether a medical image is recognized from at least one of the plurality of images based on data output by inputting a plurality of images sequentially captured by the camera into the first artificial intelligence model. Claim 4 delete Claim 5 delete Claim 6 In paragraph 3, the step of obtaining diagnostic information of a subject corresponding to the identified medical image comprises the server inputting the identified medical image into the second artificial intelligence model to obtain the diagnostic information including at least one of an area where a lesion is identified on the identified medical image and a type of lesion. Claim 7 In claim 6, the method of operation of the system comprises: the step of the server transmitting the diagnostic information to the wearable device; and the step of the wearable device providing the diagnostic information to the user. Claim 8 In claim 7, the step of providing the diagnostic information to the user comprises: the wearable device identifying, based on the diagnostic information, a bounding box including a lesion region containing at least one lesion on a medical image identified from a first image taken at a first time point, and outputting the bounding box through an optical display included in the wearable device. Claim 9 In claim 8, the step of providing the diagnostic information to the user comprises, when an area corresponding to the lesion area is identified on a medical image identified from a second image captured by the camera at a second time point after the first time point, the wearable device identifying a location where the bounding box is displayed on the optical display based on the location of the area corresponding to the lesion area on the second image, and outputting the bounding box according to the identified location. Claim 10 A method of operating a wearable device comprises: a step of capturing at least one image through a camera formed to capture at least one image within a field of view angle range set based on the direction of an optical display formed to be located within the field of view of a user wearing the wearable device; and a step of identifying whether a medical image is recognized within the image based on output data obtained by inputting the image into at least one first artificial intelligence model for recognizing a medical image in which at least a part of the body is captured. A method of operation of a wearable device comprising: a step of, when it is identified that a medical image is recognized within the image, transmitting the identified medical image to a server and obtaining diagnostic information of a subject corresponding to the identified medical image from the server; wherein the step of obtaining diagnostic information of a subject corresponding to the identified medical image from the server comprises: identifying communication status information according to the packet transmission and reception speed between the server and the wearable device; setting a number of medical images that can be transmitted per unit time for the medical images transmitted to the server based on the identified communication status information; selecting at least one of the identified medical images based on the set number of transmissions; transmitting the selected medical image to the server, wherein if the number of medical images identified per unit time exceeds the set number of transmissions, the last medical image transmitted to the server is set as a reference medical image; identifying the similarity between the reference medical image and the identified medical image; selecting a medical image among the identified medical images in which the identified similarity is less than a threshold within the set number of transmissions; and transmitting the selected medical image to the server. Claim 11 In claim 10, the step of obtaining diagnostic information of a subject corresponding to the identified medical image from the server comprises obtaining the diagnostic information based on output data obtained by inputting the identified medical image into at least one second artificial intelligence model stored in the server that is trained to generate diagnostic information. Claim 12 A non-transient computer-readable medium storing at least one instruction that is executed by at least one electronic device and causes said electronic device to perform the method of operation of claim 10.

Images