Portable electronic device and method of wound size measurement

Abstract

A method for measuring a wound size includes: obtaining an input image by using a camera device of a portable electronic device; identifying the input image by using a convolutional neural network model, and selecting a part of the input image having a highest probability of being a wound as an output wound image; and calculating an actual height and an actual width of a wound area in the output wound image according to a lens focal length parameter reported by an operating system, a plurality of reference correction parameters corresponding to a horizontal pitch angle of the portable electronic device, and a pixel height ratio and a pixel width ratio of the output wound image.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to image processing, and in particular, to a portable electronic device and a wound size measurement method. BACKGROUND

[0002] Nowadays, hospitals encounter many problems in the wound care of patients. For example, wound types are diverse, and traditionally, clinical personnel are divided into specialties, resulting in the current hospital wound records and databases being scattered, and there is a lack of integrated care platforms on the market that meet the needs. In addition, the size record of wound photography is complicated, and the traditional square relies on manual measurement with a ruler or with an external hardware device, which is not conducive to application or is hindered by cost considerations, making it difficult to promote. Furthermore, the patient wound care process needs to be continuous, but the community long-term care sites and the situation after the patient returns home are difficult to track and judge, and there is a lack of experts with wound professional experience for continuous care. SUMMARY

[0003] In view of the above, embodiments of the present application provide a portable electronic device and a wound size measurement method to solve the above problems.

[0004] Embodiments of the present application provide a portable electronic device, comprising a display panel, an inertial measurement unit, a camera device, a storage device, and a computing unit. The inertial measurement unit is used to detect a horizontal pitch angle of the portable electronic device. The camera device is used to obtain an input image. The storage device is used to store an operating system, a wound measurement program, a region candidate network model, and a convolutional neural network model. The computing unit is used to execute the wound measurement program to perform the following steps: using the convolutional neural network model to identify the input image, and selecting a part of the input image with the highest probability of being a wound as an output wound image; and calculating the actual height and actual width of the output wound image according to the lens focal length parameter returned by the operating system, the reference correction parameters corresponding to the horizontal pitch angle, and the pixel height ratio and pixel width ratio of the output wound image.

[0005] In some embodiments, the reference correction parameters include a reference object actual height, a reference object actual width, a reference object pixel height ratio, and a reference object focus distance. In a horizontal correction process of the wound measurement procedure, the portable electronic device takes a picture of a reference object at a horizontal tilt angle of 0 degree to obtain a first reference object image, and obtains a first reference object focus distance from an application programming interface of an operating system, and the reference object has the reference object actual height and the reference object actual width. In a vertical correction process of the wound measurement procedure, the portable electronic device takes a picture of the reference object at a horizontal tilt angle of 90 degree to obtain a second reference object image, and obtains a second reference object focus distance from the application programming interface of the operating system. The calculation unit divides a first pixel height of the first reference object image or the second reference object image displayed on a display panel of the portable electronic device by a second pixel height of the display panel to obtain a first reference object pixel height ratio or a second reference object pixel height ratio.

[0006] In some embodiments, in response to the horizontal tilt angle being between 0 and 45 degrees, the calculation unit uses the first reference object focus distance as the reference object focus distance, and uses the first reference object pixel height ratio as the reference object pixel height ratio. In response to the horizontal tilt angle being between 45 and 90 degrees, the calculation unit uses the second reference object focus distance as the reference object focus distance, and uses the second reference object pixel height ratio as the reference object pixel height ratio.

[0007] In some embodiments, the calculation unit calculates equations (1) and (2) to obtain the actual height and the actual width of the output wound image:

[0008]

[0009]

[0010] wherein h c is the reference object actual height; g c is the reference object focus distance; p c is the reference object pixel height ratio; h m is the actual height of the output wound image; g m is the lens focal length parameter; p m is the pixel height ratio; w m is the actual width of the output wound image; w c is the reference object actual width.

[0011] In some embodiments, the operation unit further executes a machine learning segmentation algorithm to segment the output wound image into a wound region and a normal skin region. The operation unit further calculates a first number of pixels in the output wound image and a second number of pixels in the wound region, and divides the second number of pixels by the first number of pixels to obtain a wound region pixel ratio. The operation unit further multiplies the actual height of the output wound image by the actual width to obtain an actual area of the output wound image, and multiplies the actual area by the wound region pixel ratio to obtain an actual area of the wound region.

[0012] In some embodiments, the operation unit further calculates a first red average, a first green average, and a first blue average of red, green, and blue sub-pixels of each pixel in the wound region, and calculates a second red average, a second green average, and a second blue average of red, green, and blue sub-pixels of each pixel in the normal skin region, and calculates a Euclidean distance between the wound region and the normal skin region according to the first red average, the first green average, the first blue average, the second red average, the second green average, and the second blue average to represent a severity of the wound region.

[0013] In some embodiments, in response to the operation unit determining that the actual area of the output wound image is greater than a first predetermined ratio of an actual area of a previous output wound image, the operation unit notifies a server to add a name of a user of the portable electronic device to a care list for relevant inspection by medical personnel. In other embodiments, in response to the operation unit determining that the severity of the output wound image is greater than a second predetermined ratio of a severity of a previous output wound image, the operation unit notifies a server to add a name of a user of the portable electronic device to a care list for relevant inspection by medical personnel.

[0014] In some embodiments, before the operation unit uses the convolutional neural network model to recognize the input image, the operation unit uses the region proposal network model to generate a plurality of first bounding boxes using the input image, and filters a plurality of second bounding boxes having a wound probability greater than a predetermined value from the first bounding boxes. The convolutional neural network model selects the second bounding box having the highest probability of being a wound as the output wound image.

[0015] Embodiments of the present disclosure further provide a method for measuring a size of a wound for a portable electronic device including a display panel and a camera device. The method includes obtaining an input image using the camera device, identifying the input image using a convolutional neural network model, and selecting a portion of the input image having a highest probability of being a wound as an output wound image. The method further includes calculating actual height and actual width of a wound region in the output wound image based on a focal length parameter reported by an operating system, a plurality of reference correction parameters corresponding to a horizontal tilt angle of the portable electronic device, and a pixel height ratio and a pixel width ratio of the output wound image. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A block diagram of a portable electronic device according to an embodiment of the present disclosure.

[0017] Figure 2A A schematic diagram of a model training process of a convolutional neural network model according to an embodiment of the present disclosure.

[0018] Figure 2B A schematic diagram of an architecture of a region proposal network model according to an embodiment of the present disclosure.

[0019] Figure 2C A flowchart of a process for identifying a wound using a convolutional neural network model and a region proposal network model according to an embodiment of the present disclosure.

[0020] Figure 2D A schematic diagram of a portable electronic device according to an embodiment of the present disclosure. Figure 2C A schematic diagram of a process for segmenting an input image into a plurality of first bounding boxes using a region proposal network model according to an embodiment of the present disclosure.

[0021] Figure 2E A schematic diagram of a process for determining a plurality of second bounding boxes from the plurality of first bounding boxes according to an embodiment of the present disclosure. Figure 2C

[0022] A schematic diagram of a process for determining a probability of each second bounding box being a wound using a convolutional neural network model according to an embodiment of the present disclosure. Figure 2F Figure 2C A schematic diagram of an output wound image according to an embodiment of the present disclosure.

[0023] Figure 2G Figure 2C A schematic diagram of a portable electronic device according to an embodiment of the present disclosure.

[0024] Figure 3A A schematic diagram of a portable electronic device according to an embodiment of the present disclosure with a horizontal tilt angle of 0 degrees.

[0025] Figure 3B ​​A schematic diagram of a horizontal correction process of a portable electronic device according to an embodiment of the present application.

[0026] Figure 3C A schematic diagram of a portable electronic device according to an embodiment of the present application with a horizontal tilt angle of 90 degrees.

[0027] Figure 3D A schematic diagram of a vertical correction process of a portable electronic device according to an embodiment of the present application.

[0028] Figure 4A And Figure 4B A schematic diagram of imaging at different distances by a camera device according to an embodiment of the present application.

[0029] Figures 5A-5C A schematic diagram of grouping of output wound images according to an embodiment of the present application.

[0030] Figure 6 A block diagram of a wound care system according to an embodiment of the present application.

[0031] Figure 7A And Figure 7B A schematic diagram of different severity and area of a wound region according to an embodiment of the present application.

[0032] Figure 8 A flowchart of a wound size measurement method according to an embodiment of the present application.

[0033] Wherein, the reference signs are explained as follows:

[0034] 100: portable electronic device

[0035] 101: front surface

[0036] 102: back surface

[0037] 105: camera device

[0038] 110: camera module

[0039] 111: lens

[0040] 112: color filter array

[0041] 113: image sensor

[0042] 115: controller

[0043] 116: auto focus module

[0044] 120: arithmetic unit

[0045] 130: memory unit

[0046] 140: storage device

[0047] 141: operating system

[0048] 142: wound measurement program

[0049] 143: convolutional neural network model

[0050] 144: region proposal network model

[0051] 145: database

[0052] 150: communication interface

[0053] 160: display panel

[0054] 170: inertial measurement unit

[0055] 171: gyroscope

[0056] 172: accelerometer

[0057] 173: magnetometer

[0058] 1430: input image

[0059] 1431: feature map

[0060] 1432: network layer

[0061] 1433: output wound image

[0062] 1441: sliding window

[0063] 1442: bounding box

[0064] 1443: intermediate layer

[0065] 1444: classification layer

[0066] 1445: regression layer

[0067] 210: input image

[0068] 211: first bounding box

[0069] 212: second bounding box

[0070] 230: box

[0071] 240: output wound image

[0072] 310: horizontal line

[0073] 320: reference object

[0074] 330: reference object image

[0075] 510: output wound image

[0076] 511, 521: wound area

[0077] 512, 522: normal skin area

[0078] 520: grouping image

[0079] 600: wound care system

[0080] 610: server

[0081] 615: patient database

[0082] 620: network

[0083] 710, 720: output wound image

[0084] 711, 721: wound area

[0085] 712, 722: normal skin area

[0086] S810-S830: steps

[0087] fd: predetermined distance

[0088] h: actual height

[0089] d: distance of object to be measured

[0090] fp: focal point

[0091] Pm: imaging height

[0092] hi: height

[0093] wi: width

[0094] h2, H: pixel height

[0095] w2, W: pixel width DETAILED DESCRIPTION

[0096] In order to make the above objectives, features and advantages of the present application more obvious and comprehensible, a preferred embodiment is specifically described below, and the accompanying drawings are referred to for further explanation.

[0097] It must be understood that the terms "comprise", "include", and the like used in the present specification are intended to indicate the presence of a specific technical feature, numerical value, method step, operation process, element, and / or component, but do not exclude the addition of more technical features, numerical values, method steps, operation processes, elements, components, or any combination thereof.

[0098] Figure 1A block diagram of a portable electronic device according to an embodiment of the present application is shown. The portable electronic device 100 can be, for example, a smartphone, a tablet computer, a notebook computer, or the like, but embodiments of the present application are not limited thereto. As shown, the portable electronic device 100 includes a camera device 105, a computing unit 120, a memory unit 130, a storage device 140, a communication interface 150, a display panel 160, and an inertial measurement unit 170. The camera module 110 is used to capture an image of an object. For example, the camera device 105 includes at least one camera module 110, a controller 115, and an auto-focus module 116. Figure 1

[0099] The camera module 110 includes a lens 111, a color filter array 112, and an image sensor 113. The color filter array 112 includes a plurality of red color filters, green color filters, and blue color filters arranged in a predetermined pattern, such as a Bayer pattern or other types of patterns. The image sensor 113 can be a color image sensor, such as a charge coupled device (CCD) sensor or a complementary metal-oxidesemiconductor (CMOS) sensor. The controller 115 can be, for example, a microcontroller, but embodiments of the present application are not limited thereto.

[0100] Light from a scene of a target object is passed through the lens 111 and the color filter array 112 and is imaged on the image sensor 113, such that the photoelectric elements of each pixel in the image sensor 113 convert the sensed light into an electrical signal and transmit the electrical signal to the controller 115. The controller 115 can then transmit each pixel of the captured image to the computing unit 120. The auto-focus module 116 includes, for example, a step motor to adjust the focal length of the lens 111 or the entire camera module 110 according to a control signal from the controller 115.

[0101] ​The controller 115 can perform a passive AF algorithm (e.g., a contrast detection or phase detection AF algorithm) on the image captured by the image sensor 113, or can receive a focus touch signal from the display panel 160, to control the autofocus module 116 to fine tune the position of the lens 111 or the entire camera module 110 so that the image sensor 113 can accurately focus on the target object to capture the image of the object. In addition, the controller 115 transmits the focus information of the camera module 110 to the operation unit 120, where the focus information can be, for example, the focal length, the step motor step number, etc., but embodiments of the present application are not limited thereto.

[0102] In some embodiments, the portable electronic device 100 includes two or more camera modules 110, where the lenses 111 of different camera modules 110 have different focal length ranges, for example, and the controller 115 can use the images captured by different camera modules 110 to perform autofocus and control the autofocus module 116 to fine tune the lenses 111 or the camera modules 110 having corresponding focal length ranges so that the image sensors 113 in the above-mentioned camera modules 110 can correctly focus on the target object. The controller 115 can also transmit the focus information of the camera module 110 selected by the autofocus to the operation unit 120, where the focus information can be, for example, the focal length, the step motor step number, etc., but embodiments of the present application are not limited thereto.

[0103] The operation unit 120 is electrically connected to the camera device 105, and the operation unit 120 can be implemented in various ways, such as a dedicated hardware circuit or a general-purpose hardware implementation (e.g., a single processor, a multi-processor with parallel processing capability, or other processors with computing capability), which can be, for example, a central processing unit (CPU), a general-purpose processor, or a microcontroller, but embodiments of the present application are not limited thereto.

[0104] The storage device 140 is, for example, a non-volatile memory such as a hard disk drive, a solid-state disk, or a read-only memory, but embodiments of the present application are not limited thereto. The storage device 140 is used to store an operating system 141 (such as an Android or iOS operating system, etc.) used by the portable electronic device 100 to operate, a wound measurement program 142, a convolutional neural network (CNN) model 143, and a region proposal network (RPN) model 144.

[0105] Assuming that the CNN model 143 and the RPN model have undergone a model training process, the RPN model 144 can segment an object image (such as an input image) captured by the camera device 105 into a plurality of regions (bounding boxes) and find one or more candidate regions with a higher probability from among them to input to the CNN model 143. The CNN model 143 can then perform image recognition on each candidate region to obtain a candidate region with the highest probability (confidence) as a wound image as an output wound image. The wound measurement program 142 is used to estimate dimension information of the output wound image according to a field of view (FOV) of the camera device 105 and a lens focal length parameter reported by the operating system 141, where the dimension information can be, for example, the width and height of the target object.

[0106] The memory unit 130 is, for example, a volatile memory such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), but embodiments of the present application are not limited thereto. The memory unit 130 can serve as an execution space for the operating system 141 and a storage space for temporary data generated by the wound measurement program 142, as well as an image buffer. For example, the operation unit 120 can read the operating system 141 and the wound measurement program 142 stored in the storage device 140 to the memory unit 130 and execute them. The communication interface 150 includes, for example, a wired / wireless transmission interface to connect the portable electronic device 100 to other electronic devices or servers, etc.

[0107] The display panel 160 may, for example, be a liquid crystal display panel, a light-emitting diode display panel, an organic light-emitting diode display panel, e-Ink, or the like, but embodiments of the present application are not limited thereto. In some embodiments, the display panel 160 may, for example, incorporate a touch device (not shown) for touch operations, such as a capacitive or resistive touch device, and the display panel 160 may, for example, be referred to as a touch panel, but embodiments of the present application are not limited thereto.

[0108] The inertial measurement unit 170 includes a gyroscope 171, an accelerometer 172, and a magnetometer 173. The gyroscope 171 is used to measure the orientation and angular speed of the portable electronic device 100, the accelerometer 172 is used to measure the acceleration of the portable electronic device 100, and the magnetometer 173 is used to measure the magnetic field strength and direction of the portable electronic device 100, wherein the data measured by the gyroscope 171, the accelerometer 172, and the magnetometer 173 all belong to inertial information. For example, the accelerometer 172 and the magnetometer 173 in the inertial measurement unit 170 can detect the horizontal pitch angle of the portable electronic device 100.

[0109] In an embodiment, the operating system 141 used by the portable electronic device 100 is, for example, an Android operating system, and after auto-focusing, the lens focal length parameter returned by the Android Camera2 application program interface (API) of the operating system 141 may, for example, be LENS_FOCUS_DISTANCE or LENS_INFO_FOCUS_DISTANCE_CALIBRATION. The above-mentioned lens focal length parameter is, for example, a value corrected by the application program interface (API) of the operating system 141, and the unit is diopter = 1 / meter. The value 0 represents the farthest distance that the lens 111 can focus on, but the farthest distance does not represent infinity. For example, when the distance between the target object and the lens 111 is within a certain focal length range (for example, about 10-25 cm) and the inclination angle of the portable electronic device 100 is within a certain angle (for example, 0 degrees or 90 degrees), the focal length f used by the lens 111 may, for example, be calculated by formula (1):

[0110]

[0111] It is noted that the value of the lens focal length parameter LENS_FOCUS_DISTANCE or LENS_INFO_FOCUS_DISTANCE_CALIBRATION reported by the operating system 141 also varies with the horizontal pitch angle of the portable electronic device 100.

[0112] In another embodiment, if the portable electronic device 100 is an iPhone 4S or later model, and the operating system 141 is iOS 8 or later version, the lens focal length parameter reported by the operating system 141 is, for example, lensPosition. The lens focal length parameter lensPosition is, for example, a value between 0 and 1, where a value of 0 represents the closest distance that the lens 111 can focus on, and a value of 1 represents the farthest distance that the lens 111 can focus on, but the farthest distance is not represented as infinity. It is noted that the lens focal length parameter lensPosition does not directly represent the focal length value of the lens 111, but is a value converted by the operating system 141, and is not equal to (fixed constant / focal length). Furthermore, the value of the lens focal length parameter lensPosition reported by the operating system 141 also varies with the horizontal pitch angle of the portable electronic device 100.

[0113] Figure 2A A schematic diagram of a model training process of a convolutional neural network model in an embodiment of the present application.

[0114] In an embodiment, during the model training process, a professional medical personnel can first mark the wound images (i.e., the area of the wound images is less than or equal to each training image) on each training image, and the wound images are integrated into a feature data set. The model training process of the CNN model 143 and the RPN model 144 can use the same feature data set to simultaneously train the CNN model 143 and the RPN model 144, for example. In some embodiments, the CNN model 143 and the RPN model 144 can be implemented using, for example, Unet, Faster R-CNN, or Mask R-CNN models, but embodiments of the present application are not limited thereto.

[0115] The above-mentioned feature data set can include wound images of chronic wounds or acute wounds of a patient. Chronic wounds can include diabetic foot ulcers, bedsores or pressure sores, venous ulcers, etc. Acute wounds can include chronic poorly healing pressure sores, hemangiomas, ulcers, burns and scalds, diabetic foot and toe infections necrosis, etc., but embodiments of the present application are not limited thereto. It is noted that because the same feature data set is used for model training, the CNN model 143 and the RPN model 144 can use a common feature map.

[0116] For example, in the model training process, the wound images (e.g., only including the wound part) labeled by professional medical personnel are used as the input images 1430 of the CNN model 143. The CNN model 143 can also be referred to as a wound recognition model, which includes a feature map 1431 and a network layer 1432. The feature map 1431 can be implemented by a plurality of convolutional layers (Conv) and pooling layers (MaxPool), for example. The convolutional layers can perform convolution operations on the input images 1430 and extract features, and the pooling layers can amplify the extracted features. The use of the pooling layers has the advantage that the judgment result of the CNN model 143 will not be affected when the input image has a few pixel shifts, and has good noise resistance. Finally, the CNN model 143 flattens the feature extraction results to input into the network layer 1432. The network layer 1432 includes at least two full connection layers (FC), for example. After the flattened feature extraction results pass through the network layer 1432, the output wound images 1433 and their corresponding confidence levels are obtained.

[0117] In some embodiments, in order to reduce the problem of over fitting of the CNN model 143, the weights of some layers (e.g., full connection layers or convolutional layers) in the CNN model 143 can be randomly updated during the model training process. In addition, the function of data augmentation can also be added during the model training process to greatly increase the amount of training data. For example, the operation unit 120 can execute a training data augmentation program (not shown) to mirror, rotate 0 / 90 / 180 / 270 degrees, randomly scale, adjust the contrast and exposure, and the like of each wound image in the feature data set to obtain different augmented training data, and the model training process uses the above augmented training data to train the CNN model 143. When the CNN model 143 is trained to converge, it indicates that the training is complete.

[0118] Figure 2B A schematic diagram of the architecture of the region proposal network model according to an embodiment of the present application.

[0119] The RPN model 144 can also be referred to as a wound positioning model, and the model architecture is as follows Figure 2BThe input of the RPN model 144 is the feature map 1431 of the CNN model 143. In another embodiment, the RPN model 144 can include individual feature maps (not shown) that are different from the feature map 1431 of the CNN model 143. In addition, during the model training process of the RPN model 144, the input of the feature map of the RPN model 144 is the images that are not cropped but have been pre-labeled with the locations of the wounds, i.e., the training images that have been labeled in the training data set.

[0120] For the purpose of illustration, Figure 2B The input of the RPN model 144 is the feature map 1431 of the CNN model 143. For example, the RPN model 144 takes a sliding window 1441 (e.g., a 3x3 convolutional layer) on the feature map 1431, and uses k different anchor boxes 1442 corresponding to each pixel to calculate the probability that each anchor box contains an object. For example, an intermediate layer 1443 with 256 dimensions can be calculated, and a classification layer 1444 and a regression layer 1445 corresponding to each sliding window 1441 can be obtained, where the classification layer 1444 has 2k scores, and the regression layer 1445 has 4k coordinate positions. The RPN model 144 can obtain the bounding box that is most likely to contain an object based on the scores corresponding to each sliding window 1441.

[0121] Figure 2C A flowchart of a process for wound recognition using a convolutional neural network model and a region proposal network model according to an embodiment of the present application. Figure 2D A flowchart of a process for wound recognition using a convolutional neural network model and a region proposal network model according to an embodiment of the present application. Figure 2C A schematic diagram of segmenting an input image into a plurality of first bounding boxes using a region proposal network model according to an embodiment of the present application. Figure 2E A flowchart of a process for wound recognition using a convolutional neural network model and a region proposal network model according to an embodiment of the present application. Figure 2C A schematic diagram of determining a plurality of second bounding boxes from the plurality of first bounding boxes according to an embodiment of the present application. Figure 2F A flowchart of a process for wound recognition using a convolutional neural network model and a region proposal network model according to an embodiment of the present application. Figure 2C A schematic diagram of using a convolutional neural network model to determine the probability that each second bounding box is a wound according to an embodiment of the present application. Figure 2G A flowchart of a process for wound recognition using a convolutional neural network model and a region proposal network model according to an embodiment of the present application. Figure 2C A schematic diagram of outputting a wound image according to an embodiment of the present application.

[0122] Please refer to Figures 2C-2GIn the image recognition stage, the input image 210 is fed into the CNN model 143 to obtain a feature map 1431, and the RPN model 144 can segment the input image 210 into a plurality of first bounding boxes 211 according to the feature map 1431, as shown in FIG. 2B. The first bounding boxes 211 from top to bottom have a probability of containing a wound of 32%, 21%, 72%, and 90%, respectively. Figure 2D

[0123] The RPN model 144 filters out a plurality of second bounding boxes 212 from the first bounding boxes 211 that have a higher probability of containing a wound. For example, the RPN model 144 can set a threshold probability, and if the first bounding box 211 has a probability of containing a wound greater than the threshold probability, the RPN model 144 puts the first bounding box 211 into a proposal, where the first bounding box 211 in the proposal is the second bounding box 212, as shown in FIG. 2C. Figure 2E

[0124] The RPN model 144 inputs each second bounding box 212 into the network layer 1432 of the CNN model 143 to obtain the probability of whether the image of each second bounding box 212 is a wound, as shown in FIG. 2D. The second bounding boxes 212 from left to right have a probability of being a wound of 95% and 62%, respectively. Finally, in block 230, the CNN model 143 uses the second bounding box 212 with the highest probability of being a wound (e.g., 95%, which can also be referred to as confidence) as the output wound image 240, and the output wound image 240 can be displayed on the display panel 160, as shown in FIG. 2E. Figure 2F Figure 2G In detail, because the RPN model 144 can find the second bounding boxes 212 with the highest probability of containing a wound from the input image and the CNN model 143 can determine the probability of each second bounding box being a wound, the CNN model 143 in combination with the RPN model 144 can increase the operation speed of the portable electronic device 100 in determining whether the input image is a wound image and increase the accuracy of wound recognition.

[0125] Figure 4A Figure 4B FIG. 1A is a schematic diagram of a camera device according to an embodiment of the present disclosure.

[0126] In an embodiment, the wound measurement program 142 can calculate the distance d of the object to be measured according to equation (2):

[0127]

[0128] where h is the actual height of the object, f is the focal length of the camera device 105, p is the number of pixels of the image height, and p is the number of pixels of the object height. w m ​​​​​is the number of pixels the target object occupies in the high edge of the image; s is the length of the image sensor. Since most portable electronic devices on the market currently use fixed-focus lenses, the focal length f and image height p... w The length s of the photosensitive element can be considered a fixed value, and these three parameters can be simplified into a lens parameter p, as shown in equation (3):

[0129]

[0130] However, when the wound measurement program 142 executed by the computing unit 120 calculates the wound size in the output wound image, the distance d of the object to be measured (i.e., the distance from the camera device 105 to the wound) and the actual height h of the object to be measured are both unknown values. Therefore, the wound measurement program 142 uses the lens focal length parameter g reported by the operating system 141 to calculate the distance d of the object to be measured. For example, the lens focal length parameter g is inversely related to the distance d of the object to be measured, but the value of the lens focal length parameter g after inverse conversion is not equal to the distance d of the object to be measured, and the inverse of the lens focal length parameter g needs to be multiplied by the offset w to obtain the distance d of the object to be measured, as shown in equation (4):

[0131]

[0132] like Figure 4A As shown, assuming the actual height of the object to be measured (e.g., a wound) is fixed at h and the distance to the object is d, the reflected light from the object passes through the focal point fp and forms an image at height p on the display panel 160 of the portable electronic device 100. m .like Figure 4B As shown, if the distance to the object under test increases to 2d, the imaging height of the reflected light from the object under test on the display panel 160 of the portable electronic device 100 through the focal point fp is [data missing].

[0133] Therefore, based on equations (2) and (3), the proportional relationship in equation (5) can be obtained:

[0134]

[0135] Similarly, according to equation (5), when the distance d of the object to be measured is fixed, if the heights of the object to be measured are 2h and h respectively, then the imaging height of the displayed image is 2p. m and p m Furthermore, according to equation (5), when the height of the object to be measured is fixed at h and the distances to the object to be measured are d and 2d respectively, the imaging height of the displayed image is p. m and

[0136] When the operation unit 120 first executes the wound measurement program 142, the wound measurement program can enter a calibration mode. In the calibration mode, the operation unit 120 can turn on the camera device 105 to focus on a reference object of known size and take a picture, and the user can frame the reference object on the display panel 160, so that the operation unit 120 can obtain a focus distance reference value and a reference object height. Then, according to equation (4), and in combination with the known scale of the reference object, equation (6) can be obtained:

[0137]

[0138] where h c is the actual height of the reference object; g c is the focus distance of the reference object; p c is the pixel height scale of the reference object; h m is the actual height of the object to be measured; g m is the focus distance of the object to be measured; and p m is the pixel height scale of the object to be measured. In some embodiments, the reference object can be, for example, a health insurance card or an ID card, or other objects with fixed known sizes.

[0139] Equation (6) can be simplified to obtain equation (7) to obtain the actual height h m of the object to be measured:

[0140]

[0141] For example, the size of a health insurance card is 53.5 mm (width) * 85.5 mm (height), so the actual height h c of the reference object in equation (7) is a known value. In addition, the portable electronic device 100 focuses by using a stepping motor of the auto-focus module 116 to adjust the position of the lens 111, however, the stepping motor is affected by the gravity of the earth, so the lens focal length parameter reported by the operating system 141 is different when the horizontal pitch angle is 0 degrees and 90 degrees. It should be noted that equation (7) is used to calculate the actual height h m of the object to be measured. If equation (7) is used to calculate the actual width w m of the object to be measured, the actual height h m of the object to be measured and the actual height h c of the reference object in equation (7) can be replaced by the actual width w m of the object to be measured and the actual width w c of the reference object, respectively, as shown in equation (8):

[0142]

[0143] The pixel width ratio of the reference object and the pixel width ratio of the object under test are similar to the pixel height ratio of the reference object. c and the pixel height ratio p of the object under test m Therefore, in equation (8), the reference object pixel height ratio p can be used directly. c and the pixel height ratio p of the object under test m In some embodiments, the arithmetic unit 120 may further calculate the reference object pixel width ratio r. c and the pixel width ratio r of the object under test m Replace the reference object pixel height ratio p in equation (8) respectively c and the pixel height ratio p of the object under test m .

[0144] For example, when the front surface 101 and rear surface 102 of the portable electronic device 100 are completely horizontal with the horizontal line 310, the inertial measurement unit 170 can detect that the horizontal pitch angle of the portable electronic device 100 is 0 degrees. Figure 3A As shown.

[0145] The horizontal calibration process of portable electronic device 100 is as follows: Figure 3B As shown. The user can use a portable electronic device 100 to photograph the reference object 320 at a horizontal tilt angle of 0 degrees and a predetermined distance (e.g., between 10 and 15 cm) fd to obtain an image 330 of the reference object. The height and width of the reference object 320 are h1 and w1, respectively, corresponding to the actual height h of the reference object. c and the actual width w of the reference object c Users can observe the clarity of the captured reference object image on the display panel 160 and press the shutter button to capture the reference object image 330. Users can also adjust the box on the display panel 160 to indicate the size range of the reference object image 330 and save the calibration parameters.

[0146] For example, the resolution of the display panel 160 is W (pixel width) * H (pixel height), and the reference object image 330 presented on the display panel 160 has a pixel height h2 and a pixel width w2. If the pixel height h2 of the reference object image 330 is 1344 pixels, and the pixel height of the display panel 160 is 1920 pixels, then the processing unit 120 can calculate the reference object pixel height ratio p. c =0.70. At this time, if the predetermined distance is approximately 13 cm, the lens focal length parameter LENS_FOCUS_DISTANCE reported by the application programming interface of the operating system 141 is, for example, 7.59. Therefore, the calculation unit 120 can focus the reference object at a distance g. cThe value is set to 7.59. Therefore, multiple reference correction parameters can be obtained during the horizontal correction process, such as the actual height h of the reference object. c Reference object pixel height ratio p c and the focusing distance g of the reference object c .

[0147] When the front surface 101 and rear surface 102 of the portable electronic device 100 are perpendicular to the horizontal line 310, the inertial measurement unit 170 can detect that the horizontal pitch angle of the portable electronic device 100 is 90 degrees. Figure 3C As shown.

[0148] The vertical calibration process of the portable electronic device 100 is as follows: Figure 3D As shown. The user can use a portable electronic device 100 to photograph the reference object 320 at a horizontal tilt angle of 90 degrees and a predetermined distance (e.g., between 10 and 15 cm) fd to obtain an image 330 of the reference object. The height and width of the reference object 320 are h1 and w1, respectively, corresponding to the actual height h of the reference object. c and the actual width w of the reference object c Furthermore, the resolution of the display panel 160 is W (pixel width) * H (pixel height), and the reference object image 330 presented in the display panel 160 has a pixel height h2 and a pixel width w2.

[0149] For example, if the pixel height h2 of the reference object image 330 is 1382 pixels and the pixel height of the display panel 160 is 1920 pixels, then the processing unit 120 can calculate the pixel height ratio p of the reference object. c =0.72. At this time, if the aforementioned predetermined distance is approximately 13 cm, the lens focal length parameter LENS_FOCUS_DISTANCE reported by the application programming interface of the operating system 141 is, for example, 8.65. Therefore, the calculation unit 120 can focus the reference object at a distance g. c The value is set to 8.65. Therefore, multiple reference correction parameters can be obtained during the vertical correction process, such as the actual height h of the reference object. c Reference object pixel height ratio p c and the focusing distance g of the reference object c .

[0150] It should be noted that the reference object focusing distance g obtained during the horizontal and vertical correction processes... care different reference correction parameters. For example, when the horizontal tilt angle of the portable electronic device 100 is between 0 degree and less than 45 degrees, the wound measurement program 142 uses the above-mentioned reference correction parameters obtained by the horizontal correction process to substitute into equations (7) and (8) to calculate the actual height h m and the actual width w m of the object to be measured (for example, in centimeters). When the horizontal tilt angle of the portable electronic device 100 is between greater than or equal to 45 degrees and 90 degrees, the wound measurement program 142 uses the above-mentioned reference correction parameters obtained by the vertical correction process to substitute into equations (7) and (8) to calculate the actual height h m and the actual width w m of the object to be measured (for example, in centimeters).

[0151] Figures 5A-5C is a schematic diagram of grouping the output wound image according to an embodiment of the present application.

[0152] Figure 5A The output wound image 510 generated by the CNN model 143 and the RPN model 144 is, for example, an RGB image, that is, each pixel of the output wound image is composed of red, green, and blue sub-pixels with brightness between 0 and 255. The wound measurement program 142 can use a machine learning grouping algorithm to group each pixel of the output wound image 510, for example, into a wound group and a normal skin group. The machine learning grouping algorithm can be, for example, a K-Means grouping method, a hierarchical grouping method, or other grouping algorithms in the field of the present application, but embodiments of the present application are not limited thereto. For example, in the case of the output wound image 510 shown in Figure 5A After the image grouping processing of the output wound image 510, a grouped image 520 as shown in Figure 5B can be obtained, which can be divided into a wound area 521 and a normal skin area 522.

[0153] After the wound measurement program 142 obtains the wound region 521 and the normal skin region 522 from the grouped image 520, it can calculate the area of ​​the wound region 521 in the grouped image 520. For example, in the aforementioned embodiment, the wound measurement program 142 can calculate the actual height and actual width of the object to be measured in the output wound image. Assuming that the actual height and actual width of the object to be measured are 3 cm and 4 cm respectively, the actual area corresponding to the output wound image 510 is 12 square centimeters. If the output wound image 510 has 50,000 pixels and the wound measurement program 142 calculates that the number of pixels in the wound region 521 is 45,000 pixels, then the wound measurement program 142 can calculate the pixel ratio of the wound region as 45,000 / 50,000 = 0.9. Therefore, the wound measurement program 142 can calculate that the actual area corresponding to the wound region 521 is 12 * (45,000 / 50,000) = 10.8 square centimeters.

[0154] In one embodiment, after the wound measurement program 142 obtains the wound region 521 and the normal skin region 522, the output wound image 510 can be segmented into the wound region 511 and the normal skin region 512, and the values ​​of the red, green, and blue sub-pixels of each pixel in the wound region 511 and the normal skin region 512 can be determined, such as... Figure 5C As shown. The wound measurement program 142 calculates the average values ​​of the red, green, and blue sub-pixels of each pixel in the wound region 521, for example, using W_R respectively. avg W_G avg and W_B avg The wound measurement procedure 142 calculates the average values ​​of the red, green, and blue sub-pixels of each pixel in the normal skin area 522, for example, using N_R... avg N_G avg and N_B avg express.

[0155] Next, the wound measurement procedure 142 can use the Euclidean distance formula to calculate the severity of the wound area 521, as shown in equation (9):

[0156]

[0157] Among them, N_R avg N_G represents the average value of all red subpixels in the normal skin area of ​​522; avg N_B represents the average value of all green subpixels within the normal skin region of 522. avg The average value of all blue subpixels in the normal skin area of ​​522; W_R avg W_G represents the average value of all red subpixels in the wound region 521.avg represents the average value of all green sub-pixels in the wound area 521; W_B avg represents the average value of all blue sub-pixels in the wound area 521.

[0158] The severity is a floating point number between 0 and 255 in terms of Euclidean distance. The closer the severity is to 255, the higher the severity of the wound area 521 compared to the normal skin area 522 (i.e. the lower the similarity). The closer the severity is to 0, the lower the severity of the wound area 521 compared to the normal skin area 522 (i.e. the higher the similarity).

[0159] Figure 6 is a block diagram of a wound care system in accordance with an embodiment of the present application. Please refer to FIG. 6A and FIG. 6B simultaneously. Figure 1 and Figure 6 .

[0160] In an embodiment, the wound care system 600 comprises one or more portable electronic devices 100 and a server 610, wherein each portable electronic device 100 is connectable to the server 610 through a network 620. A patient or a medical staff can use the corresponding portable electronic device 100 to take pictures of the wound area of the patient at regular intervals to obtain input images. The wound measurement program 142 executed by each portable electronic device 100 can use the CNN model 143 and the RPN model 144 to recognize the input images to obtain output wound images (i.e. the bounding box cropped from the input image has the highest probability of being a wound).

[0161] Whenever the patient or the medical staff takes pictures of the wound area of the patient using the portable electronic device 100, the wound measurement program 142 can store the output wound images generated by the CNN model 143 and the RPN model 144, the corresponding time information (e.g. the time of taking the input images), the size information (including the height, the width and the area), and the severity of the wound area in the output wound images compared to the normal skin area in the database 145 for the subsequent care process.

[0162] In some embodiments, each portable electronic device 100 can further synchronize the content of the respective database 145 to the server 610, wherein the server 610 also comprises a patient database 615 to record the history of the user name, the wound area, the output wound images, the time information, the size information and the severity of the output wound images of each portable electronic device 100. In addition, the server 610 can further sort the user names of different patients according to the above information in the patient database 615 to establish a care list for the medical staff to review on the server 610.

[0163] For example, if a patient's most recently captured wound image shows a relatively large area and high severity (long Euclidean distance, i.e., low similarity), server 610 can prioritize this patient in the care list. Figure 7A As shown, assuming the average value (W_R) of the red, green, and blue sub-pixels of the wound region 711 in the output wound image 710 is... avg, W_G avg ,W_B avg The value is (230, 172, 148), and the average value (N_R) of the red, green, and blue subpixels in the normal skin region 712 in the output wound image 710 is (230, 172, 148). avg, N_G avg ,N_B avg Given (160, 106, 92), the wound measurement procedure 142 can calculate the severity of the wound area 711 according to equation (9). If the severity level is greater than a predetermined threshold (e.g., 70, not limited), the wound measurement procedure 142 can determine that the wound area 711 is a more severe wound area.

[0164] If a patient's most recently captured wound image shows a relatively small area and low severity (shorter Euclidean distance, i.e., higher similarity), server 610 can place this patient further down the care list. For example... Figure 7B As shown, assuming the average value (W_R) of the red, green, and blue sub-pixels in the wound region 721 of the output wound image 720. avg, W_G avg ,W_B avg The value is (169, 114, 121), and the average value (N_R) of the red, green, and blue subpixels in the normal skin region 722 of the output wound image 720 is (169, 114, 121). avg, N_G avg ,N_B avg Given (176, 143, 119), the wound measurement procedure 142 can calculate the severity of the wound area 711 according to equation (9). If the severity level is less than a predetermined threshold (e.g., 70, not limited), the wound measurement procedure 142 can determine that the wound area 721 belongs to a less severe wound area.

[0165] For example, the wound measurement program 142 can compare the size information or the severity of the current output wound image with the previous output wound image(s) obtained by the previous photographing. In an embodiment, when the area of the current output wound image of a certain portable electronic device 100 is greater than the area of the previous output wound image by a predetermined percentage (e.g., 5%, non-limiting), the wound measurement program 142 determines that the wound of the user of the portable electronic device 100 has a sign of expansion, and thus the wound measurement program 142 notifies the server 710 to add the name of the user of the portable electronic device 100 (e.g., Zhang San) to the care list and set an alert notification in the care list to graphically present “area” for the medical staff to perform relevant review.

[0166] In addition, when the severity of the current output wound image of the portable electronic device 100 is greater than the severity of the previous output wound image by a predetermined percentage (e.g., 10%, non-limiting), the wound measurement program 142 determines that the wound of the user of the portable electronic device 100 has a sign of deterioration, and thus the wound measurement program 142 notifies the server 710 to add the name of the user of the portable electronic device 100 (e.g., Zhang San) to the care list and set an alert notification in the care list to graphically present “severity” for the medical staff to perform relevant review.

[0167] Figure 8 A flowchart of a wound size measurement method according to an embodiment of the present application.

[0168] At step S810, an input image is obtained by the camera device 105 of the portable electronic device 100. For example, the camera device 105 photographs a wound site of a user at a first horizontal pitch angle to obtain an input image. For example, when photographing is performed by the portable electronic device 100, the horizontal pitch angle of the electronic device 100 can vary from 0 degrees to 90 degrees, for example, and the inertial measurement unit 170 can detect the change in the horizontal pitch angle of the portable electronic device 100.

[0169] At step S820, the operation unit 120 identifies the input image by using the CNN model 143 and selects a part of the input image with the highest probability of being a wound as an output wound image. The operation unit 120 cuts the input image into a plurality of first bounding boxes by using the RPN model 144, and filters a plurality of second bounding boxes containing wounds with a probability greater than a predetermined value from the first bounding boxes. For example, as shown in FIG. 8, the operation unit 120 can cut the input image into a plurality of first bounding boxes 810, and filter a plurality of second bounding boxes 820 containing wounds with a probability greater than a predetermined value from the first bounding boxes 810. Figure 2DAs shown, in the image recognition stage, the input image 210 is subjected to feature extraction by the CNN model 143 to obtain a feature map 1431, and the RPN model 144 can segment the input image 210 into a plurality of first bounding boxes 211 according to the feature map 1431. In addition, the RPN model 144 can set a threshold probability, and if the probability that the first bounding box 211 contains a wound is greater than the threshold probability, the RPN model 144 will put the first bounding box 211 into the candidate region (proposals), wherein the first bounding box 211 in the candidate region is the second bounding box 212, as shown. Figure 2E

[0170] In step S830, the operation unit 120 calculates the actual height and the actual width of the output wound image by using the focal length parameter reported by the operating system 141 running on the portable electronic device 100, the plurality of reference correction parameters corresponding to the horizontal tilt angle of the portable electronic device 100, and the pixel height ratio and the pixel width ratio of the output wound image (e.g., displayed on the display panel 160). For example, when the horizontal tilt angle of the portable electronic device 100 is between 0 degrees and less than 45 degrees, the wound measurement program 142 uses the above-mentioned reference correction parameters obtained by the horizontal correction process to substitute into equations (7) and (8) to calculate the actual height h m and the actual width w m of the measured object (e.g., in centimeters). When the horizontal tilt angle of the portable electronic device 100 is greater than or equal to 45 degrees and less than 90 degrees, the wound measurement program 142 uses the above-mentioned reference correction parameters obtained by the vertical correction process to substitute into equations (7) and (8) to calculate the actual height h m and the actual width w m of the measured object (e.g., in centimeters).

[0171] In summary, the embodiments of the present application provide a portable electronic device and a wound size measurement method, which can calculate the actual height and the actual width of the output wound image according to the focal length parameter reported by the operating system, the plurality of reference correction parameters corresponding to the first horizontal tilt angle, and the pixel height ratio and the pixel width ratio of the output wound image. The portable electronic device can obtain the above-mentioned reference correction parameters through the horizontal correction process and the vertical correction process.

[0172] ​The portable electronic device and the wound size measurement method in the embodiments of the present application can accurately calculate the actual height, the actual width and the area of the wound area in the input image in an objective manner, and can calculate the severity of the wound area. In addition, the portable electronic device and the wound size measurement method in the embodiments of the present application can compare the area or the severity of the current output wound image with the previous output wound image to determine whether the wound area has signs of expansion or deterioration, and then transmit a warning notification to the server for the medical staff to perform relevant inspection, so that such patients can be cared for.

[0173] The words such as "first", "second", "third" and the like used in the claims are used to modify the elements in the claims, and are not used to indicate the priority order, the prior relationship, or the time sequence when the method steps are executed, but are used to distinguish the elements with the same name.

[0174] Although the embodiments of the present application are disclosed as above, they are not used to limit the scope of the present application, and anyone with ordinary knowledge in the art can make some changes and modifications without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A portable electronic device, comprising: an inertial measurement unit configured to detect a horizontal tilt angle of the portable electronic device; a camera device configured to obtain an input image; a storage device configured to store an operating system, a wound measurement program, a region candidate network model, and a convolutional neural network model; and a computing unit configured to execute the wound measurement program to perform the following steps: recognize the input image using the convolutional neural network model, and select a portion of the input image having a highest probability of being a wound as an output wound image; and calculate an actual height and an actual width of the output wound image according to a lens focal length parameter reported by the operating system, a plurality of reference correction parameters corresponding to the horizontal tilt angle, and a pixel height ratio and a pixel width ratio of the output wound image, wherein the computing unit further performs a machine learning clustering algorithm to divide the output wound image into a wound region and a normal skin region; wherein the computing unit further calculates a first number of pixels in the output wound image and a second number of pixels in the wound region, and divides the second number of pixels by the first number of pixels to obtain a wound region pixel ratio; wherein the computing unit further multiplies the actual height of the output wound image by the actual width of the output wound image to obtain an actual area of the output wound image, and multiplies the actual area of the output wound image by the wound region pixel ratio to obtain an actual area of the wound region.

2. The portable electronic device of claim 1, wherein the reference correction parameter comprises: a reference object actual height, a reference object actual width, a reference object pixel height ratio, and a reference object focus distance. 3.The portable electronic device of claim 2, wherein in a horizontal correction process of the wound measurement program, the portable electronic device photographs a reference object to obtain a first reference object image with the horizontal tilt angle being 0 degree, and obtains a first reference object focus distance from an application programming interface of the operating system, and the reference object has the reference object actual height and the reference object actual width; wherein in a vertical correction process of the wound measurement program, the portable electronic device photographs the reference object to obtain a second reference object image with the horizontal tilt angle being 90 degree, and obtains a second reference object focus distance from the application programming interface of the operating system; wherein, the computing unit divides a first pixel height of the first reference object image or the second reference object image displayed on a display panel of the portable electronic device by a second pixel height of the display panel to obtain a first reference object pixel height ratio or a second reference object pixel height ratio. 4.The portable electronic device of claim 3, wherein in response to the horizontal tilt angle being between 0 and 45 degrees, the computing unit uses the first reference object focus distance as the reference object focus distance, and uses the first reference object pixel height ratio as the reference object pixel height ratio. wherein In response to the horizontal tilt angle being between 45 and 90 degrees, the operation unit uses the second reference object focus distance as the reference object focus distance and uses the second reference object pixel height ratio as the reference object pixel height ratio.

5. The portable electronic device of claim 4, wherein the operation unit calculates equations (1) and (2) to obtain the actual height and the actual width of the output wound image: wherein h c is the actual height of the reference object; g c focal distance for the reference object; p c pixel height ratio for the reference object; h m actual height of the output wound image; g m lens focal length parameter; p m pixel height ratio of the output wound image; w m actual width of the output wound image; w c is the actual width of the reference object.

6. The portable electronic device of claim 1, wherein the operation unit further calculates a first red average, a first green average, and a first blue average of red, green, and blue sub-pixels of each pixel in the wound area, calculates a second red average, a second green average, and a second blue average of red, green, and blue sub-pixels of each pixel in the normal skin area, and calculates a Euclidean distance between the wound area and the normal skin area based on the first red average, the first green average, the first blue average, the second red average, the second green average, and the second blue average to represent a severity of the wound area.

7. The portable electronic device of claim 6, wherein in response to the operation unit determining that the actual area of the output wound image is greater than an actual area of a previous output wound image by more than a first predetermined ratio, the operation unit notifies a server to add a name of a user of the portable electronic device to a care list for relevant inspection by medical personnel.

8. The portable electronic device of claim 6, wherein in response to the operation unit determining that the severity of the output wound image is greater than a severity of a previous output wound image by more than a second predetermined ratio, the operation unit notifies a server to add a name of a user of the portable electronic device to a care list for relevant inspection by medical personnel.

9. The portable electronic device of claim 1, wherein before the operation unit uses the convolutional neural network model to recognize the input image, the operation unit uses the region proposal network model to generate a plurality of first bounding boxes using the input image and filters a plurality of second bounding boxes from the first bounding boxes that have a probability of containing a wound greater than a predetermined value. wherein The convolutional neural network model selects the second bounding box having the highest probability of being a wound as the output wound image.

10. A method for measuring a size of a wound, the method being used in a portable electronic device, the portable electronic device comprising a display panel and a camera device, the method comprising: obtaining an input image using the camera device; recognizing the input image using a convolutional neural network model and selecting a portion of the input image having a highest probability of being a wound as an output wound image; calculating an actual height and an actual width of a wound area in the output wound image based on a lens focal length parameter reported by an operating system, a plurality of reference correction parameters corresponding to a horizontal tilt angle of the portable electronic device, and a pixel height ratio and a pixel width ratio of the output wound image. executing a machine learning clustering algorithm to divide the output wound image into a wound region and a normal skin region; calculating a first number of pixels in the output wound image and a second number of pixels in the wound region, and dividing the second number of pixels by the first number of pixels to obtain a wound region pixel ratio; and multiplying the actual height of the output wound image by the actual width to obtain an actual area of the output wound image, and multiplying the actual area by the wound region pixel ratio to obtain an actual area of the wound region.

11. The wound size measurement method of claim 10, wherein the reference correction parameter comprises: a reference object actual height, a reference object actual width, a reference object pixel height ratio, and a reference object focus distance.

12. The wound size measurement method of claim 11, wherein during a horizontal calibration process of a wound measurement program, the portable electronic device photographs a reference object at a horizontal tilt angle of 0 degree to obtain a first reference object image, and obtains a first reference object focus distance from an application programming interface of an operating system, and the reference object has the reference object actual height and the reference object actual width; wherein during a vertical calibration process of the wound measurement program, the portable electronic device photographs the reference object at a horizontal tilt angle of 90 degree to obtain a second reference object image, and obtains a second reference object focus distance from the application programming interface of the operating system; wherein an operation unit divides a first pixel height of the first reference object image or the second reference object image displayed on a display panel by a second pixel height of the display panel to obtain a first reference object pixel height ratio or a second reference object pixel height ratio.

13. The wound size measurement method of claim 12, wherein in response to the horizontal tilt angle being between 0 and 45 degrees, the operation unit uses the first reference object focus distance as the reference object focus distance, and uses the first reference object pixel height ratio as the reference object pixel height ratio; wherein in response to the horizontal tilt angle being between 45 and 90 degrees, the operation unit uses the second reference object focus distance as the reference object focus distance, and uses the second reference object pixel height ratio as the reference object pixel height ratio.

14. The wound size measurement method of claim 13, further comprising: calculating equations (1) and (2) to obtain the actual height and the actual width of the output wound image: wherein, h c is the actual height of the reference object; g c is the focus distance of the reference object; p c is the pixel height ratio of the reference object; h m is the actual height of the output wound image; g m is the lens focal length parameter; p m is the pixel height ratio of the output wound image; w m is the actual width of the output wound image; w c is the actual width of the reference object.

15. The wound size measurement method of claim 10, further comprising: calculating a first red average, a first green average, and a first blue average of red, green, and blue sub-pixels of each pixel in the wound region; calculating a second red average, a second green average, and a second blue average of red, green, and blue sub-pixels of each pixel in the normal skin region; and A Euclidean distance between the wound area and the normal skin area is calculated according to the first red average, the first green average, the first blue average, the second red average, the second green average, and the second blue average to represent a severity of the wound area.

16. The method of measuring the size of a wound of claim 15, further comprising: In response to determining that the actual area of the output wound image is greater than an actual area of a previous output wound image by more than a first predetermined percentage, a server is notified to add a name of a user of the portable electronic device to a care list for relevant inspection by medical personnel.

17. The wound size measurement method of claim 15, further comprising: In response to determining that the severity of the output wound image is greater than a severity of a previous output wound image by more than a second predetermined percentage, a server is notified to add a name of a user of the portable electronic device to a care list for relevant inspection by medical personnel.

18. The wound size measurement method of claim 10, wherein before recognizing the input image using the convolutional CNN model, the method further comprises: using the region proposal network model to generate a plurality of first bounding boxes using the input image, and filtering a plurality of second bounding boxes from the first bounding boxes that have a probability of containing a wound greater than a predetermined value; and using the convolutional CNN model to select the second bounding box with the highest probability of being a wound as the output wound image.

Images