A health examination system based on palm information

Abstract

This invention provides a health checkup system based on palm information. An image acquisition device acquires images of the user's hands, and an information processing device processes these images, extracts image feature points, and identifies the health status of the internal organs and / or limbs corresponding to each feature point according to a first preset rule. The system also displays symptom names corresponding to a second preset rule. Because the user's hand images are acquired through an image acquisition device, the user's hands do not need to come into contact with the information acquisition device, greatly reducing the risk of infection. Furthermore, since only image acquisition is required, the hardware cost is relatively low. Simultaneously, because diagnosis is based on the user's hand characteristics, the results are closer to those found in Western medicine, making them more acceptable to ordinary users.

Description

Technical Field

[0001] This invention relates to the field of health checkup systems, and more particularly to a health checkup system based on palm information. Background Technology

[0002] In the health checkup industry, there are many health checkup methods, but each has its own advantages and disadvantages. Only a limited number of methods can detect specific diseases, primarily including tongue diagnosis, pulse diagnosis, facial diagnosis, and electronic pulse hand diagnosis. However, these methods have limitations in applicability, failing to accurately diagnose diseases related to Western medicine, mainly relying on Traditional Chinese Medicine (TCM) diagnoses. Furthermore, their data collection methods have drawbacks, hindering rapid popularization and applicability to the general public. Therefore, providing a health checkup system based on palm information could significantly improve the efficiency of health diagnosis and broaden its target user base. Summary of the Invention

[0003] The purpose of this invention is to provide a health check system based on palm information, which can greatly improve the efficiency of health diagnosis and the scope of user targeting.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] The first aspect of this application provides a health check system based on palm information, including:

[0006] An image acquisition device for acquiring images of a user's hand;

[0007] The information processing device processes the user's hand image, extracts image feature points, identifies the health status of the internal organs and / or limbs corresponding to each image feature point according to a first preset rule, and displays the symptom names corresponding to a second preset rule.

[0008] In one alternative implementation, the first prediction rule is a rule based on traditional Chinese medicine theory; the second prediction rule is a rule based on Western medical symptoms.

[0009] A second aspect of this application provides a health checkup system based on palm information, comprising:

[0010] Near-end devices are used to capture images of the user's hands using a camera;

[0011] The remote server obtains symptom diagnosis results and recommended treatment plans based on the image feature point information.

[0012] The near-end device or the remote server processes the user's hand image and extracts the image feature point information.

[0013] In one alternative implementation, the remote server also sends the symptom diagnosis result information or the recommended treatment plan information to the proximal device, and the proximal device displays the symptom diagnosis result information or the recommended treatment plan information accordingly.

[0014] In one alternative implementation, the remote server also sends the symptom diagnosis result information or the recommended treatment plan information to the mobile terminal, and the mobile terminal displays the symptom diagnosis result information or the recommended treatment plan information accordingly.

[0015] In one alternative implementation, the proximal device includes a display screen for displaying information based on the symptom diagnosis results or the recommended treatment plan.

[0016] In one alternative implementation, the near-end device includes a touchscreen and a keyboard, the touchscreen and the keyboard being used to acquire user information.

[0017] In one alternative implementation, the near-end device includes a speaker; the speaker is used to prompt the user and to interact with the user.

[0018] One alternative implementation is characterized in that the proximal device is a health checkup device;

[0019] The user inserts their palm into the health check device, performs the operation according to the first prompt and changes to a valid palm, changes to a different palm according to the second prompt and enters relevant user information, and waits for the result according to the third prompt or obtains the diagnostic identification information prompted by the device using a mobile terminal.

[0020] The health checkup device detects whether a user's palm is inserted. If no palm is detected, the device issues the first prompt message. If a palm is detected, the device takes a picture or video to check if it is a valid palm. If the palm is invalid, the device issues the first prompt message. If the palm is valid, the device issues the second prompt message. After obtaining valid information about the user's palms, the device processes the user's hand images and issues the third prompt message. The device also displays the user's diagnostic identification information, diagnostic information, and recommended treatment plan.

[0021] The remote server preprocesses the user's hand image processed by the health checkup device, performs a disease knowledge graph comparison query based on the preprocessed image to obtain symptom diagnosis information, obtains the corresponding recommended treatment plan information based on the symptom diagnosis information, and sends the symptom diagnosis information and the recommended treatment plan information to the health checkup device or mobile terminal.

[0022] The mobile terminal obtains the user's diagnostic identification information, retrieves the symptom diagnosis result information and the recommended treatment plan information from the remote server based on the user's diagnostic identification information, and displays the symptom diagnosis result information and the recommended treatment plan information.

[0023] In one alternative implementation, the mobile terminal is a mobile phone or a tablet device.

[0024] One alternative implementation involves processing the user's hand image to extract image feature point information, including:

[0025] Process at least one of the following factors in the hand image: whether gloves are worn, whether the palm is in the detection area, whether the palm is left or right or right, whether the fingers are bent or spread;

[0026] An affine transformation is performed on the user's hand image to obtain the skin mask and extract the palm contour; the palm contour is the image feature point information.

[0027] One alternative implementation involves detecting the texture information corresponding to the coordinates of 21 preset key points on the palm to determine whether the hand in the hand image is wearing gloves.

[0028] The center point is obtained by weighted average of the coordinates of 21 preset key points of the palm. The radius is obtained by taking the average distance from the center point to the vertex of the convex hull of the palm. The rectangular area where the palm is located is calculated and compared with the detection area to determine whether the palm is located in the detection area.

[0029] Let the first, second, and third horizontal creases of the fingers in the palm and the fingertip be points a, b, c, and d, respectively. Then, make a trapezoid with points a, b, c, and d. Determine whether the fingers are bent by judging the ratio of the length of ad to the length of abcd. When the direction of the bend of the fingers is towards the shooting direction, combine the perspective relationship in space to determine whether the fingers are bent.

[0030] The ratio of the distance between the first and second transverse creases of two adjacent fingers in the palm determines whether the fingers are spread apart.

[0031] The process of performing affine transformation on the user's hand image, obtaining skin mask and extracting palm contour includes: determining the transformation center through the hand image, and calculating the affine transformation matrix through the transformation center and the following formula;

[0032]

[0033] Then, perform an affine transformation based on the affine transformation matrix to obtain the first processed image;

[0034] The first processed image is converted to a preset color space to obtain a second processed image. The pixel value distribution of the second processed image is adjusted using the following formula to obtain a third processed image.

[0035]

[0036] Where cdf(v) represents the cumulative distribution function, cdf min The minimum non-zero value of the cumulative distribution function, M×N gives the number of pixels in the image, and L is the number of gray levels used;

[0037] The third processed image is subjected to noise reduction to obtain the fourth processed image, which is specifically adjusted using the following formula;

[0038]

[0039] The fourth image is binarized to obtain the palm outline.

[0040] One alternative implementation, wherein obtaining symptom diagnosis results and recommended treatment plan information based on the image feature point information, includes:

[0041] Key point localization and zone localization of the palm outline;

[0042] The key point localization based on the palm contour includes: using 4 key points at the base of the fingers, 1 key point above the wrist crease, 16 key points in the finger area, and 16 key points in the center area of ​​the palm to locate the key points of the palm contour.

[0043] Based on the key points of the palm contour, the palm contour is divided into multiple detection areas.

[0044] The system detects and assesses the symptoms in the multiple detection areas, and obtains symptom diagnosis results and recommended treatment plans.

[0045] One alternative implementation method, wherein detecting the disease in the plurality of detection areas includes:

[0046] The images of the multiple detection areas are converted to a strong color gamut space, and color features are extracted from the images;

[0047] The second moment of the color feature is obtained using the following formula;

[0048]

[0049]

[0050]

[0051] The corresponding color directionality characteristics are then obtained using the following formula;

[0052]

[0053] Where a, b, and c are the composite expressions for the zeroth moment, first moment, and second moment of the matrix, respectively.

[0054] Compared with the prior art, the solution of the present invention has the following advantages:

[0055] In this invention, the health check system based on palm information acquires images of the user's hands through an image acquisition device. The user's hands do not need to come into contact with the information acquisition device, which greatly reduces the risk of infection. Since it only requires image acquisition, the hardware cost is relatively low. At the same time, since diagnosis is based on the user's hand characteristics, the results can be more similar to those of Western medicine, making it easier for ordinary users to accept. Attached Figure Description

[0056] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0057] Figure 1 This is a first schematic block diagram of the health checkup system based on palm information of the present invention;

[0058] Figure 2 This is a second schematic block diagram of the health checkup system based on palm information of the present invention;

[0059] Figure 3 This is a third schematic block diagram of the health checkup system based on palm information of the present invention;

[0060] Figure 4 This is a flowchart illustrating the health checkup system based on palm information according to the present invention.

[0061] Figure 5 This is a schematic diagram illustrating whether the palm of the hand is located in the detection area according to the present invention;

[0062] Figure 6 This is a schematic diagram illustrating the hand bending determination method of the present invention;

[0063] Figure 7 This is a schematic diagram illustrating the method for determining whether a finger is open according to the present invention.

[0064] Figure 8 This is a schematic diagram of the affine transformation of the hand image according to the present invention;

[0065] Figure 9 This is a schematic diagram of the color space conversion for hand images according to the present invention;

[0066] Figure 10 The skin mask and palm outline of the hand image are for the present invention;

[0067] Figure 11 This is a schematic diagram of the key points of the finger root and wrist crease in this invention.

[0068] Figure 12 This is a schematic diagram of key points in the finger area of ​​the present invention;

[0069] Figure 13 This is a schematic diagram of the central area of ​​the palm in this invention;

[0070] Figure 14 This is a schematic diagram of the palm partition positioning of the present invention. Detailed Implementation

[0071] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0072] Current medical examination diagnostic methods that can provide a relatively comprehensive diagnosis mainly include the following:

[0073] Facial diagnostic devices require taking images of the user's face on the device. Based on the connection between different parts of the face and internal organs in traditional Chinese medicine theory, the device can diagnose diseases of various internal organs. However, this method requires accurate facial imaging, which necessitates contact between the face and the device. If many people use the device, there is a risk of disease transmission. In addition, the diagnostic results based on this theory are usually only those of traditional Chinese medicine and cannot be correlated with current Western medical conditions. Furthermore, the high cost prevents this type of device from becoming widespread.

[0074] Tongue diagnosis device: This device needs to capture an image of the user's tongue. Based on the connection between different parts of the tongue and internal organs in Traditional Chinese Medicine (TCM), it can diagnose diseases of various internal organs. However, this method requires accurate capture of tongue features, which necessitates face-to-face contact with the device and tongue extension. If many people use the device, there is a risk of disease transmission. In addition, without guidance, it is sometimes impossible to capture identifiable tongue features. Furthermore, the diagnostic results based on this theory are usually only TCM-based and cannot be correlated with current Western medical conditions. The high cost also prevents this type of device from becoming widespread.

[0075] Pulse diagnosis device: This device needs to collect the user's pulse information and, based on the connection between different parts of the pulse and internal organs in traditional Chinese medicine theory, determine the diseases of various internal organs. However, this method requires precise acquisition of pulse characteristics, and sometimes clear pulse information cannot be obtained without guidance. In addition, the diagnosis results based on this theory are usually only TCM theoretical results and cannot be correlated with current Western medicine diseases. Furthermore, the cost is high, which prevents this type of device from becoming widespread.

[0076] Electronic pulse hand diagnostic instrument: This requires collecting the user's current body electronic pulse information through the hand on the device. The diagnostic results obtained in this way cannot be correlated with current Western medicine symptoms, and the cost is high, which prevents this type of device from becoming widespread.

[0077] See Figure 1 This application provides a health check system based on palm information, including an image acquisition device and an information processing device.

[0078] Image acquisition device 110, used to acquire an image of the user's hand;

[0079] The information processing device 120 processes the user's hand image, extracts image feature points, identifies the health status of the internal organs and / or limbs corresponding to each image feature point according to a first preset rule, and displays the symptom names corresponding to a second preset rule.

[0080] The health check system based on palm information provided in this embodiment acquires images of the user's hands through an image acquisition device 110. The user's hands do not need to come into contact with the information acquisition device, which greatly reduces the risk of infection. Since it only requires image acquisition, the hardware cost is relatively low. At the same time, since diagnosis is made based on the user's hand characteristics, the results can be more similar to those of Western medicine, making it easier for ordinary users to accept.

[0081] In one alternative implementation, the first prediction rule is a rule based on traditional Chinese medicine theory; the second prediction rule is a rule based on Western medical symptoms.

[0082] The image acquisition device 110 is a high-definition camera that acquires images of the user's two hands. The information processing device 120 processes these hand images and extracts corresponding image feature points. Based on traditional Chinese medicine theory, it identifies the health status of the internal organs and limbs corresponding to each feature point and displays the names of the diseases associated with Western medicine symptoms. The diagnostic results can be displayed directly on the information processing device or via a mobile phone or tablet.

[0083] See Figure 2The second aspect of this application provides a health check system based on palm information, including a near-end device 210 and a remote server 220.

[0084] The near-end device 210 is used to acquire an image of the user's hand using a camera;

[0085] The remote server 220 obtains symptom diagnosis results and recommended treatment plans based on the image feature point information.

[0086] The near-end device 210 or the remote server 220 processes the user's hand image and extracts the image feature point information.

[0087] The health check system based on palm information provided in this embodiment acquires images of the user's hands through a camera in the near-end device 210. The user's hands do not need to come into contact with the information collection device, which greatly reduces the risk of infection. Since only a camera is needed to acquire images of the user's hands, the hardware cost is relatively low. At the same time, since diagnosis is made based on the user's hand characteristics, the results can be more similar to those in Western medicine, making it easier for ordinary users to accept.

[0088] In one alternative implementation, the remote server 220 also sends the symptom diagnosis result information or the recommended treatment plan information to the near-end device 210, and the near-end device 210 displays the symptom diagnosis result information or the recommended treatment plan information.

[0089] In one alternative implementation, the remote server 220 also sends the symptom diagnosis result information or the recommended treatment plan information to the mobile terminal, and the mobile terminal displays the symptom diagnosis result information or the recommended treatment plan information accordingly.

[0090] See Figure 3 In one alternative implementation, the proximal device includes a display screen for displaying information based on the symptom diagnosis results or the recommended treatment plan.

[0091] In one alternative implementation, the near-end device includes a touchscreen and a keyboard, the touchscreen and the keyboard being used to acquire user information.

[0092] In one alternative implementation, the near-end device includes a speaker; the speaker is used to prompt the user and to interact with the user.

[0093] See Figure 4 One alternative implementation is characterized in that the proximal device is a health checkup device.

[0094] The user inserts their palm into the health check device, performs the operation according to the first prompt and changes to a valid palm, changes to a different palm according to the second prompt, enters relevant user information, and waits for the result according to the third prompt or obtains the diagnostic identification information prompted by the device using a mobile terminal. The first prompt is to prompt the user to insert a valid palm, the second prompt is to prompt the user to change to a different palm, the third prompt is to prompt the user to wait for the diagnostic result, and the diagnostic identification information is information that allows the user to view the diagnostic result through a mobile terminal, such as a QR code.

[0095] The health checkup device detects whether a user's palm is inserted. If no palm is detected, it issues the first prompt message. If a palm is detected, it takes a photo or video to determine if the palm is valid. If the palm is invalid, it issues the first prompt message again; if the palm is valid, it issues the second prompt message. After obtaining valid palm information for both of the user's hands, the device processes the user's hand images and issues the third prompt message, displays the user's diagnostic identifier information, and shows the diagnostic information and recommended treatment plan. The health checkup device processes the user's hand images by directly sending the images to a remote server, or by performing preliminary processing on the hand images before sending them to the remote server.

[0096] The remote server preprocesses the user's hand image after processing by the health checkup device, performs a disease knowledge graph comparison query based on the preprocessed image to obtain symptom diagnosis information, retrieves the corresponding recommended treatment plan information from the background operation system based on the symptom diagnosis information, and sends the symptom diagnosis information and the recommended treatment plan information to the health checkup device or mobile terminal. The remote server can perform preprocessing directly based on the hand image, or it can perform preprocessing based on the hand image initially processed by the health checkup device.

[0097] The mobile terminal obtains the user's diagnostic identification information, retrieves the symptom diagnosis result information and the recommended treatment plan information from the remote server based on the user's diagnostic identification information, and displays the symptom diagnosis result information and the recommended treatment plan information.

[0098] In one alternative implementation, the mobile terminal is a mobile phone or a tablet device.

[0099] In conjunction with one of the alternative implementation methods, processing the user's hand image to extract the image feature point information includes:

[0100] Process at least one of the following factors in the hand image: whether gloves are worn, whether the palm is in the detection area, whether the palm is left or right or right, whether the fingers are bent or spread;

[0101] An affine transformation is performed on the user's hand image to obtain the skin mask and extract the palm contour; the palm contour is the image feature point information.

[0102] One alternative implementation involves detecting the texture information corresponding to the coordinates of 21 preset key points on the palm to determine whether the hand in the hand image is wearing gloves.

[0103] Combination Figure 5 As shown, the center point is obtained by weighted average of the coordinates of 21 preset key points of the palm. The radius is then obtained by taking the average distance from the center point to the vertex of the convex hull of the palm. The rectangular area where the palm is located is calculated and compared with the detection area to determine whether the palm is located in the detection area.

[0104] Combination Figure 6 As shown, let the first, second, and third horizontal lines of the fingers in the palm and the fingertip be points a, b, c, and d, respectively. Then, let points a, b, c, and d form a trapezoid. Determine whether the finger is bent by judging the ratio of the length ad to the length abcd. When the finger is bent in the direction of the shot (towards the screen), it is difficult to determine whether the finger is bent by the aforementioned method due to perspective. Therefore, we need to combine the perspective relationship in space to determine whether the finger is bent.

[0105] Combination Figure 7 As shown, the ratio of the distance between the first transverse crease (distance between ld and rd) and the distance between the second transverse crease (distance between lu and ru) of two adjacent fingers in the palm determines whether the fingers are spread apart.

[0106] Combination Figure 8 As shown, the affine transformation of the user's hand image, the extraction of skin mask and palm contour, includes: determining the transformation center through the hand image, and calculating the affine transformation matrix through the transformation center and the following formula;

[0107]

[0108] Then, an affine transformation is performed based on the affine transformation matrix to obtain the first processed image, which is a more standard pose.

[0109] Combination Figure 9 As shown, the first processed image is converted to a preset color space to obtain a second processed image. This is because changes in lighting conditions can alter the color distribution in the RGB color space, making gesture recognition and analysis more difficult. Therefore, by converting the first processed image to a new color space, a second processed image is obtained. Figure 9 The second processed image shown can reduce the impact of lighting changes on gesture recognition and improve the robustness of recognition.

[0110] Furthermore, image contrast enhancement can improve the visual quality, detail visibility, and information extraction capabilities of an image, making it more attractive and readable. This is primarily achieved by adjusting the pixel value distribution of the image to make it closer to a uniform distribution. Specifically, enhancing image contrast involves adjusting the pixel value distribution of the second-processed image using the following formula to obtain the third-processed image;

[0111]

[0112] Where cdf(v) represents the cumulative distribution function, cdf min The minimum non-zero value of the cumulative distribution function, M×N gives the number of pixels in the image, and L is the number of gray levels used.

[0113] Furthermore, Gaussian noise is a common type of noise that can cause images to become blurry or distorted. To reduce the impact of Gaussian noise, linear smoothing filtering is widely used. Specifically, linear smoothing filtering is a weighted averaging process applied to the entire image, where the value of each pixel is obtained by weighting its own value and the values ​​of other pixels in its neighborhood. This denoising process is then applied to the third-processed image to obtain the fourth-processed image, specifically adjusted using the following formula;

[0114]

[0115] Combination Figure 10 As shown, the fourth processed image is subjected to image binarization processing, that is, the pixel value is divided into two threshold regions, such as black and white, to obtain the palm outline.

[0116] One alternative implementation, wherein obtaining symptom diagnosis results and recommended treatment plan information based on the image feature point information, includes:

[0117] Key point localization and zone localization of the palm outline;

[0118] The key point localization based on the palm contour includes: using 4 key points at the base of the fingers, 1 key point above the wrist crease, 16 key points in the finger area, and 16 key points in the center area of ​​the palm to locate the key points of the palm contour.

[0119] By acquiring the palm outline and combining it with 21 key points on the palm, including coordinates of fingertips and bases of fingers, preliminary palm localization and analysis can be achieved. To further improve localization accuracy and enhance feature representation capabilities, the algorithm uses a pre-defined mathematical method to expand the number of key points to 37. Compared to the 21 key points that focus more on the fingers, these expanded key points more comprehensively cover the entire palm area. By increasing the number of key points, more precise palm localization can be achieved, thereby enabling higher-level palm feature detection and analysis.

[0120] like Figure 11 As shown, the four key points located at the base of the fingers and the key point above the wrist crease play the following main roles in palm positioning:

[0121] (1) Location of certain areas of the palm: These 5 key points can be used to locate certain areas of the palm, for example... Figure 11 The area indicated by the red circle. The four key points at the base of the finger, combined with other key point information, enable precise localization of this area; while the key point at the wrist crease can be used alone for localization of the area near the wrist.

[0122] (2) Determine palm parameters: These 5 key points can determine some basic parameters of the palm, such as the proportion and range of the palm, which can be better applied to the division of the palm area;

[0123] (3) Palm region division: Key points on the fingers alone are insufficient for dividing the palm region. Therefore, these 5 coordinate points play a crucial role in palm region division. When used in conjunction with other key points and palm parameters, they provide key information for palm region positioning and division, and allow for further division of the palm according to the specific project requirements.

[0124] like Figure 12 As shown, the 16 key points located in the finger area play the following main roles in palm positioning:

[0125] (1) Finger area localization and detection: The key points located in the finger area can accurately locate the finger joints and capture the bending of the fingers, the rotation of the palm and the opening and closing of the hand, thereby realizing the localization and detection of the finger area.

[0126] (2) Determine finger parameters: By analyzing key points between adjacent fingers, finger parameters such as finger width and length can be obtained. These parameters help to accurately locate the finger area and provide more accurate gesture recognition and finger movement analysis.

[0127] like Figure 13As shown, although 21 key points can locate most areas of the palm, there are still some drawbacks. Figure 13 The area where 21 points cannot be located is marked in blue:

[0128] (1) Lack of precise palm localization: 21 points focuses more on the localization of the finger area, but lacks more precise localization of the palm area, as shown in the blue detection box in the figure, making it difficult to perform more refined palm area segmentation operations.

[0129] (2) Lack of applicability: The coordinates of 21 points are singular and do not include any information about the palm, such as finger width, palm width, etc., which is detrimental to subsequent processing.

[0130] Therefore, 16 additional key points were added to address the aforementioned 21 key shortcomings, bringing the total number of key points to 37. These expanded key points are primarily located in the central area of ​​the palm, compensating for the limitation of the original 21 key points which only targeted the finger area, and further enriching the palm information.

[0131] like Figure 14 As shown, the palm contour is divided into multiple detection regions based on key point localization. Each detection region is then deeply partitioned and localized, extending key points to their corresponding key areas. This process not only allows for a more accurate understanding of the palm's structure and features but also provides a solid foundation for further analysis. More importantly, considering the differences in hand size among individuals, the region size is adaptively adjusted for different hand sizes. This personalized adjustment ensures the algorithm's good applicability and accuracy across various populations, providing more reliable and precise support for palm feature recognition and analysis.

[0132] The system detects and assesses the symptoms in the multiple detection areas, and obtains symptom diagnosis results and recommended treatment plans.

[0133] One alternative implementation method, wherein detecting the disease in the plurality of detection areas includes:

[0134] The images of the multiple detection areas are converted to a strong color gamut space, and color features are extracted from the images;

[0135] The second moment of the color feature is obtained using the following formula;

[0136]

[0137]

[0138]

[0139] The corresponding color directionality characteristics are then obtained using the following formula;

[0140]

[0141] Where a, b, and c are the composite expressions for the zeroth moment, first moment, and second moment of the matrix, respectively.

[0142] In one optional embodiment, to display the judgment results more intuitively, the results are visualized using density as the standard.

[0143] In one optional embodiment, to facilitate data transmission, the obtained image feature coordinate data is encrypted in the form of a string and then output to the remote diagnostic system for identification and judgment processing via the device.

[0144] Those skilled in the art will understand that, unless specifically stated otherwise, the term "comprising" as used in this specification means the presence of the stated features, integers, steps, operations, parts, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, parts, components, and / or groups thereof. It should be understood that when we say a part is "connected" to another part, it can be directly connected to the other part, or there may be intermediate parts. The term "and / or" as used herein includes all or any unit and all combinations of one or more associated listed items.

[0145] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional elements and circuits is merely an example. In practical applications, the above functions can be assigned to different functional elements and circuits as needed, that is, the internal structure of the circuit can be divided into different functional elements or circuits to complete all or part of the functions described above. The functional elements and circuits in the embodiments can be integrated into one processing element, or each element can exist physically separately, or two or more elements can be integrated into one element. The integrated element can be implemented in hardware or as a software functional element. Furthermore, the specific names of the functional elements and circuits are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the elements and circuits in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0146] Throughout this specification, references to "various embodiments," "in embodiments," "one embodiment," or "implementation," etc., mean that a particular feature, structure, or characteristic described with respect to an embodiment is included in at least one embodiment. Therefore, the appearance of the phrases "in various embodiments," "in some embodiments," "in one embodiment," or "in embodiments," etc., in appropriate places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any suitable manner in one or more embodiments. Thus, a particular feature, structure, or characteristic shown or described with respect to one embodiment may be combined, in whole or in part, with features, structures, or characteristics of one or more other embodiments without presuming that such a combination is not illogical or nonfunctional.

[0147] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0148] Those skilled in the art will recognize that the elements and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0149] In the embodiments provided by this invention, it should be understood that the disclosed circuits / terminal devices and methods can be implemented in other ways. For example, the circuit / terminal device embodiments described above are merely illustrative. For instance, the division of circuits or components is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple components or circuits may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of circuits or components may be electrical, mechanical, or other forms.

[0150] The components described as separate parts may or may not be physically separate. The components shown as elements may or may not be physical elements; that is, they may be located in one place or distributed across multiple network elements. Some or all of the components can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] Furthermore, the functional elements in the various embodiments of the present invention can be integrated into a single processing element, or each element can exist physically separately, or two or more elements can be integrated into a single element. The integrated elements described above can be implemented in hardware or as software functional elements.

[0152] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0153] If the integrated circuit / component is implemented as a software functional element and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or circuit capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0154] The above description is only a partial embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A health checkup system based on palm information, characterized in that, include: Near-end devices are used to capture images of the user's hands using a camera; The remote server obtains symptom diagnosis results and recommended treatment plans based on the image feature point information. The near-end device or the remote server processes the user's hand image and extracts the image feature point information; The process of processing the user's hand image and extracting the image feature point information includes: Process at least one of the following factors in the hand image: whether gloves are worn, whether the palm is in the detection area, whether the palm is left or right or right, whether the fingers are bent or spread; An affine transformation is performed on the user's hand image to obtain the skin mask and extract the palm contour; the palm contour is the image feature point information; The texture information corresponding to the coordinates of 21 preset key points on the palm is detected to determine whether the hand in the hand image is wearing gloves; The center point is obtained by weighted average of the coordinates of 21 preset key points of the palm. The radius is obtained by taking the average distance from the center point to the vertex of the convex hull of the palm. The rectangular area where the palm is located is calculated and compared with the detection area to determine whether the palm is located in the detection area. Let the first, second, and third horizontal creases of the fingers in the palm and the fingertip be points a, b, c, and d, respectively. Then, make a trapezoid with points a, b, c, and d. Determine whether the fingers are bent by judging the ratio of the length of ad to the length of abcd. When the direction of the bend of the fingers is towards the shooting direction, combine the perspective relationship in space to determine whether the fingers are bent. The ratio of the distance between the first and second transverse creases of two adjacent fingers in the palm determines whether the fingers are spread apart. The process of performing affine transformation on the user's hand image, obtaining skin mask and extracting palm contour includes: determining the transformation center through the hand image, and calculating the affine transformation matrix through the transformation center and the following formula; Then, perform an affine transformation based on the affine transformation matrix to obtain the first processed image; The first processed image is converted to a preset color space to obtain a second processed image. The pixel value distribution of the second processed image is adjusted using the following formula to obtain a third processed image. in Represents the cumulative distribution function. The minimum non-zero value of the cumulative distribution function. Given the number of pixels in the image, The number of gray levels used; The third processed image is subjected to noise reduction to obtain the fourth processed image, which is specifically adjusted using the following formula; The fourth image is binarized to obtain the palm outline; The step of obtaining symptom diagnosis results and recommended treatment plans based on the image feature point information includes: Key point localization and zone localization of the palm outline; Key point localization based on the palm contour includes: using 4 key points at the base of the fingers, 1 key point above the wrist crease, 16 key points in the finger area, and 16 key points in the center area of ​​the palm to locate the key points of the palm contour. Based on the key points of the palm contour, the palm contour is divided into multiple detection areas. The system detects and assesses the symptoms in the multiple detection areas to obtain symptom diagnosis results and recommended treatment plans. The detection of the disease in the multiple detection areas includes: The images of the multiple detection areas are converted to a strong color gamut space, and color features are extracted from the images; The second moment of the color feature is obtained using the following formula; The corresponding color directionality characteristics are then obtained using the following formula; in These are the composite expressions for the zeroth moment, first moment, and second moment of the matrix, respectively.

2. The health checkup system based on palm information as described in claim 1, characterized in that, The remote server also sends the symptom diagnosis results or the recommended treatment plan information to the near-end device or mobile terminal. The near-end device displays the symptom diagnosis results or the recommended treatment plan information, and the mobile terminal displays the symptom diagnosis results or the recommended treatment plan information.

3. The health checkup system based on palm information as described in claim 2, characterized in that, The proximal device includes a touchscreen, a keyboard, a speaker, and / or a display screen. The touchscreen and the keyboard are used to acquire user information. The speaker is used to prompt the user and interact with the user. The display screen is used to display information based on the symptom diagnosis results or the recommended treatment plan.

4. The health checkup system based on palm information as described in any one of claims 1-3, characterized in that, The near-end device is a health checkup device; The user inserts their palm into the health check device, performs the operation according to the first prompt and changes to a valid palm, changes to a different palm according to the second prompt and enters relevant user information, and waits for the result according to the third prompt or obtains the diagnostic identification information prompted by the device using a mobile terminal. The health checkup device detects whether a user's palm is inserted. If no palm is detected, the device issues the first prompt message. If a palm is detected, the device takes a picture or video to check if it is a valid palm. If the palm is invalid, the device issues the first prompt message. If the palm is valid, the device issues the second prompt message. After obtaining valid information about the user's palms, the device processes the user's hand images and issues the third prompt message. The device also displays the user's diagnostic identification information, diagnostic information, and recommended treatment plan. The remote server preprocesses the user's hand image processed by the health checkup device, performs a disease knowledge graph comparison query based on the preprocessed image to obtain symptom diagnosis information, obtains the corresponding recommended treatment plan information based on the symptom diagnosis information, and sends the symptom diagnosis information and the recommended treatment plan information to the health checkup device or mobile terminal. The mobile terminal obtains the user's diagnostic identification information, retrieves the symptom diagnosis result information and the recommended treatment plan information from the remote server based on the user's diagnostic identification information, and displays the symptom diagnosis result information and the recommended treatment plan information.

Images