Picture identification method, system, device, medium and program

Abstract

The invention discloses a picture recognition method, system and device, a medium and a program, and belongs to the technical field of medical image analysis. The method comprises the steps that a current nail cover picture of a user is acquired, the current nail cover picture is preprocessed, and preprocessing comprises graph enhancement and standardization processing; inputting the pre-processed nail cover picture into a target neural network model, and outputting a corresponding picture recognition result by the target neural network model; and generating a current hemoglobin value result of the user according to the identification result, and generating a corresponding prompt instruction according to the current hemoglobin value result to the client for display. According to the method, blood drawing is not needed, the pain and fear of a patient are relieved, the detection convenience is improved, the waiting cost and time of the patient are reduced, and the applicability is high.

Description

Technical Field

[0001] This application relates to the technical field of medical image analysis, and particularly relates to a picture recognition method, system, device, medium and program. Background Art

[0002] Hemoglobin is a special protein that transports oxygen in red blood cells and makes the blood red. Hemoglobin determination is of great significance in clinical diagnosis and plays a key role in the diagnosis and treatment of various diseases such as anemia and leukemia.

[0003] Traditional hemoglobin detection methods usually require collecting blood samples for laboratory analysis, which is not only time-consuming and costly, but may also cause inconvenience and pain to patients. Summary of the Invention

[0004] The purpose of this application is to provide a picture recognition method, system, device, medium and program, which avoids the discomfort caused by traditional blood drawing measurement, is more easily accepted by patients, and solves the problems raised in the above background art.

[0005] To achieve the above purpose, this application provides the following technical solutions: The first aspect embodiment of this application provides a picture recognition method, including the following steps: S1. Obtain the current fingernail picture of the user, and preprocess the current fingernail picture, where the preprocessing includes graphic enhancement and normalization processing; S2. Input the preprocessed fingernail picture into the target neural network model, and the target neural network model outputs the corresponding picture recognition result, where the target neural network model includes a first neural network and a second neural network. The first neural network is used to extract the fingernail picture features to generate a feature vector set, and the second neural network is used to capture the local features and global features of the fingernail picture, and analyze the fused feature vector set and the local features and global features of the picture to generate the corresponding picture recognition result; S3. Generate the current hemoglobin value result of the user according to the recognition result, and generate the corresponding prompt instruction to the client for display according to the current hemoglobin value result.

[0006] Further, the generating the current hemoglobin value result of the user according to the recognition result includes: obtaining the pixel values corresponding to the picture in multiple color channels in the recognition result; calculating the average value of the pixel values of each color channel; calculating the corresponding deviation value according to the pixel value and the average value of the pixel values, and sorting the deviation values according to the target rule to generate a deviation value sequence; dividing the fingernail picture into multiple target regions according to the deviation value sequence, and generating the current hemoglobin value result of the user according to the multiple target regions.

[0007] Further, dividing the fingernail image into multiple target regions according to the deviation value sequence includes: determining the number of target pixels to be screened according to the product of a preset proportional gradient and the total number of pixels; screening the deviation values in the deviation value sequence that meet the number of target pixels, and identifying the pixel points corresponding to the deviation values to generate a node cluster; and dividing the fingernail image into multiple target regions according to the proportion in the node cluster and the preset proportional gradient.

[0008] Further, generating the current hemoglobin value result of the user according to the multiple target regions includes: generating a node contour sequence according to all node clusters of each color channel in the multiple target regions; identifying whether there is a closed region that can be formed in the node contour sequence; if there is no such closed region, calculating the coincidence degree of each node in all channels, and fitting the nodes with a coincidence degree greater than a preset threshold to generate a closed region contour; using the closed region as the effective definition range of the target fingernail, and comparing the effective definition range with the normal value range in the healthy state of the fingernail to generate the current hemoglobin value result of the user.

[0009] Further, generating a corresponding prompt instruction to be displayed on the client according to the current hemoglobin value result includes: if the current hemoglobin value is within a preset numerical range, issuing a normal instruction; if the current hemoglobin value is higher than the preset numerical range, issuing an instruction indicating a higher hemoglobin value; if the current hemoglobin value is lower than the preset numerical range, issuing an instruction indicating a lower hemoglobin value.

[0010] Further, before obtaining the current fingernail image of the user, it includes: identifying the login data input by the user to the client; registering the user's account according to the login data, where the login data includes identity information and pathological information; and associating the account information, client identifier, and identity identifier and storing them in the database.

[0011] The second aspect of the present application provides an image recognition system, which is characterized by including: an acquisition module for acquiring the current nail image of the user and preprocessing the current nail image, where the preprocessing includes image enhancement and normalization; a processing module for inputting the preprocessed nail image into a target neural network model, and the target neural network model outputs a corresponding image recognition result, where the target neural network model includes a first neural network and a second neural network. The first neural network is used to extract nail image features to generate a feature vector set, and the second neural network is used to capture local and global features of the nail image, analyze the fused feature vector set and local and global features of the image to generate a corresponding image recognition result; a generation module for generating the current hemoglobin value result of the user according to the recognition result, and generating a corresponding prompt instruction to the client for display according to the current hemoglobin value result.

[0012] The third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes the program to perform the image recognition method as described in the above embodiments.

[0013] The fourth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, and the program is executed by a processor to perform the image recognition method as described in the above embodiments.

[0014] The fifth aspect of the present application provides a computer program product, including a computer program or instruction, which is characterized in that when the computer program or instruction is executed, it realizes the image recognition method as described in the above embodiments.

[0015] Compared with the prior art, the beneficial effects of the present application are as follows: In the present application, by loading the image recognition server on the examiner's client, ensuring that the patient takes a picture of the nail with the client, extracting image features based on graph convolutional neural network and Transformer, fusing the features and then inputting them into the Transformer network for recognition, thereby judging whether the current hemoglobin value is abnormal; if the result shows that the value is within the normal range, a normal prompt is sent to the examiner's client; otherwise, an abnormal prompt is sent, and then further in-depth inspection is carried out. Thus, the present application reduces the pain and fear of patients without blood drawing, is especially suitable for children, the elderly, and those with blood phobia, and improves the convenience of detection, reduces the need for professionals, reduces the cost and time of patients, and is applicable to families, community medical points, and remote areas. Description of the Drawings

[0016] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of embodiments in conjunction with the accompanying drawings, where: Figure 1 It is a flowchart of the picture recognition method provided by the present application; Figure 2 It is a schematic diagram of the process of a picture recognition method provided by the present application; Figure 3 It is a block diagram of the picture recognition device provided by the present application; Figure 4 It is a composition diagram of a picture recognition system provided by the present application; Figure 5 It is a structural diagram of the electronic device provided by the present application. Detailed implementation manners

[0017] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0018] To avoid the discomfort caused by traditional blood drawing measurements and obtain measurement results more conveniently and quickly, especially in the case of multiple or continuous measurements, it is very practical.

[0019] Specifically, Figure 1 It is a schematic diagram of the process of a picture recognition method provided by an embodiment of the present application.

[0020] As Figure 1 shown, the picture recognition method includes the following steps: In step S101, the current fingernail picture of the user is obtained, and the current fingernail picture is preprocessed, where the preprocessing includes graphic enhancement and normalization processing.

[0021] It can be understood that the embodiments of the present application can obtain the current fingernail picture of the user, preprocess the current fingernail picture, so as to improve the clarity of the fingernail picture, and further improve the accuracy of the fingernail picture trait recognition processing.

[0022] It should be noted that the graphic enhancement processing mainly adjusts brightness and contrast, color balance and channel enhancement, noise suppression, background cropping and region positioning, and sharpening processing, which are used to improve the image quality, highlight the key features of the nail area, and reduce noise and background interference; the standardization processing mainly includes unified size and resolution, color channel standardization, pixel value range standardization, color deviation calculation, and data enhancement, which are used to unify the size, color distribution, and statistical characteristics of the input images, eliminate the influence of device differences and shooting conditions, and ensure the consistency of the model input.

[0023] In the embodiment of the present application, before obtaining the current nail picture of the user, it includes: identifying the login data input by the user to the client; registering the user's account according to the login data, where the login data includes identity information and pathological information; associating the account information, client identifier, and identity identifier and storing them in the database.

[0024] It can be understood that the embodiment of the present application can identify the login data input by the user to the client; register the user's account according to the login data, where the login data includes identity information and pathological information; associate the account information, client identifier, and identity identifier and store them in the database, so as to facilitate successful registration when the user logs in for the first time and directly obtain relevant information when logging in again.

[0025] It should be noted that the identity information includes: name, gender, age, ID number, address, contact information, etc.; the pathological information includes: the name of the hospital where the sample is sent for inspection, the inspection date, the responsible doctor, and the reason for seeing a doctor, etc.

[0026] In step S102, the preprocessed nail picture is input into the target neural network model, and the target neural network model outputs the corresponding picture recognition result. Among them, the target neural network model includes a first neural network and a second neural network. The first neural network is used to extract the nail picture features to generate a feature vector set, and the second neural network is used to capture the local features and global features of the nail picture, analyze the fused feature vector set and the local features and global features of the picture to generate the corresponding picture recognition result.

[0027] Among them, the first neural network is a GCN neural network model, and the second neural network is a Transformer neural network.

[0028] It can be understood that the embodiment of the present application can input the preprocessed nail picture into the target neural network model, and the target neural network model outputs the corresponding picture recognition result. Without blood drawing, it reduces the pain and fear of patients, especially suitable for children, the elderly, and those with blood phobia. Moreover, it improves the convenience of detection, reduces the need for professional personnel, and reduces the cost and time of patients, which is suitable for families, community medical points, and remote areas.

[0029] In step S103, the current hemoglobin value result of the user is generated according to the recognition result, and a corresponding prompt instruction is generated according to the current hemoglobin value result and displayed on the client.

[0030] It can be understood that in the embodiment of the present application, the current hemoglobin value result of the user can be generated according to the recognition result, and a corresponding prompt instruction is generated according to the current hemoglobin value result and displayed on the client, directly feeding back the result to the user to improve the user experience.

[0031] In the embodiment of the present application, generating the current hemoglobin value result of the user according to the recognition result includes: obtaining the pixel values corresponding to the picture in multiple color channels in the recognition result; calculating the average pixel value of each color channel; calculating the corresponding deviation value according to the pixel value and the average pixel value, and sorting the deviation values according to the target rule to generate a deviation value sequence; dividing the nail picture into multiple target regions according to the deviation value sequence, and generating the current hemoglobin value result of the user according to the multiple target regions.

[0032] It can be understood that in the embodiment of the present application, by analyzing the deviation characteristics of the nail color channels and combining with a machine learning model, non-invasive hemoglobin value assessment can be realized, improving work efficiency.

[0033] In the embodiment of the present application, dividing the nail picture into multiple target regions according to the deviation value sequence includes: determining the number of target pixels to be screened according to the product of the preset proportional gradient and the total number of pixels; screening the deviation values in the deviation value sequence that meet the number of target pixels, and identifying the pixel points corresponding to the deviation values to generate a node cluster; dividing the nail picture into multiple target regions according to the ratio in the node cluster and the preset proportional gradient.

[0034] Among them, the preset proportional gradient can be set according to actual needs and is not specifically limited.

[0035] It can be understood that in the embodiment of the present application, through the region division driven by the proportional gradient, the accurate mapping from pixel-level deviation analysis to pathological correlation is realized, providing a reliable technical basis for non-invasive hemoglobin detection, and its hierarchical design takes into account both detection sensitivity and specificity.

[0036] In the embodiment of the present application, generating the current hemoglobin value result of the user according to the multiple target regions includes: generating a node contour sequence according to all node clusters of each color channel in the multiple target regions; identifying whether there is a closed region in the node contour sequence; if there is no closed region, calculating the coincidence degree of each node in all channels, and fitting the nodes with a coincidence degree greater than the preset threshold to generate a closed region contour; using the closed region as the effective definition range of the target nail cover, and comparing the effective definition range with the normal numerical range in the healthy state of the nail cover to generate the current hemoglobin value result of the user.

[0037] It can be understood that in the embodiments of the present application, the contour sequence can identify the regions with significant color deviation. Since the contours of different channels reflect different spectral characteristics related to hemoglobin, through multi-channel closed region definition and feature fusion, an accurate mapping from the nail image to the hemoglobin value is achieved, which has the advantages of non-invasiveness, convenience, and low cost.

[0038] In the embodiments of the present application, a corresponding prompt instruction is generated according to the current hemoglobin value result and displayed on the client, including: if the current hemoglobin value is within the preset numerical range, a normal instruction is issued; if the current hemoglobin value is higher than the preset numerical range, an instruction indicating a higher hemoglobin value is issued; if the current hemoglobin value is lower than the preset numerical range, an instruction indicating a lower hemoglobin value is issued.

[0039] Among them, the preset numerical range can be set according to actual needs without specific limitation.

[0040] It can be understood that the monitoring and feedback mechanism based on the hemoglobin value in the embodiments of the present application helps to detect health problems early, take intervention measures in time, and improve the user experience.

[0041] According to the image recognition method proposed in the embodiments of the present application, the image recognition server is installed on the examiner's client. After ensuring that the patient uses the client to take a picture of the fingernail, the preprocessed fingernail image is input into the target neural network model, and the target neural network model outputs the corresponding image recognition result, thereby determining whether the current hemoglobin protein value is abnormal; if the result shows that the value is within the normal range, a normal prompt is sent to the examiner's client; otherwise, an abnormal prompt is sent, and then further in-depth inspection is carried out. Thus, the present application does not require blood drawing, which reduces the pain and fear of patients, is especially suitable for children, the elderly, and those with blood phobia, and improves the convenience of detection, reduces the need for professional personnel, reduces the cost and time of patients, and is applicable to families, community medical points, and remote areas.

[0042] Next, in combination with Figure 2 The embodiments of the image recognition method of the present application will be elaborated in detail as follows: S1. Install the image recognition server on the examiner's client, thereby logging in and running the image recognition system; after successfully installing and running the system, the examiner also needs to register an identity account and input personal real information and the current medical examination situation.

[0043] Specifically, after the examiner logs in to the image recognition server through the client, the registration process of identity information needs to be completed. For example, use the client number as the login account and set a password. Then, fill in personal information and relevant details of the current medical examination in detail, such as name, gender, age, ID number, address, contact information, etc. At the same time, information such as the name of the submitting hospital, submission date, responsible doctor, and reason for seeing a doctor also needs to be provided to ensure the authenticity during case review.

[0044] S2. Collect the current nail image of the examiner through the client camera function and upload the nail image to the image adjustment unit for image preprocessing to ensure the clarity of the nail image; Specifically: Through the methods of image enhancement and normalization processing, ensure that the images of each finger are consistent in terms of brightness, contrast, and color, and reduce the noise in the image, especially the interference between the finger skin and the background, so that the nail area is more prominent; avoid the differences between the images of different fingers from affecting the model performance. Thereby improving the clarity of the nail image and further enhancing the accuracy of the nail image trait recognition process.

[0045] S3. Extract medical features from the nail image based on GCN (Graph Convolutional Network) and Transformer, and fuse the extracted features and then input them into the Transformer network for classification.

[0046] Specifically: By constructing a GCN model and encoding the processed nail image to obtain a set of feature vectors; then input these feature vectors into the GCN model for training, thereby outputting the image features of the nail. Thus, the nail images of multiple fingers can be regarded as a kind of "structured data". Taking the nail image of each finger as a node, edges can be established between these nodes, which can make the left and right adjacent fingers (such as the index finger and the middle finger) have a stronger association. Subsequently, through the layer-by-layer transmission and aggregation process of the GCN model, capture the mutual relationship between fingers and the mutual relationship of feature propagation, so as to obtain a richer feature representation.

[0047] At the same time, construct a Transformer model, divide the nail images of multiple fingers into multiple overlapping window blocks to generate several small nail images; these image blocks extract the local features of each image block and the global dependence between fingers through the self-attention mechanism of the Transformer model, thereby outputting the feature representation of each image block.

[0048] The features extracted by GCN and Transformer are fused through the concat operation to form a new feature set; this fused feature is used as the input feature set and fed into the Transformer network for classification processing; in the Transformer network, the self-attention mechanism is used to model the features, and the global context information is encoded through multiple self-attention modules to obtain the local features and global context information of the fingernail image. That is, by examining the local features and global context information of the fingernail image to facilitate the judgment of whether the current hemoglobin is abnormal.

[0049] Among them, the pixels or feature representations in the fingernail image are used as the nodes of the graph, and multiple regions are constructed based on all the nodes included in the fingernail image. Specifically: in order to evaluate the hemoglobin content of different fingers through the fingernail image, first, the numerical values of each pixel in each image in the three color channels of red (R), green (G), and blue (B) are extracted.

[0050] Next, the mean value of the pixel values of each color channel is calculated to characterize the overall color distribution of the image. For each pixel, the deviation value from the corresponding channel mean in the R, G, and B channels is calculated respectively to obtain the color deviation of each channel. These deviation values reflect the degree of difference between the pixel color and the overall average color. Subsequently, the deviation values within each channel are sorted by size to form a deviation value sequence. According to the product of the preset proportional gradient and the total number of pixels, the number of target pixels to be screened is determined.

[0051] Select the first several largest deviation values in the deviation value sequence, identify the pixel points corresponding to these deviation values, and mark them as "node clusters". Using these node clusters, multiple regions are divided in the fingernail image, where the node clusters represent image features. Further, in the fingernail image, multiple regions are divided according to the node clusters of each channel and the ratios in the screening ratio gradient list.

[0052] Specifically: in the fingernail image processing, first, the node contours are fitted according to a single node cluster, and these node clusters represent a set of pixel points with significant color changes in the image. Then, for all the node clusters in each color channel, their corresponding node contours are sorted in ascending order of the screening ratio to form a node contour sequence. Next, it is judged whether there are node contours that can enclose a closed area in the node contour sequences of all channels. If there is a closed area, these closed areas are used as the significant areas in the fingernail image. If no closed area is found, the coincidence degree of each node in all channels is calculated.

[0053] Among them, the calculation formula of the coincidence degree is as follows: In the above formula, represents the coincidence degree of the current node in the node contour sequences of all channels. is the total number of node contours of all channels containing this node. And is the total number of node contours in the

[0054] Smooth fitting is performed on the nodes with high coincidence degree to form the contour of the closed area. Finally, these identified closed areas are used as the defined range of the fingernail image. After such processing, the area with significant color change can be more accurately extracted, providing reliable feature information for the subsequent evaluation of hemoglobin content.

[0055] Among them, in S3, by inputting the picture of the examiner's fingernail, it is judged whether the hemoglobin value of the current examiner is abnormal, and the corresponding prompt instruction is given.

[0056] Specifically: The current identity information of the examiner is input through the information input interface, and combined with the feature information of the fingernail picture, it is input into the Transformer network for recognition processing at the same time, so as to obtain the fingernail recognition result; if the recognition result shows that the current hemoglobin value is within the normal range, a normal prompt is sent to the examiner's client, otherwise, an abnormal prompt is sent.

[0057] S4. The examiner takes corresponding measures based on the given prompt instruction.

[0058] In summary, this application does not require blood drawing, reducing the pain and fear of patients, and is especially suitable for children, the elderly, and those with blood phobia. In addition, this application improves the convenience of detection, reduces the need for professional personnel, and reduces the cost and time of patients, and is applicable to families, community medical points, and remote areas.

[0059] Secondly, a picture recognition device according to an embodiment of the present application is described with reference to the accompanying drawings.

[0060] Figure 2 is a block diagram of the picture recognition device according to an embodiment of the present application.

[0061] As Figure 2 shown, the picture recognition device 10 includes: an acquisition module 100, a processing module 200, and a generation module 300.

[0062] Among them, the acquisition module 100 is used to acquire the current nail image of the user and preprocess the current nail image, where the preprocessing includes image enhancement and normalization processing; the processing module 200 is used to input the preprocessed nail image into the target neural network model, and the target neural network model outputs the corresponding image recognition result, where the target neural network model includes a first neural network and a second neural network. The first neural network is used to extract nail image features to generate a feature vector set, and the second neural network is used to capture the local features and global features of the nail image, and analyze the fused feature vector set and the local features and global features of the image to generate the corresponding image recognition result; the generation module 300 is used to generate the current hemoglobin value result of the user according to the recognition result, and generate a corresponding prompt instruction to the client for display according to the current hemoglobin value result.

[0063] It should be noted that the foregoing explanation of the embodiment of the image recognition method also applies to the image recognition device of this embodiment, and will not be repeated here.

[0064] According to the image recognition device proposed in the embodiment of the present application, the image recognition server is installed on the examiner's client. After ensuring that the patient uses the client to take a picture of the nail, the preprocessed nail image is input into the target neural network model, and the target neural network model outputs the corresponding image recognition result, thereby judging whether the current hemoglobin value is abnormal; if the result shows that the value is within the normal range, a normal prompt is sent to the examiner's client; otherwise, an abnormal prompt is sent, and then further in-depth inspection is carried out. Thus, the present application does not require blood sampling, which reduces the pain and fear of patients, especially suitable for children, the elderly and those with blood phobia. Moreover, it improves the convenience of detection, reduces the need for professional personnel, reduces the cost and time of patients, and is applicable to families, community medical points and remote areas.

[0065] Next, it will be combined with Figure 4 The image automatic recognition system of the present application will be elaborated in detail as follows: The image recognition server is used to be installed on the examiner's client, so as to log in and run the image recognition system: after successfully installing and running the system, the examiner also needs to register an identity account and input personal real information and current medical examination conditions. Specifically, after the examiner logs in to the image recognition server through the client, the identity information registration process needs to be completed. Continuing with the above embodiment, for example: the client number can be used as the login account, and a password can be set. Then, fill in personal information and relevant details of the current medical examination in detail, such as name, gender, age, ID number, address, contact information, etc. At the same time, information such as the name of the sending hospital, the sending date, the responsible doctor and the reason for seeing a doctor also need to be provided to ensure the authenticity during case review.

[0066] The image acquisition unit collects images of the examiner's fingernails through the client camera and uploads these images to the image adjustment unit for preprocessing. Continuing from the above embodiment, for example: The examiner logs in and runs an automatic recognition system for fingernail images, authorizes the system to access the client camera. Takes a photo of the fingernail using the client camera, and then uploads and performs preprocessing for clarity improvement to enhance the accuracy of trait recognition.

[0067] The image adjustment unit is used to receive the image information of the examiner's fingernails and perform two necessary steps: image enhancement and normalization processing. It ensures that the images of each finger are consistent in terms of brightness, contrast, and color, avoids the influence of differences between different finger images on the model performance, reduces the noise in the images, especially the interference of finger skin and background, and makes the fingernail area more prominent. Then uploads the processed picture to the image processing unit for image feature extraction; Continuing from the above embodiment, for example: The examiner takes and uploads a picture of the fingernail based on the client camera function. The image adjustment unit crops the redundant background and edges in the fingernail picture to ensure that the fingernail is in the middle position of the picture; and performs clarity processing on the picture, appropriately adjusts the indexes of image brightness, color, and contrast, thereby improving the clarity of the picture and preventing the fingernail picture taken by the photographer from being blurred.

[0068] The information input interface, based on the client logging in and running the image recognition system, is used to input the current hemoglobin value of the examiner and upload the input data information to the image analysis unit; Continuing from the above embodiment, for example: The inspection date is: XX year XX month XX day, and the examiner is: Zhang XX. Based on this, the above information is uploaded to the image processing unit as a recognition reference factor and imported into the Transformer network together with the image features for recognition processing. The image analysis unit serves as a recognition reference factor, thereby improving the accuracy of hemoglobin value judgment.

[0069] The image processing unit is used to receive the processed fingernail picture, perform medical image feature extraction on the processed fingernail picture based on GCN and Transformer, then fuse the features extracted by GCN and Transformer using concat, and input the fused features into the Transformer network for recognition processing and classification, thereby obtaining the local features and global context information of the output fingernail, and thus judging whether the current hemoglobin value is abnormal; The image processing unit includes: a model creation module that creates a GCN model and a Transformer network model based on GCN and Transformer respectively, for extracting feature information of the fingernail image through a dual-channel; an encoding processing module for encoding the processed fingernail image to obtain a set of feature vectors; a model training module for importing the set of feature vectors into the GCN and Transformer network models for training, and then importing the test set into the trained Transformer network model for testing, thereby obtaining the test results of the test set; and a result output module for outputting the fingernail image recognition results obtained by the GCN model and the Transformer network model. Specifically, feature extraction is performed through a dual-channel. For example, GCN and Transformer are used for image feature extraction, and then the extracted features are input into the Transformer through feature fusion for classification. A further description is as follows: a GCN model is constructed, and the processed fingernail image is encoded to obtain a set of feature vectors; then these feature vectors are input into the GCN model for training, thereby outputting the image features of the fingernail. In this method, the fingernail images of multiple fingers can be regarded as a kind of "structured data". Taking the fingernail image of each finger as a node, edges can be established between these nodes, which can make the left and right adjacent fingers (such as the index finger and the middle finger) have a stronger association. Subsequently, through the layer-by-layer transmission and aggregation process of the GCN model, the mutual relationships between fingers and the mutual relationships of feature propagation are captured, thereby obtaining a richer feature representation; at the same time, a Transformer model is constructed, and the fingernail images of multiple fingers are segmented into multiple overlapping window blocks to generate several small fingernail images. These image blocks extract the local features of each image block and the global dependency relationships between fingers through the self-attention mechanism of the Transformer model, thereby outputting the feature representations of each image block; the features extracted by GCN and Transformer are fused through a concat operation to form a new feature set. This fused feature is used as the input feature set and sent into the Transformer network for classification processing.

[0070] An instruction transceiver unit for receiving the fingernail image judgment result transmitted by the image processing unit; and making corresponding medical instructions based on the judgment result. Referring to the above embodiment, for example: the inspection date is: XX year XX month XX day, the examiner is: Zhang XX. If the recognition result shows that the current hemoglobin value is within the normal range, a normal prompt is sent to the examiner's client. Otherwise, an abnormal prompt is sent (the abnormal prompt includes: low hemoglobin value, low, high).

[0071] The instruction sending module makes corresponding appropriate medical guidance instructions based on the judgment result. Specifically: if the judgment result shows that the current hemoglobin value is within the normal range, a normal instruction is issued; if the judgment result shows that the current hemoglobin value is not within the normal range, an abnormal indication is issued (the abnormal prompt includes: low hemoglobin value, low, high).

[0072] The working principle of the above content: By installing the image recognition server on the examiner's client, ensuring that the patient takes pictures of the nail covers of eight fingers (excluding the thumbs) of both hands with the client, extracting image features based on GCN and Transformer, fusing the features and then inputting them into the Transformer network for recognition, thereby judging whether the current hemoglobin value is abnormal; if the result shows that the value is within the normal range, a normal prompt is sent to the examiner's client; otherwise, an abnormal prompt is sent, and then further in-depth examinations are carried out. This application does not require blood drawing, reducing the pain and fear of patients, and is especially suitable for children, the elderly, and those with blood phobia. In addition, this application improves the convenience of detection, reduces the need for professional personnel, reduces the cost and time of patients, and is applicable to families, community medical points, and remote areas.

[0073] Figure 5 The following is a schematic structural diagram of the electronic device provided by the embodiment of the present application. The electronic device may include: A memory 501, a processor 502, and a computer program stored on the memory 501 and executable on the processor 502.

[0074] When the processor 502 executes the program, it implements the image recognition method provided in the above embodiment.

[0075] Further, the electronic device further includes: A communication interface 503 for communication between the memory 501 and the processor 502.

[0076] The memory 501 is used to store a computer program executable on the processor 502.

[0077] The memory 501 may include a high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0078] If the memory 501, the processor 502, and the communication interface 503 are implemented independently, the communication interface 503, the memory 501, and the processor 502 can be interconnected via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 only a thick line is used in Figure 5 , but it does not mean that there is only one bus or one type of bus.

[0079] Optionally, in a specific implementation, if the memory 501, the processor 502, and the communication interface 503 are integrated on a single chip, the memory 501, the processor 502, and the communication interface 503 can communicate with each other through an internal interface.

[0080] The processor 502 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0081] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the above-mentioned picture recognition method is implemented.

[0082] The embodiments of the present application further provide a computer program product, including a computer program or instruction, characterized in that when the computer program or instruction is executed, the above-mentioned picture recognition method is implemented.

[0083] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0084] Although embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present application. The scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for image recognition, characterized in that: The following steps are involved: S101, obtaining a current fingernail picture of a user, and preprocessing the current fingernail picture, wherein the preprocessing includes image enhancement and standardization processing; S102, inputting the preprocessed fingernail image into a target neural network model, the target neural network model outputting a corresponding image recognition result, wherein the target neural network model includes a first neural network and a second neural network, the first neural network is used to extract fingernail image features to generate a feature vector set, the second neural network is used to capture local features and global features of the fingernail image, and analyzing the fused feature vector set and the local features and global features of the image to generate a corresponding image recognition result; S103: Generate a current hemoglobin value result of the user according to the recognition result, and generate a corresponding prompt instruction according to the current hemoglobin value result to display on the client.

2. The image recognition method according to claim 1, characterized in that: Generating the current hemoglobin value result of the user according to the recognition result includes: Get the pixel values ​​corresponding to multiple color channels of the image in the recognition result; Calculate the mean pixel value for each color channel; Calculating a corresponding deviation value according to the pixel value and the pixel value mean, and sorting the deviation values ​​according to a target rule to generate a deviation value sequence; The nail image is divided into a plurality of target areas according to the deviation value sequence, and the current hemoglobin value results of the user are generated according to the plurality of target areas.

3. The image recognition method according to claim 2, characterized in that: The step of dividing the nail image into a plurality of target areas according to the deviation value sequence includes: Determine the number of target pixels to be screened according to the product of the preset proportional gradient and the total number of pixels; Filtering the deviation values ​​in the deviation value sequence that meet the target number of pixels, identifying the pixel points corresponding to the deviation values ​​to generate a node cluster; The nail image is divided into a plurality of target areas according to the node cluster and a ratio in a preset ratio gradient.

4. The image recognition method according to claim 3, characterized in that: The generating the current hemoglobin value result of the user according to the multiple target areas comprises: generating a sequence of node outlines according to all node clusters of each color channel in a plurality of target regions; Identify whether the node outline sequence exists and can enclose a closed area; If the closed area does not exist, the overlap of each node in all channels is calculated, and the closed area outline is generated by fitting the nodes whose overlap is greater than a preset threshold; The closed area outline is used as the effective definition range of the target fingernail, and the effective definition range is compared with the normal value range of the fingernail in a healthy state to generate the user's current hemoglobin value result.

5. The image recognition method according to claim 4, characterized in that: Generate corresponding prompt instructions according to the current hemoglobin value result and display them to the client, including: If the current hemoglobin value is within the preset value range, a normal instruction is issued; If the current hemoglobin value is higher than the preset value range, an instruction for a higher hemoglobin value is issued; If the current hemoglobin value is lower than the preset value range, an instruction for a lower hemoglobin value is issued.

6. The image recognition method according to claim 1, characterized in that: Before obtaining the current fingernail picture of the user, the following steps are included: Identify the login data entered by the user into the client; Registering a user's account according to the login data, wherein the login data includes identity information and pathology information; The account information, client identifier and identity identifier are associated and stored in a database.

7. A picture recognition system, characterized in that: include: An acquisition module, used for acquiring a current fingernail picture of a user, and preprocessing the current fingernail picture, wherein the preprocessing includes image enhancement and standardization processing; A processing module, used for inputting the preprocessed nail image into a target neural network model, and the target neural network model outputs a corresponding image recognition result, wherein the target neural network model includes a first neural network and a second neural network, the first neural network is used for extracting features of the nail image to generate a feature vector set, and the second neural network is used for capturing local features and global features of the nail image, and analyzing the fused feature vector set and the local features and global features of the image to generate a corresponding image recognition result; A generation module is used to generate a current hemoglobin value result of the user according to the recognition result, and to generate a corresponding prompt instruction according to the current hemoglobin value result to display on the client.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image recognition method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the image recognition method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, is used to implement the image recognition method according to any one of claims 1 to 6.

Images