Data interpretation method, device, equipment and storage medium based on image recognition

Abstract

The present invention relates to the field of artificial intelligence, and discloses a data interpretation method, device, equipment and storage medium based on image recognition, which is used to solve the problem of low efficiency in interpreting physical examination reports with different layout contents in the prior art. The method includes: receiving and extracting a physical examination report image in a data interpretation request; identifying the text position of the physical examination result therein, and cropping it to obtain a cropped image set; performing text recognition on the cropped image block in the cropped image set to obtain the text content; and determining the inspection category and inspection result contained in the cropped image block, and obtaining the corresponding standard indicator range; judging whether the inspection result is within the standard indicator range; if not, marking the inspection result and inspection category as abnormal inspection result and abnormal inspection category; retrieving medical information in a medical knowledge graph, and outputting the data interpretation result. In addition, the present invention also relates to blockchain technology, and relevant information of the physical examination report can be stored in the blockchain.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a data interpretation method, device, equipment and storage medium based on image recognition. Background Art

[0002] With the development of social economy, the improvement of people's living standards and the continuous improvement of the basic medical security system for urban and rural residents, people's demand for medical services, especially for health management such as physical examinations, is growing. In order to meet the physical examination needs of different groups of people in society, the hospital has continuously improved the physical examination operation process of medical staff during reform and development, provided a humanized physical examination management method, and added multiple functions such as supporting disease auxiliary diagnosis, health management, and remote consultation.

[0003] In the existing technology, the method of automatically interpreting physical examination reports can only recognize the contents of physical examination reports with the same layout. However, the layout of physical examination reports for different physical examination institutions or different examination items is different. It is necessary to retrain the recognition model for each layout, which makes it difficult to universally interpret the physical examination reports, and further leads to low interpretation efficiency. Summary of the invention

[0004] The main purpose of the present invention is to solve the problem of low efficiency in interpreting physical examination reports with different formats in the prior art.

[0005] The first aspect of the present invention provides a data interpretation method based on image recognition, comprising: receiving a data interpretation request, and extracting a physical examination report image contained in the data interpretation request; calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report image, and cropping the physical examination report image according to the text position to obtain a cropped image set, wherein the cropped image set includes at least one cropped image block; calling a preset optical character recognition model to perform text recognition on the cropped image block to obtain the text content in the cropped image block; determining the inspection category contained in the cropped image block and the inspection result contained in the inspection category according to the text content, and obtaining the corresponding standard indicator range according to the inspection category, wherein the inspection result is at least one; judging whether the inspection result is within the standard indicator range; if not, marking the inspection result as an abnormal inspection result, and marking the inspection category as an abnormal inspection category; searching in a preset medical knowledge graph according to the abnormal inspection result to obtain medical information corresponding to the abnormal inspection category, and outputting a data interpretation result according to the medical information.

[0006] Optionally, in a first implementation method of the first aspect of the present invention, determining the inspection category contained in the cropped image block and the inspection results contained in the inspection category based on the text content, and obtaining the corresponding standard indicator range based on the inspection category includes: calling a preset named entity model to identify the named entities contained in the text content to obtain inspection items, and extracting the inspection results corresponding to the inspection items from the text content; based on the inspection items, searching for the inspection category corresponding to the inspection item in a preset medical knowledge graph; and extracting the standard indicator range corresponding to each inspection item contained in the inspection category in the medical knowledge graph based on the inspection category.

[0007] Optionally, in a second implementation method of the first aspect of the present invention, searching for the inspection category corresponding to the inspection item in a preset medical knowledge graph based on the inspection item includes: calling a preset classification judgment model to determine the alternative inspection category to which the inspection item belongs based on the inspection item; searching for standard inspection items included in the alternative inspection category in the preset medical knowledge graph; judging whether the inspection item is the same as the standard inspection item; and if so, using the alternative inspection category as the inspection category corresponding to the inspection item.

[0008] Optionally, in a third implementation method of the first aspect of the present invention, receiving a data interpretation request and extracting a physical examination report image contained in the data interpretation request includes: receiving a data interpretation request, extracting a PDF format file of the physical examination report contained in the data interpretation request; parsing the PDF format file to obtain a binary file of the physical examination report; converting the binary file into a picture format file through a preset conversion tool to obtain a physical examination report image, wherein the preset conversion tool is constructed based on the PDF2image tool.

[0009] Optionally, in a fourth implementation method of the first aspect of the present invention, after converting the binary file into a picture format file through a preset conversion tool, it also includes: calling a preset direction adjustment tool to perform direction identification on the picture format file to obtain an actual direction angle of the picture, rotating the picture format file according to the actual direction angle to obtain a first adjusted picture file with a uniform direction angle; detecting whether the first picture file has stretching deformation, and if so, calling a preset perspective adjustment tool to perform perspective transformation on the rotated picture file to obtain a second adjusted picture file, and denoising the second adjusted picture file.

[0010] Optionally, in a fifth implementation manner of the first aspect of the present invention, calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report image, and cropping the physical examination report image according to the text position to obtain a cropped image set includes: inputting the physical examination report image into a preset content positioning model to identify the text content range of the content in the physical examination report image, and obtaining the boundary point coordinates of the text position of the physical examination result; cropping the physical examination report image according to the boundary point coordinates to obtain at least one text image block; scaling at least one of the text image blocks to obtain at least one cropped image block, and forming a cropped image set based on at least one of the cropped image blocks, wherein the short side length of the cropped image block is a preset length.

[0011] Optionally, in a sixth implementation method of the first aspect of the present invention, after outputting the data interpretation result according to the medical information, it also includes: calculating the difference index between the abnormal inspection result in the abnormal inspection category and the standard indicator range; querying the risk disease and the risk probability corresponding to the difference index in the medical knowledge graph according to the difference index; querying the corresponding medical advice according to the risk disease and the risk probability, and outputting the medical advice.

[0012] The second aspect of the present invention provides a data interpretation device based on image recognition, comprising: an extraction module, used to receive a data interpretation request, and extract a physical examination report image contained in the data interpretation request; an image cropping module, used to call a preset content positioning model to identify the text position of the physical examination result in the physical examination report image, and crop the physical examination report image according to the text position to obtain a cropped image set, wherein the cropped image set includes at least one cropped image block; a text recognition module, used to call a preset optical character recognition model to perform text recognition on the cropped image block, and obtain the text content in the cropped image block. content; a content determination module, used to determine the inspection category contained in the cropped image block and the inspection result contained in the inspection category according to the text content, and obtain the corresponding standard indicator range according to the inspection category; a judgment module, used to judge whether the inspection result is within the standard indicator range; a labeling module, used to label the inspection result as an abnormal inspection result and the inspection category as an abnormal inspection category if not; an output module, used to search in a preset medical knowledge graph according to the abnormal inspection result, obtain the medical information corresponding to the abnormal inspection category, and output the data interpretation result according to the medical information.

[0013] Optionally, in a first implementation method of the second aspect of the present invention, the content determination module includes: a content identification unit, used to call a preset named entity model to identify the named entities contained in the text content, obtain detection items, and extract the inspection results corresponding to the detection items in the text content; a category search unit, used to find the inspection category corresponding to the detection item in a preset medical knowledge graph according to the detection item; and a range extraction unit, used to extract the standard indicator range corresponding to each detection item contained in the inspection category in the medical knowledge graph according to the inspection category.

[0014] Optionally, in a second implementation method of the second aspect of the present invention, the category search unit includes: a category judgment subunit, used to call a preset classification judgment model to judge the alternative inspection category to which the inspection item belongs based on the inspection item; a standard item search subunit, used to find the standard inspection items included in the alternative inspection category in a preset medical knowledge graph; an item judgment subunit, used to judge whether the inspection item is the same as the standard inspection item; and an inspection category determination subunit, used to, if so, use the alternative inspection category as the inspection category corresponding to the inspection item.

[0015] Optionally, in a third implementation method of the second aspect of the present invention, the extraction module includes: a file extraction unit, used to receive a data interpretation request, and extract the PDF format file of the physical examination report contained in the data interpretation request; a file parsing unit, used to parse the PDF format file to obtain a binary file of the physical examination report; and an image conversion unit, used to convert the binary file into an image format file through a preset conversion tool to obtain a physical examination report image, wherein the preset conversion tool is constructed based on the PDF2image tool.

[0016] Optionally, in a fourth implementation method of the second aspect of the present invention, the extraction module also includes: a direction conversion unit, used to call a preset direction adjustment tool to perform direction identification on the picture format file, obtain an actual direction angle of the picture, rotate the picture format file according to the actual direction angle, and obtain a first adjusted picture file with a uniform direction angle; a deformation adjustment unit, used to detect whether the first picture file has stretching deformation, and if so, call a preset perspective adjustment tool to perform perspective transformation on the rotated picture file to obtain a second adjusted picture file, and perform denoising on the second adjusted picture file.

[0017] Optionally, in a fifth implementation manner of the second aspect of the present invention, the image cropping module includes: a content position determination unit, used to input the physical examination report image into a preset content positioning model to identify the text content range of the content in the physical examination report image, and obtain the boundary point coordinates of the text position of the physical examination result; a cropping unit, used to crop the physical examination report image according to the boundary point coordinates to obtain at least one text image block; a scaling unit, used to geometrically scale at least one of the text image blocks to obtain at least one cropped image block, and to form a cropped image set based on at least one of the cropped image blocks, wherein the short side length of the cropped image block is a preset length.

[0018] Optionally, in a sixth implementation manner of the second aspect of the present invention, the image recognition-based data interpretation device also includes a medical advice generation module, and the medical advice generation module includes: a phase difference index calculation unit, used to calculate the phase difference index between the abnormal inspection result in the abnormal inspection category and the standard indicator range; a query unit, used to query the risk disease and risk probability corresponding to the phase difference index in the medical knowledge graph according to the phase difference index; and a suggestion output unit, used to query the corresponding medical advice according to the risk disease and risk probability, and output the medical advice.

[0019] The third aspect of the present invention provides a data interpretation device based on image recognition, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the data interpretation device based on image recognition performs the steps of the above-mentioned data interpretation method based on image recognition.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the above-mentioned image recognition-based data interpretation method.

[0021] The technical solution provided by the present invention can be applied to auxiliary diagnosis and treatment of diseases and remote consultation. The method receives a data interpretation request and extracts a physical examination report image contained therein; calls a preset content positioning model to identify the text position of the physical examination result in the physical examination report image, and crops the physical examination report image according to the text position to obtain a cropped image set, wherein the cropped image set includes at least one cropped image block; calls a preset optical character recognition model to perform text recognition on the cropped image block to obtain text content; determines the inspection category contained in the cropped image block and the inspection result contained in the inspection category according to the text content, and obtains the corresponding standard indicator range according to the inspection category, wherein the inspection result is at least one; determines whether the inspection result is within the standard indicator range; if not, marks the inspection result and the inspection category as abnormal inspection result and abnormal inspection category; searches in a preset medical knowledge graph according to the abnormal inspection result to obtain corresponding medical information, and outputs the data interpretation result according to the medical information. The solution in the embodiment of the present invention can automatically interpret the content in a variety of physical examination reports of different formats to obtain medical information in the physical examination report, thereby improving the efficiency of interpreting the physical examination report. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a first embodiment of a data interpretation method based on image recognition in an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of a second embodiment of a data interpretation method based on image recognition in an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of a third embodiment of a data interpretation method based on image recognition in an embodiment of the present invention;

[0025] Figure 4 is a schematic diagram of a fourth embodiment of a data interpretation method based on image recognition in an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of an embodiment of a data interpretation device based on image recognition in an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of another embodiment of a data interpretation device based on image recognition in an embodiment of the present invention;

[0028] Figure 7 Schematic diagram of an embodiment of a data interpretation device based on image recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The embodiment of the present invention provides a data interpretation method, device, equipment and storage medium based on image recognition, which receives and extracts a physical examination report image contained in a data interpretation request; identifies the text position of the physical examination result in the image, and crops it to obtain a cropped image set; performs text recognition on the cropped image block in the cropped image set to obtain the text content; and determines the inspection category and inspection result contained in the cropped image block, and obtains the corresponding standard indicator range; determines whether the inspection result is within the standard indicator range; if not, marks the inspection result and inspection category as abnormal inspection result and abnormal inspection category; retrieves medical information in the medical knowledge graph, and outputs the data interpretation result. The scheme in the embodiment of the present invention can automatically interpret the content in physical examination reports of various different formats to obtain the medical information in the physical examination report, thereby improving the efficiency of interpreting the physical examination report; the method can be applied to auxiliary disease diagnosis and treatment and remote consultation.

[0030] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 An embodiment of the data interpretation method based on image recognition of the embodiment of the present invention includes:

[0032] 101. Receive a data interpretation request, and extract a physical examination report image included in the data interpretation request;

[0033] It is understandable that the execution subject of the present invention may be a physical examination report interpretation device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0034] The physical examination report interpretation method in this embodiment can identify the physical examination report that the user wants to interpret and output the interpreted information, so that the user can quickly understand the physical examination conclusion and physical examination information in the physical examination report.

[0035] The server in this embodiment first receives a data interpretation request from the user, wherein the data interpretation request may be a request for interpretation of a physical examination report, and the report interpretation request includes a physical examination report file to be interpreted. After receiving the report interpretation request, the physical examination report file included in the report interpretation request is extracted, and the obtained physical examination report file is converted into a physical examination report image; specifically, the physical examination report image in this embodiment may be a report image obtained by scanning the physical examination report file or a physical examination report photo obtained by taking a photo; it may also be a PDF format file, and a preset format conversion tool may be used to convert the physical examination report file in PDF format into a picture format to obtain a physical examination report image and perform subsequent recognition. In addition, the physical examination report file may be multi-page, and when the physical examination report file is multi-page and the format conversion is performed, the converted images are arranged in the order of the original page numbers.

[0036] 102. Calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report image;

[0037] 103. Crop the physical examination report image according to the text position to obtain a cropped image set;

[0038] The obtained physical examination report image is input into a preset content positioning model to identify the text position contained in the physical examination report image, wherein the content positioning model is established by using a DBNet (Differentiable binarizationnet) tool.

[0039] In this embodiment, the layouts of physical examination report pictures for different physical examination items and physical examination units may be very different. For example, some pictures may contain test sheets and tables of test results, and some may contain imaging reports, and the positions of text in different reports are different; when the physical examination institutions are different, the backgrounds of the physical examination report pictures may also be different. These differences will reduce the accuracy of text recognition. Establishing different recognition models for each different layout will increase the training time of the recognition model. Therefore, in this step, the content positioning model is used to identify the text position of the physical examination results contained in the physical examination report picture.

[0040] After identifying the location of the text content in each physical examination report image, the text content will be cropped according to the text location, and the content that does not contain text content, such as images, template patterns, etc., which are irrelevant to the physical examination results, will be removed to obtain cropped image blocks. All the obtained cropped image blocks will be combined into a cropped image set, wherein the cropped image set contains at least one cropped image block.

[0041] 104. Calling a preset optical character recognition model to perform text recognition on the cropped image block to obtain text content in the cropped image block;

[0042] Subsequently, the cropped image blocks in the cropped image set obtained above are transmitted to a preset optical character recognition (OCR) model to recognize the text content in the physical examination report.

[0043] Specifically, after receiving the cropped image block, the optical character recognition model of this step first performs tilt correction on the physical examination report image. If it is detected that the physical examination report image is tilted, the tilt angle is detected, and the image is perspectively transformed to correct the tilt according to the tilt angle.

[0044] After the above processing is completed, the text in the cropped image block is recognized, the edge of each character in the text part is identified, and the cropped image block is cut according to the character edge to obtain the character image. Then each character image is recognized to obtain the recognition result of each character, and each character is converted into a language that can be recognized by the computer; and the conversion result is regularized and corrected to finally obtain the text content.

[0045] 105. Determine, according to the text content, an inspection category included in the cropped image block and an inspection result included in the inspection category;

[0046] 106. Obtain the corresponding standard indicator range according to the inspection category;

[0047] In this embodiment, an unadjusted named entity recognition (NER) tool is obtained in advance; at the same time, a medical dictionary is obtained, and parameters of the unadjusted named entity recognition tool are adjusted according to the content in the medical dictionary to obtain a named entity recognition model, the named entity recognition model is called to recognize named entities on the obtained text content, and the specific meaning of the recognized named entities is obtained in combination with the content in the medical dictionary, so as to identify the inspection category corresponding to each test item contained in the text content; and according to the character distribution position in the aforementioned physical examination report picture, the inspection result corresponding to the inspection category is determined; after obtaining the inspection category, the standard indicator range corresponding to the inspection category is queried in a preset medical knowledge base.

[0048] 107. Determine whether the inspection results are within the standard index range;

[0049] 108. If not, the inspection result is marked as an abnormal inspection result, and the inspection category is marked as an abnormal inspection category;

[0050] 109. According to the abnormal examination results, a search is performed in the preset medical knowledge graph to obtain the medical information corresponding to the abnormal examination category, and the data interpretation results are output based on the medical information.

[0051] After obtaining the standard indicator range corresponding to the inspection category, all inspection results in the inspection category are judged to be abnormal.

[0052] Specifically, it is determined whether each inspection result exceeds the standard index range corresponding to the inspection item. If all inspection results do not exceed the standard index range corresponding to the inspection item, the inspection category is considered to be normal, and a preset notification text of normal inspection category is output, wherein the preset notification text of normal inspection category may also include pre-input healthy living guidelines, such as: "balanced diet", "moderate exercise" and other contents.

[0053] If there is at least one detection item in the inspection result that exceeds the standard indicator range, the inspection result is marked as an abnormal inspection result, and the inspection category is marked as an abnormal inspection category.

[0054] According to the abnormal inspection category, medical information corresponding to the abnormal inspection result is retrieved based on each abnormal inspection result in the preset medical knowledge graph, wherein the medical information may be a possible cause of the abnormality.

[0055] The solution in the embodiment of the present invention can automatically interpret the contents of physical examination reports in various formats to obtain the medical information in the physical examination reports, thereby improving the efficiency of interpreting the physical examination reports.

[0056] See also Figure 2 The second embodiment of the data interpretation method based on image recognition of the embodiment of the present invention includes:

[0057] 201. Receive a data interpretation request, and extract a physical examination report image included in the data interpretation request;

[0058] 202. Calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report image;

[0059] 203. Crop the physical examination report image according to the text position to obtain a cropped image set;

[0060] 204. Calling a preset optical character recognition model to perform text recognition on the cropped image block to obtain text content in the cropped image block;

[0061] The contents of step 201 to step 204 in this embodiment are substantially the same as the contents of step 101 to step 104 in the aforementioned embodiment, and thus will not be described in detail herein.

[0062] 205. Calling a preset named entity model to identify the named entities contained in the text content, obtaining detection items, and extracting inspection results corresponding to the detection items from the text content;

[0063] 206. Calling a preset classification judgment model to judge the candidate inspection category to which the inspection item belongs according to the inspection item;

[0064] In this embodiment, the parameters of the unadjusted NER tool are adjusted in advance according to the content in the medical dictionary to obtain a named entity recognition model, and the named entity recognition model is called to identify and annotate the named entities in the obtained text content to obtain the detection items in the text content.

[0065] After obtaining the test items in the text content, the actual test results corresponding to the test categories are determined based on the character distribution positions in the aforementioned physical examination report image.

[0066] A classification judgment model established in advance according to a CRF (Conditional Random Fields) tool is called, and each identified detection item is input into the classification judgment model to obtain a judgment result of the inspection category to which the detection item may belong and a confidence probability corresponding to each inspection category judgment result, wherein at least one inspection category judgment result is obtained in this step, and each inspection category judgment result is sorted according to the obtained confidence probability to obtain a judgment result sequence, and at least one inspection category judgment result is extracted from the extracted judgment result sequence; specifically, the top several rankings or top several percentages of inspection category judgment results with the highest category probability can be screened out in the judgment result sequence as the candidate inspection categories to which the detection item belongs.

[0067] 207. Find the standard examination items included in the alternative examination category in the preset medical knowledge graph;

[0068] 208. Determine whether the test items are the same as the standard inspection items;

[0069] 209. If yes, the alternative inspection category shall be used as the inspection category corresponding to the test item;

[0070] 210. Extract the standard indicator range corresponding to each test item contained in the examination category from the medical knowledge graph according to the examination category;

[0071] Obtain a preset medical knowledge graph, wherein the preset knowledge graph is pre-organized based on data in a medical dictionary and a medical knowledge base; query the standard test items corresponding to each alternative inspection category based on the medical knowledge graph, and determine whether the aforementioned inspection items belong to the corresponding standard inspection items. If so, use the alternative inspection category with the highest overlap between the inspection item and the standard inspection item as the inspection category corresponding to the inspection item.

[0072] 211. Determine whether the inspection results are within the standard index range;

[0073] 212. If not, the inspection result is marked as an abnormal inspection result, and the inspection category is marked as an abnormal inspection category;

[0074] 213. According to the abnormal examination results, a search is performed in the preset medical knowledge graph to obtain the medical information corresponding to the abnormal examination category, and the data interpretation results are output based on the medical information.

[0075] The contents of step 211 to step 213 in this embodiment are substantially the same as those of step 107 to step 109 in the aforementioned embodiment and thus will not be described in detail herein.

[0076] The scheme in the embodiment of the present invention can determine the category to which the examination items belong for the contents in physical examination reports of various different formats, and determine the standard indicator range according to the examination category, and automatically interpret the contents of the physical examination report according to the actual examination results and the marked indicator range to obtain the medical information in the physical examination report, thereby improving the accuracy and efficiency of the interpretation of the physical examination report.

[0077] See also Figure 3 The third embodiment of the data interpretation method based on image recognition of the embodiment of the present invention includes:

[0078] 301. Receive a data interpretation request, and extract a PDF file of a physical examination report included in the data interpretation request;

[0079] 302. Parse the PDF format file to obtain the binary file of the physical examination report;

[0080] 303. Convert the binary file into a picture format file using a preset conversion tool;

[0081] The server in this embodiment first receives a report interpretation request from the user, wherein the report interpretation request includes a physical examination report file to be interpreted. After receiving the report interpretation request, the physical examination report file included in the report interpretation request is extracted, and the obtained physical examination report file is converted into a picture format file. In this embodiment, the extracted physical examination report file to be interpreted can be in PDF format. When the extracted physical examination report file is in PDF format, the picture conversion tool established based on Python language and PDF2image toolkit in this embodiment is called to convert the format of the physical examination report to obtain a picture format file, wherein the obtained picture format file can be multi-page.

[0082] When the obtained picture format file is multi-page, the PDF format file is converted into a binary file according to the page number sequence, and the obtained binary file is input into a preset picture conversion tool to obtain the picture format file.

[0083] 304. Calling a preset direction adjustment tool to identify the direction of the image format file to obtain the actual direction angle of the image;

[0084] 305. Rotate the image format file according to the actual direction angle to obtain a first adjusted image file with a unified direction angle;

[0085] After obtaining the converted image format file, call the preset direction adjustment tool to first identify the direction of the image format file, wherein the direction adjustment tool in this step has a direction angle detection function and a rotation function, wherein the direction angle detection function can be constructed based on a deep neural network for feature extraction and a DFL classification network (DFL-CNN, Learning a Discriminative Filter Bank within a CNN) for classifying features, wherein the deep neural network for feature extraction can be, for example, a VGG network (a deep neural network proposed by the Visual Geometry Group) or a DenseNet (Densely connected convolutional networks), etc.

[0086] In this step, the features of the image format file are first extracted, the direction of the file is classified according to the extracted features to obtain the actual direction angle, and then the image format file is rotated according to the actual direction angle to obtain a first adjusted image file with a unified direction angle.

[0087] 306. Detect whether the first image file is stretched or deformed;

[0088] 307. If yes, call a preset perspective adjustment tool to perform perspective transformation on the rotated image file to obtain a second adjusted image file, and perform denoising on the second adjusted image file to obtain a physical examination report image;

[0089] Then, the first image file is detected to see if it is stretched or deformed. If so, the physical examination report image is subjected to tilt correction and perspective transformation operations to correct its tilt and stretch deformation to obtain a second adjusted image file. Then, the second adjusted image file is subjected to denoising to improve the clarity of the image, so as to improve the accuracy of subsequent text recognition and obtain a physical examination report image.

[0090] 308. Calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report image;

[0091] 309. Crop the physical examination report image according to the text position to obtain a cropped image set;

[0092] The obtained physical examination report image is input into a preset content positioning model to identify the text position contained in the physical examination report image, wherein the content positioning model is established by using a DBNet (Differentiable binarizationnet) tool.

[0093] In this embodiment, the layouts of physical examination report images for different physical examination items and physical examination units may be very different. These differences will reduce the accuracy of text recognition. Establishing different recognition models for each different layout will increase the training time of the recognition model. Therefore, in this step, the content positioning model is used to identify the text position of the physical examination results contained in the physical examination report image.

[0094] After identifying the location of the text content in each physical examination report image, the text content will be cropped according to the text location, and the content that does not contain text content, such as images, template patterns, etc., which are irrelevant to the physical examination results, will be removed to obtain cropped image blocks. All the obtained cropped image blocks will be combined into a cropped image set, wherein the cropped image set contains at least one cropped image block.

[0095] 310. Calling a preset optical character recognition model to perform text recognition on the cropped image block to obtain text content in the cropped image block;

[0096] Subsequently, the cropped image blocks in the cropped image set obtained above are transmitted to a preset optical character recognition (OCR) model to recognize the text content in the physical examination report.

[0097] Specifically, after receiving the cropped image block, the optical character recognition model of this step first performs tilt correction on the physical examination report image. If it is detected that the physical examination report image is tilted, the tilt angle is detected, and the image is perspectively transformed to correct the tilt according to the tilt angle.

[0098] After the above processing is completed, the text in the cropped image block is recognized, the edge of each character in the text part is identified, and the cropped image block is cut according to the character edge to obtain the character image. Then each character image is recognized to obtain the recognition result of each character, and each character is converted into a language that can be recognized by the computer; and the conversion result is regularized and corrected to finally obtain the text content.

[0099] Among them, the optical character recognition model can be constructed by CRNN-CTC (Convolutional Recurrent Neural Network-Connectionist Temporal Classification). The recognition model performs character segmentation on the image to be recognized in the image set to be recognized, recognizes each character in the physical examination report picture, and converts each character into a language that can be recognized by the computer to obtain text content.

[0100] In this embodiment, DBNet is used in combination with CRNN-CTC to recognize text content, which can recognize text paragraphs in different layout positions, and can improve the versatility of the physical examination report interpretation method in this embodiment. For example, it can accurately recognize table-type test order data, and can also accurately recognize text paragraphs in imaging reports.

[0101] 311. Determine the inspection category and the inspection result contained in the cropped image block according to the text content, and obtain the corresponding standard indicator range according to the inspection category;

[0102] A NER model is pre-built based on BERT (Bidirectional Encoder Representations from Transformers) and CRF (Conditional Random Fields) tools, wherein the NER model is trained based on information in a medical dictionary and a medical knowledge base, and the NER model can classify and judge the current test item based on the test item, and calculate the probability of the test category to which the test item may belong. Based on the obtained probability of the test category and the information in the preset medical atlas, the test category corresponding to the test item and the standard indicator range corresponding to the test category are found.

[0103] 312. Determine whether the inspection results are within the standard index range;

[0104] 313. If not, the inspection result is marked as an abnormal inspection result, and the inspection category is marked as an abnormal inspection category;

[0105] 314. According to the abnormal examination results, a search is performed in the preset medical knowledge graph to obtain the medical information corresponding to the abnormal examination category, and the data interpretation results are output based on the medical information.

[0106] The contents of step 312 to step 314 in this embodiment are substantially the same as the contents of step 107 to step 109 in the above embodiment, so they will not be described again here.

[0107] The solution in the embodiment of the present invention can automatically interpret the contents of physical examination reports in various formats to obtain the medical information in the physical examination reports, thereby improving the efficiency of interpreting the physical examination reports. In addition, the solution in this embodiment also improves the accuracy of the information obtained by interpretation.

[0108] See also Figure 4 , a fourth embodiment of the data interpretation method based on image recognition in an embodiment of the present invention includes:

[0109] 401. Receive a data interpretation request, and extract a PDF file of the physical examination report contained in the data interpretation request;

[0110] 402. Parse the PDF format file to obtain the binary file of the physical examination report;

[0111] 403. Convert the binary file into a picture format file using a preset conversion tool;

[0112] 404. Calling a preset direction adjustment tool to identify the direction of the picture format file, obtaining the actual direction angle of the picture, and rotating the picture format file according to the actual direction angle to obtain a first adjusted picture file with a uniform direction angle;

[0113] 405. Detect whether the first image file is stretched or deformed;

[0114] 406. If yes, call a preset perspective adjustment tool to perform perspective transformation on the rotated image file to obtain a second adjusted image file, and perform denoising on the second adjusted image file to obtain a physical examination report image;

[0115] The specific contents of step 401 to step 406 in this embodiment are substantially the same as the specific contents of step 301 to step 308 in the aforementioned embodiment, and thus will not be described in detail here.

[0116] 407. Calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report image, and cropping the physical examination report image according to the text position to obtain a cropped image set;

[0117] The obtained physical examination report image is input into a preset content positioning model to identify the text position contained in the physical examination report image.

[0118] In this embodiment, the layouts of physical examination report pictures for different physical examination items and physical examination units may be very different. For example, some pictures may contain test sheets and tables of test results, and some may contain imaging reports, and the positions of text in different reports are different; when the physical examination institutions are different, the backgrounds of the physical examination report pictures may also be different. These differences will reduce the accuracy of text recognition. Establishing different recognition models for each different layout will increase the training time of the recognition model. Therefore, in this step, the content positioning model is used to identify the text position of the physical examination results contained in the physical examination report picture.

[0119] After identifying the location of the text content in each physical examination report image, the text content will be cropped according to the text location, and the content that does not contain text content, such as images, template patterns, etc., which are irrelevant to the physical examination results, will be removed to obtain cropped image blocks. All the obtained cropped image blocks will be combined into a cropped image set, wherein the cropped image set contains at least one cropped image block.

[0120] Specifically, the content positioning model generated by the DBNet tool is called to detect the position of the fields on the physical examination report image, and the boundary point coordinates of the upper left, lower left, upper right and lower right points of the field are obtained after the detection. At least one text image block is cropped based on the above coordinates, and then at least one text image block is scaled proportionally, and the short side length of the cropped image block is set to a preset length. In a specific example, the short side of the image block is scaled to 720 pixels, and the long side is scaled proportionally according to the original image size. The image obtained based on the scaling constitutes the image set to be identified.

[0121] 408. Calling a preset optical character recognition model to perform text recognition on the cropped image block to obtain text content in the cropped image block;

[0122] 409. Calling a preset named entity model to identify the named entities contained in the text content, obtaining detection items, and extracting inspection results corresponding to the detection items from the text content;

[0123] 410. Calling a preset classification judgment model to judge the candidate inspection category to which the inspection item belongs according to the inspection item;

[0124] 411. Find the standard examination items included in the alternative examination category in the preset medical knowledge graph;

[0125] 412. Determine whether the test items are the same as the standard inspection items;

[0126] 413. If yes, the alternative inspection category is used as the inspection category corresponding to the test item;

[0127] 414. Extracting the standard indicator range corresponding to each test item contained in the examination category from the medical knowledge graph according to the examination category;

[0128] The contents of step 408 to step 414 in this embodiment are substantially the same as the contents of step 206 to step 210 in the above-mentioned embodiment, and thus will not be described in detail here.

[0129] 415. Determine whether the inspection results are within the standard index range;

[0130] 416. If not, the inspection result is marked as an abnormal inspection result, and the inspection category is marked as an abnormal inspection category;

[0131] The contents of step 415 and step 416 in this embodiment are substantially the same as the contents of step 107 and step 108 in the aforementioned embodiment, and thus will not be described in detail here.

[0132] 417. According to the abnormal examination results, a search is performed in the preset medical knowledge graph to obtain the medical information corresponding to the abnormal examination category, and the data interpretation results are output based on the medical information.

[0133] After obtaining the standard indicator range corresponding to the inspection category, all inspection results in the inspection category are judged to be abnormal.

[0134] Specifically, it is determined whether each inspection result exceeds the standard index range corresponding to the inspection item. If all inspection results do not exceed the standard index range corresponding to the inspection item, the inspection category is considered to be normal, and a preset notification text of normal inspection category is output, wherein the preset notification text of normal inspection category may also include pre-input healthy living guidelines, such as: "balanced diet", "moderate exercise" and other contents.

[0135] If there is at least one detection item in the inspection result that exceeds the standard indicator range, the inspection result is marked as an abnormal inspection result, and the inspection category is marked as an abnormal inspection category.

[0136] According to the abnormal inspection category, medical information corresponding to the abnormal inspection result is retrieved based on each abnormal inspection result in the preset medical knowledge graph, wherein the medical information may be a possible cause of the abnormality.

[0137] In addition, the difference index between the actual test result of the abnormal item and the standard index is calculated; the risk disease and risk probability corresponding to the difference index are queried in the medical knowledge graph according to the difference index; the medical advice corresponding to the risk disease is queried in the medical knowledge graph based on the difference index through a clustering algorithm, wherein the DBScan algorithm in the clustering algorithm can be specifically used to detect abnormal indicators (such as increased red blood cell count in a blood routine test) or abnormal conclusions (breast color ultrasound report shows the presence of left breast nodules) in each type of check list; the risk probability is calculated according to the specific content of the medical knowledge graph, and a disease risk warning and medical advice are given. In addition, the present application can also count all the medical advice generated above, parse the specific content of the medical advice, modify the conflicting medical advice, and give a holistic medical advice.

[0138] The solution in the embodiment of the present invention can automatically interpret the contents of physical examination reports in various formats to obtain the medical information in the physical examination reports, thereby improving the efficiency of interpreting the physical examination reports. In addition, the solution in this embodiment also improves the accuracy of the information obtained by interpretation.

[0139] The above describes the data interpretation method based on image recognition in an embodiment of the present invention. The following describes the data interpretation device based on image recognition in an embodiment of the present invention. Figure 5 An embodiment of the data interpretation device based on image recognition of the embodiment of the present invention includes:

[0140] Extraction module 501, used to receive a data interpretation request and extract the physical examination report image contained in the data interpretation request;

[0141] The image cropping module 502 is used to call a preset content positioning model to identify the text position of the physical examination result in the physical examination report image, and crop the physical examination report image according to the text position to obtain a cropped image set, wherein the cropped image set includes at least one cropped image block

[0142] A text recognition module 503 is used to call a preset optical character recognition model to perform text recognition on the cropped image block to obtain text content in the cropped image block;

[0143] A content determination module 504 is used to determine the inspection category contained in the cropped image block and the inspection result contained in the inspection category according to the text content, and obtain a corresponding standard indicator range according to the inspection category;

[0144] A judgment module 505 is used to judge whether the inspection result is within the standard indicator range;

[0145] A marking module 506 is used for marking the inspection result as an abnormal inspection result and marking the inspection category as an abnormal inspection category if no;

[0146] The output module 507 is used to search in a preset medical knowledge graph according to the abnormal inspection result, obtain the medical information corresponding to the abnormal inspection category, and output the data interpretation result according to the medical information.

[0147] The solution in the embodiment of the present invention can automatically interpret the contents of physical examination reports in various formats to obtain the medical information in the physical examination reports, thereby improving the efficiency of interpreting the physical examination reports.

[0148] See also Figure 6 Another embodiment of the data interpretation device based on image recognition in the embodiment of the present invention includes:

[0149] The extraction module 501 is used to receive a data interpretation request and extract the physical examination report image contained in the data interpretation request; the image cropping module 502 is used to call a preset content positioning model to identify the text position of the physical examination result in the physical examination report image, and crop the physical examination report image according to the text position to obtain a cropped image set, wherein the cropped image set includes at least one cropped image block; the text recognition module 503 is used to call a preset optical character recognition model to perform text recognition on the cropped image block to obtain the text content in the cropped image block; the content determination module 504 is used to identify the text content in the cropped image block according to the content positioning model. The text content determines the inspection category contained in the cropped image block and the inspection result contained in the inspection category, and obtains the corresponding standard indicator range according to the inspection category; a judgment module 505 is used to judge whether the inspection result is within the standard indicator range; a labeling module 506 is used to label the inspection result as an abnormal inspection result and the inspection category as an abnormal inspection category if not; an output module 507 is used to search in a preset medical knowledge graph according to the abnormal inspection result, obtain medical information corresponding to the abnormal inspection category, and output a data interpretation result according to the medical information.

[0150] In another embodiment of the present application, the content determination module 504 includes: a content identification unit 5041, which is used to call a preset named entity model to identify the named entities contained in the text content, obtain detection items, and extract the inspection results corresponding to the detection items in the text content; a category search unit 5042, which is used to find the inspection category corresponding to the detection item in a preset medical knowledge graph according to the detection item; and a range extraction unit 5043, which is used to extract the standard indicator range corresponding to each detection item contained in the inspection category in the medical knowledge graph according to the inspection category.

[0151] In another embodiment of the present application, the category search unit 5042 includes: a category judgment subunit, used to call a preset classification judgment model to determine the alternative inspection category to which the inspection item belongs based on the inspection item; a standard item search subunit, used to find the standard inspection items included in the alternative inspection category in the preset medical knowledge graph; an item judgment subunit, used to determine whether the inspection item is the same as the standard inspection item; and an inspection category determination subunit, used to, if so, use the alternative inspection category as the inspection category corresponding to the inspection item.

[0152] In another embodiment of the present application, the extraction module 501 includes: a file extraction unit 5011, used to receive a data interpretation request, and extract the PDF format file of the physical examination report contained in the data interpretation request; a file parsing unit 5012, used to parse the PDF format file to obtain a binary file of the physical examination report; an image conversion unit 5013, used to convert the binary file into an image format file through a preset conversion tool to obtain a physical examination report image, wherein the preset conversion tool is constructed based on the PDF2image tool.

[0153] In another embodiment of the present application, the extraction module 501 also includes: a direction conversion unit, which is used to call a preset direction adjustment tool to perform direction identification on the picture format file, obtain the actual direction angle of the picture, rotate the picture format file according to the actual direction angle, and obtain a first adjusted picture file with a uniform direction angle; a deformation adjustment unit, which is used to detect whether the first picture file has stretching deformation, and if so, call a preset perspective adjustment tool to perform perspective transformation on the rotated picture file to obtain a second adjusted picture file, and perform denoising on the second adjusted picture file.

[0154] In another embodiment of the present application, the image cropping module 502 includes: a content position determination unit 5021, which is used to input the physical examination report image into a preset content positioning model to identify the text content range of the content in the physical examination report image, and obtain the boundary point coordinates of the text position of the physical examination result; a cropping unit 5022, which is used to crop the physical examination report image according to the boundary point coordinates to obtain at least one text image block; a scaling unit 5023, which is used to scale at least one of the text image blocks proportionally to obtain at least one cropped image block, and form a cropped image set based on at least one of the cropped image blocks, wherein the short side length of the cropped image block is a preset length.

[0155] In another embodiment of the present application, the image recognition-based data interpretation device also includes a medical advice generation module, which includes: a phase difference index calculation unit, used to calculate the phase difference index between the abnormal inspection result in the abnormal inspection category and the standard indicator range; a query unit, used to query the risk disease and risk probability corresponding to the phase difference index in the medical knowledge graph according to the phase difference index; a suggestion output unit, used to query the corresponding medical advice according to the risk disease and risk probability, and output the medical advice.

[0156] The solution in the embodiment of the present invention can automatically interpret the contents of physical examination reports of various formats to obtain the medical information in the physical examination reports, thereby improving the efficiency of interpreting the physical examination reports. In addition, the solution in this embodiment also improves the accuracy of the information obtained by interpretation.

[0157] above Figure 5 and Figure 6 The data interpretation device based on image recognition in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The data interpretation device based on image recognition in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0158] Figure 7 7 is a schematic diagram of the structure of a data interpretation device based on image recognition provided by an embodiment of the present invention. The data interpretation device 700 based on image recognition may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 may be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data interpretation device 700 based on image recognition. Furthermore, the processor 710 may be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the data interpretation device 700 based on image recognition.

[0159] The image recognition-based data interpretation device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 760, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that Figure 7 The structure of the data interpretation device based on image recognition shown does not constitute a limitation on the data interpretation device based on image recognition, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0160] The present invention also provides a computer device, which can be any device capable of executing the data interpretation method based on image recognition described in the above embodiments, and the computer device includes a memory and a processor, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the data interpretation method based on image recognition in the above embodiments.

[0161] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0162] In a possible implementation, the data in the present invention is medical data, such as personal health records, prescriptions, examination reports and the like.

[0163] In a possible implementation, the text content is a medical text, and the medical text may be an electronic healthcare record, an electronic personal health record, including a series of electronic records with preservation value such as medical records, electrocardiograms, and medical images.

[0164] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the image recognition-based data interpretation method.

[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0166] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0167] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data interpretation method based on image recognition, characterized in that: The data interpretation method includes: Receiving a data interpretation request, and extracting a physical examination report image contained in the data interpretation request; Calling a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture, and cropping the physical examination report picture according to the text position to obtain a cropped image set, wherein the cropped image set includes at least one cropped image block; Calling a preset optical character recognition model to perform text recognition on the cropped image block to obtain text content in the cropped image block; Calling a preset named entity model to identify the named entities contained in the text content to obtain detection items, and extracting inspection results corresponding to the detection items in the text content; Calling a preset classification judgment model to judge the candidate inspection category to which the inspection item belongs according to the inspection item; Searching for standard examination items included in the candidate examination category in a preset medical knowledge graph; Determining whether the test items are the same as the standard inspection items; If yes, the candidate inspection category is used as the inspection category corresponding to the inspection item; Extracting, according to the examination category, a standard indicator range corresponding to each test item included in the examination category in the medical knowledge graph, wherein the examination result is at least one; Determine whether the inspection result is within the standard indicator range; If not, the inspection result is marked as an abnormal inspection result, and the inspection category is marked as an abnormal inspection category; A search is performed in a preset medical knowledge graph according to the abnormal examination result to obtain medical information corresponding to the abnormal examination category, and a data interpretation result is output according to the medical information.

2. The data interpretation method based on image recognition according to claim 1, characterized in that: The receiving a data interpretation request and extracting a physical examination report image contained in the data interpretation request includes: Receiving a data interpretation request, and extracting a PDF file of a physical examination report contained in the data interpretation request; Parsing the PDF format file to obtain a binary file of the physical examination report; The binary file is converted into a picture format file by a preset conversion tool to obtain a physical examination report picture, wherein the preset conversion tool is constructed based on the PDF2image tool.

3. The data interpretation method based on image recognition according to claim 2, characterized in that: After converting the binary file into a picture format file by a preset conversion tool, the method further includes: Calling a preset direction adjustment tool to perform direction identification on the picture format file to obtain an actual direction angle of the picture, and rotating the picture format file according to the actual direction angle to obtain a first adjusted picture file with a unified direction angle; Detect whether the first adjustment picture file has any stretching deformation. If so, call a preset perspective adjustment tool to perform perspective transformation on the first adjustment picture file to obtain a second adjustment picture file, and perform denoising on the second adjustment picture file.

4. The data interpretation method based on image recognition according to claim 3, characterized in that: The calling of a preset content positioning model to identify the text position of the physical examination result in the physical examination report picture, and cropping the physical examination report picture according to the text position to obtain a cropped image set includes: Input the medical examination report image into a preset content positioning model to identify the text content range of the content in the medical examination report image, and obtain the boundary point coordinates of the text position of the medical examination result; Cropping the physical examination report image according to the boundary point coordinates to obtain at least one text image block; At least one of the text image blocks is geometrically scaled to obtain at least one cropped image block, and a cropped image set is formed based on the at least one cropped image block, wherein the short side length of the cropped image block is a preset length.

5. The data interpretation method based on image recognition according to any one of claims 1 to 4, characterized in that: After outputting the data interpretation result according to the medical information, the method further includes: Calculating a difference index between the abnormal inspection result in the abnormal inspection category and the standard index range; According to the phase difference index, the risk disease and risk probability corresponding to the phase difference index are searched in the medical knowledge graph; According to the risk disease and the risk probability, corresponding medical advice is queried and the medical advice is output.

6. A data interpretation device based on image recognition, characterized in that: The data interpretation device based on image recognition comprises: An extraction module, used to receive a data interpretation request and extract a physical examination report image contained in the data interpretation request; An image cropping module is used to call a preset content positioning model to identify the text position of the physical examination result in the physical examination report image, and crop the physical examination report image according to the text position to obtain a cropped image set, wherein the cropped image set includes at least one cropped image block A text recognition module, used to call a preset optical character recognition model to perform text recognition on the cropped image block to obtain text content in the cropped image block; A content determination module, used to call a preset named entity model to identify the named entities contained in the text content, obtain detection items, and extract the inspection results corresponding to the detection items in the text content; call a preset classification judgment model to determine the candidate inspection category to which the detection item belongs according to the detection item; find out the standard inspection items contained in the candidate inspection category in the preset medical knowledge graph; determine whether the detection item is the same as the standard inspection item; if so, use the candidate inspection category as the inspection category corresponding to the detection item; extract the standard indicator range corresponding to each detection item contained in the inspection category in the medical knowledge graph according to the inspection category, wherein the inspection result is at least one; A judgment module, used to judge whether the inspection result is within the standard indicator range; a marking module, for marking the inspection result as an abnormal inspection result and marking the inspection category as an abnormal inspection category if no; The output module is used to search in a preset medical knowledge graph according to the abnormal examination result, obtain the medical information corresponding to the abnormal examination category, and output the data interpretation result according to the medical information.

7. A data interpretation device based on image recognition, characterized in that: The data interpretation device based on image recognition includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the image recognition-based data interpretation device to perform the steps of the image recognition-based data interpretation method according to any one of claims 1 to 5.

8. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the data interpretation method based on image recognition as described in any one of claims 1 to 5 are implemented.

Images