Ocr-based medical material structuring processing method, device, equipment and medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING YIFANFENGSHUN PHARM TECH CO LTD
- Filing Date
- 2023-07-03
- Publication Date
- 2026-06-19
Smart Images
Figure CN116994254B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the medical field, and in particular to a method, apparatus, equipment, and medium for structuring medical materials based on OCR. Background Technology
[0002] In the medical field, there is a large amount of medical material taken by patients themselves using mobile devices (such as mobile phones). In order to better analyze medical information, it is necessary to perform primary structured processing on the medical materials taken by patients, that is, to extract the first-level structured information such as "diagnosis", "discharge instructions" and "treatment process" from the medical image materials, so as to provide a data foundation for the next step of extracting detailed information.
[0003] However, due to the significant differences in medical materials among hospitals across the country, existing technologies have poor support for medical materials from hospitals nationwide, low coverage, low efficiency in extracting primary structured medical information, and cannot make reasonable use of the captured medical materials. Summary of the Invention
[0004] In view of this, the present invention provides a method, apparatus, equipment and medium for structuring medical materials based on OCR, in order to solve the technical problems of poor support for medical materials in hospitals across the country, low coverage, low efficiency in extracting primary structured medical information, and inability to make reasonable use of the captured medical materials in the existing technology.
[0005] Specifically, the following technical solutions are included:
[0006] Firstly, a method for structuring medical materials based on OCR is provided, including:
[0007] Obtain images of the medical materials to be identified;
[0008] The text recognition of the medical material image to be identified is performed using OCR technology to obtain multiple identified texts, and the multiple identified texts are preprocessed to obtain multiple target texts;
[0009] Sort the multiple target texts, extract the text content from the multiple target texts and concatenate them to obtain a target text set;
[0010] Based on a pre-trained medical materials model, the types of medical materials corresponding to the target text set are determined;
[0011] Based on a pre-built structured dictionary, the names of multiple structured fields corresponding to the medical material types of the target text set are determined, as well as multiple keywords corresponding to each structured field name;
[0012] The target text set is structured based on the multiple structured field names and the multiple keywords corresponding to each structured field name.
[0013] Secondly, an OCR-based medical material structuring device is provided, comprising:
[0014] The acquisition module is used to acquire images of the medical materials to be identified.
[0015] The recognition module is used to perform text recognition on the image of the medical material to be recognized using OCR technology, obtain multiple recognized texts, and preprocess the multiple recognized texts to obtain multiple target texts;
[0016] The sorting module is used to sort multiple target texts, extract text content from multiple target texts and concatenate them to obtain a target text set;
[0017] A classification module is used to determine the type of medical material corresponding to the target text set based on a pre-trained medical material model.
[0018] The query module is used to determine, based on a pre-built structured dictionary, multiple structured field names corresponding to the medical material types of the target text set, and multiple keywords corresponding to each structured field name;
[0019] The processing module is used to perform structured processing on the target text set based on the multiple structured field names and the multiple keywords corresponding to each structured field name.
[0020] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the OCR-based medical material structuring method as described above.
[0021] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the OCR-based medical material structuring method described above.
[0022] The beneficial effects of the technical solution provided by this invention include at least the following:
[0023] This invention obtains a target text set by acquiring images of medical materials to be identified, inputs the target text set into a pre-trained medical material model, determines the type of medical material corresponding to the target text set, and determines the names of multiple structured fields and keywords corresponding to the structured processing of the target text set through a pre-constructed structured dictionary. This can greatly eliminate the differences between medical materials in hospitals across the country, has high support for medical materials in hospitals across the country, comprehensive coverage, high efficiency in extracting primary structured medical information, and can make reasonable use of the captured medical materials. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a schematic diagram of an application environment for a medical material structuring method based on OCR in one embodiment of the present invention;
[0026] Figure 2 This is a schematic flowchart of a medical material structuring method based on OCR in one embodiment of the present invention;
[0027] Figure 3 yes Figure 1 A schematic diagram of a specific implementation method for step S60;
[0028] Figure 4 yes Figure 1 A schematic diagram of a specific implementation method for step S20;
[0029] Figure 5 yes Figure 4 A schematic diagram illustrating the preprocessing of the identified text in a specific embodiment of step S20;
[0030] Figure 6 This is a schematic diagram of a medical material structuring device based on OCR in one embodiment of the present invention;
[0031] Figure 7 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;
[0032] Figure 8 This is another structural schematic diagram of a computer device according to one embodiment of the present invention.
[0033] The accompanying drawings illustrate a specific embodiment of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0035] Specifically, the medical materials addressed in this invention include images of medical materials taken by patients themselves. These images are characterized by poor quality and diverse material types, making it difficult to directly utilize them for medical analysis and hindering the analysis and tracking of patients' health conditions. Furthermore, during the extraction of patients' medical information, some detailed information, such as medication information and examination results, is difficult to extract directly from the medical materials, presenting challenges such as extraction difficulties and low coverage.
[0036] The OCR-based medical material structuring method provided by this invention can be applied to, for example... Figure 1In the application environment shown, the client communicates with the server via a network. The server can obtain images of medical materials to be identified through the client; use OCR technology to perform text recognition on the images of the medical materials to be identified, obtaining multiple identified texts, and preprocessing the multiple identified texts to obtain multiple target texts; sort the multiple target texts, extract the text content from the multiple target texts and concatenate them to obtain a target text set; based on a pre-trained medical material model, determine the medical material type corresponding to the target text set; based on a pre-built structured dictionary, determine multiple structured field names corresponding to the medical material type corresponding to the target text set, and multiple keywords corresponding to each structured field name; and perform structured processing on the target text set according to the multiple structured field names and the multiple keywords corresponding to each structured field name. This invention obtains a target text set by acquiring images of medical materials to be identified. The target text set is then input into a pre-trained medical material model to determine the type of medical material corresponding to the target text set. A pre-built structured dictionary is used to determine the names of multiple structured fields and keywords for structured processing of the target text set. This significantly reduces the differences in medical materials among hospitals nationwide, providing high support and comprehensive coverage for medical materials from all hospitals across the country. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a dedicated server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.
[0037] Please see Figure 2 As shown, Figure 2 A flowchart illustrating the OCR-based medical material structuring method provided in this embodiment of the invention includes the following steps:
[0038] S10: Obtain an image of the medical material to be identified.
[0039] The OCR-based medical material structuring method provided by this invention can receive medical material images captured in various scenarios. The client is used to capture or receive these images, and the server obtains them through the client. For example, in online consultations, online fundraising, and online insurance scenarios, users transmit images of medical materials to be identified to the server via the client.
[0040] S20: Use OCR technology to perform text recognition on the image of the medical material to be identified, obtain multiple identified texts, and preprocess the multiple identified texts to obtain multiple target texts.
[0041] The server uses OCR technology to perform text recognition on images of medical materials to be identified, obtaining multiple recognized texts. Understandably, the types of recognized text include medical images (such as CT images, ultrasound images, etc.) and text (such as text descriptions and conclusions corresponding to the medical images), displayed in rectangular frames. Because different medical materials have different layouts, and the angle at which users photograph medical material images may be problematic—the images of the medical materials to be identified may be tilted, bent, etc.—angle correction is required for the recognized text. Specifically, for example… Figure 4 As shown, in step S20, that is, using OCR technology to perform text recognition on the image of the medical material to be identified, obtaining multiple recognized texts, and preprocessing the multiple recognized texts to obtain multiple target texts, the steps include the following:
[0042] S21: Use OCR technology to perform text recognition on the image of the medical material to be identified, and obtain multiple recognized texts.
[0043] Understandably, the recognized text is displayed in the form of a rectangle.
[0044] S22: Use OCR technology to obtain the position information and tilt angle of each of the identified texts.
[0045] OCR technology can be used to obtain the coordinates of the four vertices of the rectangle of the recognized text, as well as the tilt angle θ between the rectangle of the recognized text and the horizontal plane. Based on the coordinates of the four vertices of the rectangle of the recognized text, the center coordinates (x, y), the length of the long side w, and the length of the short side h of the rectangle of the recognized text can be calculated.
[0046] S23: Based on the position information and tilt angle of each identified text, perform position correction on each identified text.
[0047] The center coordinates (x`, y`) of the target text's rectangle can be calculated based on the center coordinates (x, y), the length w of the long side of the rectangle, the length h of the short side of the rectangle, and the tilt angle θ.
[0048] It is understood that, in another embodiment, the center coordinates (x`, y`) of the target text's bounding box can also be obtained by calculating the coordinates of the lower left corner of the bounding box of the identified text.
[0049] For steps S21-S23, after acquiring the image of the medical material to be identified, OCR technology is first used to process the image, obtaining the OCR processing result. Then, the OCR processing result is preprocessed to obtain the target text. This preprocessing includes angle correction of the bounding box of the identified text. The angle correction process utilizes the coordinates of the four vertices of the bounding box of the identified text obtained through OCR technology, as well as the tilt angle θ between the bounding box and the horizontal plane. This solves the problem of low recognition efficiency caused by tilted or bent images of the medical material to be identified, and improves the accuracy of structured processing.
[0050] S30: Sort the multiple target texts, extract the text content from the multiple target texts and concatenate them to obtain a target text set.
[0051] In step S20, multiple target texts can be identified. The types of target texts include medical images (such as CT images, ultrasound images, etc.) and text (such as text descriptions and conclusions corresponding to medical images). The target texts are displayed in the form of rectangular boxes. The center coordinates (x', y') of the rectangle of each target text are obtained. The x' of the center coordinates of the rectangles of multiple target texts are compared to complete the sequential arrangement of multiple target texts in the same row; the y' of the center coordinates of the rectangles of multiple target texts are compared to complete the sequential arrangement of multiple target texts in the same column. For example, if the center coordinates of the rectangles containing multiple target texts are (10,5), (10,8), (10,15), (15,5), (15,8), and (15,15), then after sorting the multiple target texts according to their center coordinates, and in the order of increasing from left to right in the same row and increasing from top to bottom in the same column, the final sorting of the multiple target texts is as follows: the first row contains target texts with center coordinates of (10,5) and (15,5), the second row contains target texts with center coordinates of (10,8) and (15,8), and the third row contains target texts with center coordinates of (10,15) and (15,15).
[0052] After sorting multiple target texts, the text content is extracted from the target texts and concatenated to form a target text set. Sorting the target texts ensures that the text content in the target text set has strong coherence, high readability, and high accuracy.
[0053] S40: Based on a pre-trained medical materials model, determine the type of medical materials corresponding to the target text set.
[0054] Using the target text set as input data, a BERT-based medical materials model is trained to distinguish different types of medical materials.
[0055] Before step S40, that is, before determining the medical material type corresponding to the target text set based on the pre-trained medical material model, the method includes:
[0056] We can obtain images of various types of medical materials, including admission records, discharge records, admission certificates, discharge certificates, pathology reports, and inpatient medical record cover sheets. Each type of medical material includes multiple images.
[0057] OCR technology is used to perform text recognition on each type of medical material image to be identified, obtaining multiple identified texts, and the multiple identified texts are preprocessed to obtain multiple target texts.
[0058] The specific steps are detailed in steps S21-S23. After acquiring images of each type of medical material to be identified, OCR technology is first used to process each type of image, resulting in OCR results for each type. Then, preprocessing is performed on each type of OCR result to obtain multiple target texts for each type. This preprocessing includes angle correction of the bounding box of the identified text. The angle correction process utilizes the coordinates of the four vertices of the bounding box of the identified text obtained through OCR technology, as well as the tilt angle θ between the bounding box and the horizontal plane. This solves the problem of low efficiency in structured processing caused by tilted or bent images of the medical materials to be identified.
[0059] Multiple target texts are sorted, and the text content from each target text is extracted and concatenated to obtain a target text set of each type. Specific steps are detailed in step S30: after sorting the multiple target texts, the text content is extracted and concatenated to form a target text set. Sorting the target texts ensures strong coherence and readability of the text content within the target text set.
[0060] Each type of target text set is input into the medical materials model for training.
[0061] Using each type of target text set as input data, a BERT-based medical materials model is trained to distinguish between different types of medical materials.
[0062] S50: Based on a pre-built structured dictionary, determine the names of multiple structured fields corresponding to the medical material types of the target text set, and the multiple keywords corresponding to each structured field name.
[0063] By identifying the structured field names contained in the medical material types corresponding to the target text set, and the keywords contained in the structured field names, a structural basis is provided for subsequent structuring processing of the target text set.
[0064] Specifically, before step S50, that is, before determining the multiple structured field names corresponding to the medical material type of the target text set based on the pre-built structured dictionary, and the multiple keywords corresponding to each structured field name, the method includes:
[0065] We obtain various types of medical materials. These include admission records, discharge records, admission certificates, discharge certificates, pathology reports, and inpatient medical record summary sheets. Each type of medical material includes multiple images.
[0066] Each type of medical material is pre-defined with multiple structured field names and multiple keywords corresponding to each structured field name. Due to significant differences between various types of medical materials across hospitals, to improve the generalization of structured processing, the structured field names and keywords for each type of medical material can be set according to the type of medical material.
[0067] A structured dictionary is constructed based on multiple structured field names corresponding to each type of medical material and multiple keywords corresponding to each structured field name. This enables step S50 to determine multiple structured field names corresponding to the target text set and multiple keywords corresponding to each structured field name based on the type of medical material.
[0068] For example, when constructing a structured dictionary, the preset medical material types include: admission record, discharge record, admission certificate, discharge certificate, pathology report, and inpatient medical record cover sheet. When the preset medical material type is admission record, the corresponding preset structured field names include admission date, chief complaint, present illness, personal history, admission status, and treatment process. The preset keywords for the structured field name "admission status" include brief admission medical history, admission condition, patient admission status, main symptoms and signs upon admission, and main symptoms and signs in the admission medical record summary. The preset keywords for the structured field name "treatment process" include inpatient treatment summary, treatment process during admission, treatment during hospitalization, brief process of inpatient treatment and main treatment measures, and main treatment measures.
[0069] The specific step S50, based on a pre-built structured dictionary, determines multiple structured field names corresponding to the medical material type of the target text set, and multiple keywords corresponding to each structured field name, including:
[0070] The target text set is used to determine the type of medical materials corresponding to the target text set, including admission records.
[0071] The names of the multiple structured fields corresponding to the admission record are determined, including admission date, chief complaint, present illness history, admission status, etc.
[0072] Keywords corresponding to the admission information include brief medical history upon admission, condition upon admission, and patient admission status.
[0073] In this invention, the construction of keywords and structured field names, as well as the keyword search query of the target text set, make the process controllable, improve the generalization of the structured processing process, and facilitate its expansion to various types of medical materials.
[0074] S60: The target text set is structured according to the multiple structured field names and the multiple keywords corresponding to each structured field name.
[0075] After determining the structured field names and corresponding keywords for the target text set, the structured processing of the target text set can be achieved by extracting the text content from the target text set and mapping the extracted text content to the corresponding keywords and structured field names. The process is simple and easy to operate.
[0076] like Figure 3 As shown, step S60 specifically includes:
[0077] S61: Search for multiple keywords in the target text set. For example, after determining that the medical material type corresponding to the target text set is "admission record," the names of multiple structured fields corresponding to "admission record" are determined to include "admission date," "chief complaint," "present illness," and "admission status," etc., among which the keywords corresponding to "admission status" are determined to include "brief admission history," "condition under observation," and "patient admission status," etc. Then, search for keywords such as "brief admission history," "condition under observation," and "patient admission status" in the target text set.
[0078] S62: Obtain the location information of multiple keywords. For example, keywords such as "brief medical history upon admission", "condition upon observation", and "patient admission status" were found in the target text set, and the location information of keywords such as "brief medical history upon admission", "condition upon observation", and "patient admission status" was obtained.
[0079] S63: Determine the positional information of two adjacent keywords. For example, the keywords "brief medical history upon admission" and "condition upon observation" are determined to be two adjacent keywords, and the positional information of the keywords "brief medical history upon admission" and "condition upon observation" is obtained.
[0080] S64: Determine the structured text of the keyword with a relatively earlier position among two adjacent keywords according to the position information of the two adjacent keywords. For example, if the keyword "Brief medical history at admission" has a relatively earlier position and the keyword "Condition on admission to the observation unit" has a relatively later position, then extract the text content between "Brief medical history at admission" and "Condition on admission to the observation unit", and assign the extracted text content to the keyword "Brief medical history at admission".
[0081] S65: Determine the name of the structured field corresponding to the keyword corresponding to the structured text. For example, through the keyword "Brief medical history at admission", determine the name of the structured field "Condition at admission" corresponding to the extracted text content. The process of determining the name of the structured field by tracing back the keyword is the completion of the structuring process.
[0082] Specifically, in one embodiment, obtaining the position information of multiple keywords in step S62 includes obtaining the start position information and end position information of each keyword. It can be understood that, as a whole, the start position information and end position information are character positions for the target text set. For example, if the content of the target text set is "Brief medical history at admission. When admitted, the patient was conscious, with a poor complexion, shortness of breath, moist rales heard in the lungs, and cyanosis of the lips. Condition on admission to the observation unit", after searching, the keywords are determined to be "Brief medical history at admission" and "Condition on admission to the observation unit". The position information of the keyword "Brief medical history at admission" is (1, 7), and the position information of the keyword "Condition on admission to the observation unit" is (40, 44).
[0083] Specifically, in one embodiment, determining the position information of two adjacent keywords in step S63 includes determining the start position information of the i-th keyword, the end position information of the i-th keyword, the start position information of the (i + 1)-th keyword, and the end position information of the (i + 1)-th keyword. For example, the keywords "Brief medical history at admission" and "Condition on admission to the observation unit" are two adjacent keywords. Determine that the start position information of the i-th keyword "Brief medical history at admission" is the cursor position before the character "入" and is denoted as 1, the end position information of the i-th keyword "Brief medical history at admission" is the cursor position after the character "史" and is denoted as 7, the start position information of the (i + 1)-th keyword "Condition on admission to the observation unit" is the cursor position before the character "入" and is denoted as 40, and the end position information of the (i + 1)-th keyword "Condition on admission to the observation unit" is the cursor position after the character "情" and is denoted as 44.
[0084] Specifically, in one embodiment, step S64 determines the structured text of the keyword that appears earlier in the sequence of two adjacent keywords based on their positional information. This includes: obtaining the text content between the starting position information of the i-th keyword and the starting position information of the (i+1)-th keyword, and determining that the text content is the structured text of the i-th keyword. For example, determining the text content between the starting position information of the i-th keyword "brief medical history upon admission" and the starting position information of the (i+1)-th keyword "condition upon admission" as "Brief medical history upon admission: The patient was conscious, had a poor complexion, rapid breathing, moist rales heard in the lungs, and cyanosis of the lips.", and determining the text content of "Brief medical history upon admission: The patient was conscious, had a poor complexion, rapid breathing, moist rales heard in the lungs, and cyanosis of the lips." as the structured text of the keyword "brief medical history upon admission".
[0085] This invention pre-constructs a structured dictionary based on various types of medical materials, which improves the generalizability of medical material recognition during structured processing. It has a strong ability to recognize different medical materials from different hospitals and greatly eliminates the differences between medical materials in hospitals across the country.
[0086] This invention obtains a target text set by acquiring images of medical materials to be identified, inputs the target text set into a pre-trained medical material model, determines the type of medical material corresponding to the target text set, and determines the names of multiple structured fields and keywords corresponding to the structured processing of the target text set through a pre-constructed structured dictionary. This can greatly eliminate the differences between medical materials in hospitals across the country, has high support for medical materials in hospitals across the country, comprehensive coverage, high efficiency in extracting primary structured medical information, and can make reasonable use of the captured medical materials.
[0087] This invention proposes an end-to-end, fully structured workflow for capturing complex medical material images. The workflow is reliable and the final structured effect is excellent.
[0088] This invention utilizes OCR technology, deep learning technology based on BERT medical material models, and text structuring technology to perform structured processing of medical information from captured medical material images. This provides fundamental data services for subsequent data analysis, patient health tracking, and detailed medical information extraction, expanding the application scenarios of image-based medical materials and providing data support.
[0089] In one embodiment, an OCR-based medical material structuring device is provided, which corresponds one-to-one with the OCR-based medical material structuring method described in the above embodiments. For example... Figure 6As shown, this OCR-based medical material structuring device includes an acquisition module, an identification module, a sorting module, a classification module, a query module, and a processing module. Detailed descriptions of each functional module are as follows:
[0090] The acquisition module is used to acquire images of the medical materials to be identified.
[0091] The recognition module is used to perform text recognition on the image of the medical material to be recognized using OCR technology, obtain multiple recognized texts, and preprocess the multiple recognized texts to obtain multiple target texts;
[0092] The sorting module is used to sort multiple target texts, extract text content from multiple target texts and concatenate them to obtain a target text set;
[0093] A classification module is used to determine the type of medical material corresponding to the target text set based on a pre-trained medical material model.
[0094] The query module is used to determine, based on a pre-built structured dictionary, multiple structured field names corresponding to the medical material types of the target text set, and multiple keywords corresponding to each structured field name;
[0095] The processing module is used to perform structured processing on the target text set based on the multiple structured field names and the multiple keywords corresponding to each structured field name.
[0096] In some embodiments, the acquisition module is specifically used to: acquire images of medical materials via a client. For example, in online consultation scenarios, online fundraising scenarios, and online insurance scenarios, users transmit images of medical materials to be identified to the server via a client.
[0097] In some embodiments, the identification module is specifically used for:
[0098] OCR technology is used to perform text recognition on the image of the medical material to be identified, and multiple recognized texts are obtained.
[0099] The positional information and tilt angle of each identified text are obtained using OCR technology;
[0100] Based on the position information and tilt angle of each identified text, position correction is performed on each identified text.
[0101] In some embodiments, the identification module is specifically used for:
[0102] OCR technology can be used to obtain the coordinates of the four vertices of the rectangle of the recognized text, as well as the tilt angle θ between the rectangle of the recognized text and the horizontal plane. Based on the coordinates of the four vertices of the rectangle of the recognized text, the center coordinates (x, y), the length of the long side w, and the length of the short side h of the rectangle of the recognized text can be calculated.
[0103] The center coordinates (x`, y`) of the target text's rectangle can be calculated based on the center coordinates (x, y), the length w of the long side of the rectangle, the length h of the short side of the rectangle, and the tilt angle θ.
[0104] Combination Figure 5 As shown, the formula for calculating the center coordinates (x`, y`) of the target text's bounding box is as follows:
[0105]
[0106]
[0107] Where (x, y) are the center coordinates of the rectangle used to identify the text;
[0108] w represents the length of the longer side of the rectangle used to identify the text;
[0109] h represents the length of the shorter side of the rectangle used to identify the text;
[0110] θ is the tilt angle.
[0111] After acquiring the image of the medical material to be identified, OCR technology is first used to process the image, obtaining the OCR result. Then, the OCR result is preprocessed to obtain the target text. This method solves the problem of low recognition efficiency caused by tilted or bent images of the medical materials, and improves the accuracy of structured processing.
[0112] In some embodiments, the sorting module is specifically used for:
[0113] Get the center coordinates (x`, y`) of the rectangle containing each target text.
[0114] Compare the x' of the center coordinates of the rectangles containing multiple target texts to arrange them sequentially on the same line.
[0115] Compare the y' of the center coordinates of the rectangles containing multiple target texts to arrange them sequentially in the same column;
[0116] Among them, multiple target texts are sorted in order of increasing size from left to right in the same row and from top to bottom in the same column.
[0117] After sorting multiple target texts, the text content is extracted from the target texts and concatenated to form a target text set. Sorting the target texts ensures that the text content in the target text set has strong coherence, high readability, and high accuracy.
[0118] In some embodiments, the classification module is specifically used for:
[0119] Input the target text set into a pre-trained medical materials model;
[0120] Determine the type of medical material corresponding to the target text set.
[0121] In some embodiments, the query module is specifically used for:
[0122] Input the medical material types corresponding to the target text set into a pre-constructed structured dictionary;
[0123] Determine the names of multiple structured fields corresponding to the medical material type;
[0124] Identify multiple keywords corresponding to each structured field name.
[0125] In some embodiments, the processing module is specifically used for:
[0126] Search the target text set for multiple of the keywords;
[0127] Obtain the location information of multiple keywords;
[0128] Determine the positional information of two adjacent keywords;
[0129] Based on the positional information of two adjacent keywords, determine the structured text of the keyword that appears earlier in the two adjacent keywords;
[0130] Determine the name of the structured field corresponding to the keyword in the structured text.
[0131] In some embodiments, the processing module is specifically used for:
[0132] Obtain the start and end position information for each keyword;
[0133] Determine the starting position information, ending position information, starting position information, and ending position information of the (i+1)th keyword;
[0134] Obtain the text content between the starting position information of the i-th keyword and the starting position information of the (i+1)-th keyword, and determine that the text content is the structured text of the keyword that appears earlier in the two adjacent keywords.
[0135] The OCR-based medical material structuring device provided by this invention obtains a target text set by acquiring images of medical materials to be identified. The target text set is then input into a pre-trained medical material model to determine the type of medical material corresponding to the target text set. Through a pre-constructed structured dictionary, the names of multiple structured fields and keywords corresponding to the structured processing of the target text set are determined. This device can greatly eliminate the differences between medical materials from hospitals across the country, has high support for medical materials from hospitals nationwide, provides comprehensive coverage, and has high efficiency in extracting primary structured medical information. It can also make reasonable use of the captured medical materials.
[0136] Specific limitations regarding the OCR-based medical material structuring device can be found in the above description of the limitations of the OCR-based medical material structuring method, and will not be repeated here. Each module in the aforementioned OCR-based medical material structuring device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.
[0137] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a server-side method for structuring medical materials based on OCR.
[0138] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a client-side method for OCR-based medical material structuring processing.
[0139] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:
[0140] Obtain images of the medical materials to be identified;
[0141] The text recognition of the medical material image to be identified is performed using OCR technology to obtain multiple identified texts, and the multiple identified texts are preprocessed to obtain multiple target texts;
[0142] Sort the multiple target texts, extract the text content from the multiple target texts and concatenate them to obtain a target text set;
[0143] Based on a pre-trained medical materials model, the types of medical materials corresponding to the target text set are determined;
[0144] Based on a pre-built structured dictionary, the names of multiple structured fields corresponding to the medical material types of the target text set are determined, as well as multiple keywords corresponding to each structured field name;
[0145] The target text set is structured based on the multiple structured field names and the multiple keywords corresponding to each structured field name.
[0146] In one embodiment, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps:
[0147] Obtain images of the medical materials to be identified;
[0148] The text recognition of the medical material image to be identified is performed using OCR technology to obtain multiple identified texts, and the multiple identified texts are preprocessed to obtain multiple target texts;
[0149] Sort the multiple target texts, extract the text content from the multiple target texts and concatenate them to obtain a target text set;
[0150] Based on a pre-trained medical materials model, the types of medical materials corresponding to the target text set are determined;
[0151] Based on a pre-built structured dictionary, the names of multiple structured fields corresponding to the medical material types of the target text set are determined, as well as multiple keywords corresponding to each structured field name;
[0152] The target text set is structured based on the multiple structured field names and the multiple keywords corresponding to each structured field name.
[0153] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0154] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0155] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0156] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method for processing medical material structuring based on OCR, characterized in that, include: Acquire images of medical materials to be identified, which are images of different medical materials from various hospitals; The text recognition of the medical material image to be identified is performed using OCR technology to obtain multiple identified texts, and the multiple identified texts are preprocessed to obtain multiple target texts; Sort the multiple target texts, extract the text content from the multiple target texts and concatenate them to obtain a target text set, obtain the center coordinates (x`, y`) of the rectangle of each target text, compare the x` of the center coordinates of the rectangles of the multiple target texts to complete the sequential arrangement of the multiple target texts in the same row; compare the y` of the center coordinates of the rectangles of the multiple target texts to complete the sequential arrangement of the multiple target texts in the same column; Based on a pre-trained medical materials model, the types of medical materials corresponding to the target text set are determined; Based on a pre-built structured dictionary, the names of multiple structured fields corresponding to the medical material types of the target text set are determined, as well as multiple keywords corresponding to each structured field name; The target text set is structured based on the multiple structured field names and the multiple keywords corresponding to each structured field name; The text recognition of the medical material image to be identified is performed using OCR technology to obtain multiple identified texts, and the multiple identified texts are preprocessed to obtain multiple target texts, including: OCR technology is used to perform text recognition on the image of the medical material to be identified, and multiple recognized texts are obtained. The position information and tilt angle of each identified text are obtained using OCR technology. The position information of the identified text includes the coordinates of the four vertices of the rectangle of the identified text, and the tilt angle of the identified text is the tilt angle between the rectangle of the identified text and the horizontal plane. Based on the position information and tilt angle of each identified text, position correction is performed on each identified text. The step of structuring the target text based on multiple structured field names and multiple keywords corresponding to each structured field name includes: Search the target text set for multiple of the keywords; Obtaining the location information of multiple keywords, wherein obtaining the location information of multiple keywords includes obtaining the start location information and end location information of each keyword; Determine the position information of two adjacent keywords, which includes determining the starting position information of the i-th keyword, the ending position information of the i-th keyword, the starting position information of the (i+1)-th keyword, and the ending position information of the (i+1)-th keyword. Based on the position information of two adjacent keywords, determine the structured text of the keyword that appears earlier in the two adjacent keywords. The step of determining the structured text of the keyword that appears earlier in the two adjacent keywords includes: obtaining the text content between the starting position information of the i-th keyword and the starting position information of the (i+1)-th keyword, and determining the text content as the structured text of the i-th keyword. Determine the name of the structured field corresponding to the keyword in the structured text.
2. The OCR-based medical material structuring method according to claim 1, characterized in that, Before determining the names of multiple structured fields corresponding to the medical material types of the target text set, and the multiple keywords corresponding to each structured field name, based on a pre-built structured dictionary, the method includes: Obtain various types of medical materials; Each type of medical material is pre-defined with multiple structured field names and multiple keywords corresponding to each structured field name; A structured dictionary is constructed based on multiple structured field names corresponding to each type of medical material and multiple keywords corresponding to each structured field name.
3. The OCR-based medical material structuring method according to claim 1, characterized in that, The method, based on a pre-built structured dictionary, determines multiple structured field names corresponding to the medical material types in the target text set, and multiple keywords corresponding to each structured field name, including: The medical material types corresponding to the target text set include admission records; The names of the multiple structured fields corresponding to the admission record are determined, including admission date, chief complaint, present illness, admission status, etc. Keywords corresponding to the admission information include brief medical history upon admission, condition upon admission, and patient admission status.
4. The OCR-based medical material structuring method according to claim 1, characterized in that, Before determining the medical material type corresponding to the target text set based on the pre-trained medical material model, the method includes: Obtain images of various types of medical materials; OCR technology is used to perform text recognition on each type of medical material image to be identified, obtaining multiple recognized texts, and the multiple recognized texts are preprocessed to obtain multiple target texts; Sort the multiple target texts, extract the text content from the multiple target texts and concatenate them to obtain a target text set of each type; Each type of target text set is input into the medical materials model for training.
5. A medical material structuring device based on OCR, characterized in that, include: The acquisition module is used to acquire images of medical materials to be identified, which are images of different medical materials from various hospitals; The recognition module is used to perform text recognition on the image of the medical material to be recognized using OCR technology, obtain multiple recognized texts, and preprocess the multiple recognized texts to obtain multiple target texts; The sorting module is used to sort multiple target texts, extract text content from multiple target texts and concatenate them to obtain a target text set, obtain the center coordinates (x`, y`) of the rectangle of each target text, compare the x` of the center coordinates of the rectangles of multiple target texts to complete the sequential arrangement of multiple target texts in the same row; compare the y` of the center coordinates of the rectangles of multiple target texts to complete the sequential arrangement of multiple target texts in the same column; A classification module is used to determine the type of medical material corresponding to the target text set based on a pre-trained medical material model. The query module is used to determine, based on a pre-built structured dictionary, multiple structured field names corresponding to the medical material types of the target text set, and multiple keywords corresponding to each structured field name; The processing module is used to perform structured processing on the target text set based on the multiple structured field names and the multiple keywords corresponding to each structured field name; The recognition module is also used to perform text recognition on the image of the medical material to be recognized using OCR technology to obtain multiple recognized texts; to obtain the position information and tilt angle of each recognized text using OCR technology, wherein the position information of the recognized text includes the coordinates of the four vertices of the rectangle of the recognized text, and the tilt angle of the recognized text is the tilt angle between the rectangle of the recognized text and the horizontal plane; and to perform position correction on each recognized text according to the position information and tilt angle of each recognized text. The processing module is also configured to search for multiple keywords in the target text set; obtain the location information of the multiple keywords, wherein obtaining the location information of the multiple keywords includes obtaining the start location information and end location information of each keyword; Determine the position information of two adjacent keywords, which includes determining the start position information, end position information, start position information, and end position information of the (i+1)th keyword; and determine the structured text of the keyword that appears earlier in the two adjacent keywords based on their position information, which includes obtaining the text content between the start position information of the i-th keyword and the start position information of the (i+1)th keyword, and determining that the text content is the structured text of the i-th keyword. Determine the name of the structured field corresponding to the keyword in the structured text.
6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the OCR-based medical material structuring method as described in any one of claims 1 to 4.
7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the OCR-based medical material structuring method as described in any one of claims 1 to 4.